Joe Amenta's Blog

August 8, 2009

Things 3to2 does not accept

Filed under: 3to2,GSoC 2009,Google Summer of Code,Work — Tags: , , — AirBreather @ 9:50 pm

It’s time now for “what not to do if you want 3to2 to work”.  The following is a list of things to avoid if you want 3to2 to work on your code (I’ll try to keep this up-to-date):

(Pay particular attention to imports if you import names referenced in PEP 3108)

  • Incorrect syntax — If a Py3k interpreter fails on part of your code, don’t expect 3to2 to know what to do.  Much of 3to2 is already picky enough without running it on things that simply don’t work in the first place.
  • Third-party module references — If it doesn’t ship with Python, 3to2 won’t touch it.  The intention is for 3to2 to do an adequate job automatically refactoring those modules so that the distributor will ship a 2.x version more easily.
  • some_name = print; some_name(some_stuff) — Though technically this works for now, when the print function syntax is actually converted, it will not work.
  • import [metamod] (For example, import urllib.request) — A work in progress, name-imports of modules made up of multiple 2.x modules merged into one  in the 2->3 gap will not work.  Use (from metamod import [names]) for now. The support for this one is getting better, but the bugs are still being worked out.  Currently, you are golden if you only import at the top-level namespace of a module and always use at least one name from each module you import in this way.  Also note that imports of this type that are never referenced with an attribute later are automatically deleted.  Another “bug” with this is that if a module is imported, then an attribute of that module is referenced for which there never was a 2.x version, the name of that module after 3to2 becomes None without warning.  This is visible at least in urllib.parse, which has some bytes references.
  • import [metamod] as [name] — This will probably not work for a while, as the code is independently scanned for references to “metamods”, so please avoid using this form of imports.
  • from [meta] import [mod] as [some_name] (For example, from dbm import gnu as zombie) — The underlying 2to3 code for (from meta import mod) disallows matching (as some_name), possible fix in the future.  For now, refactor your code not to have that “as some_name”
  • Any feature from the Py3k branch that has no syntactical equivalent in Python 2.x — 3to2 is here to refactor your syntax, not add features to Python 2.x.  Warnings are in place for if your syntax is only valid in 2.6 or greater.
  • Docstrings at the beginning of modules — If a future_stmt needs to be added, it is added before the docstring.  This will be fixed, but be wary of it for now.
  • Encodings that do not work for 2to3 — I’m not sure what these encodings are, but if you are having problems with it, please use chars valid in ascii only for now, until I can figure out if this is something that can be fixed.  If anyone has any information that might be helpful, please comment or e-mail.  More information on this: Even though py3k is more Unicode-friendly, 3to2 runs in python 2.x, so expect errors if you use an encoding that didn’t work in python 2.x.  Even a character in a comment is not safe from crashing the parser.  (See py3k/Lib/getopt.py)
  • import [module] as [something_else] — This will probably never be fixed; 2to3 imposes this rule, so I will do the same for 3to2. (This is incorrect, please ignore.)
  • from [metamod] import * (For example, from urllib.request import *) — Soon to be working, 3to2 will not succeed on imports of this type from a module that is made up of multiple 2.x modules merged into one.  Use (from [metamod] import [names]) for now. This syntax will work now, but be careful about name conflicts related to having all the extra names in the namespace from importing * from all modules that the “metamod” depends on.
  • from [module] import (names, more names, more names, some more names) — Status unknown, this failed for me.  Use the line-continuation character \ at the end of lines of this type. This bug has been fixed.

Also, note that the status of fix_imports is “incomplete”.

4 Comments »

  1. I don’t understand your second point:

    $ 2to3 -
    RefactoringTool: Skipping implicit fixer: buffer
    RefactoringTool: Skipping implicit fixer: idioms
    RefactoringTool: Skipping implicit fixer: set_literal
    RefactoringTool: Skipping implicit fixer: ws_comma
    import copy_reg as x
    RefactoringTool: Refactored
    — (original)
    +++ (refactored)
    @@ -1,1 +1,1 @@
    -import copy_reg as x
    +import copyreg as x
    RefactoringTool: Files that need to be modified:
    RefactoringTool:

    Comment by Benjamin Peterson — August 10, 2009 @ 11:29 pm

  2. Whoops – you’re right. I meant to reference this general function / method call binding limitation in 2to3:

    m = d.has_key
    if m(5): …

    but this doesn’t apply to imports. Fixing that point now to reflect this. Thanks for pointing this out, Benjamin!

    Comment by AirBreather — August 11, 2009 @ 1:17 am

  3. [...] – again, the project page is here, and Joe’s notes on things 3to2 does not accept is here. blog comments powered by Disqus var disqus_url = [...]

    Pingback by jessenoller.com - Python 3to2: Go check it out. — August 27, 2009 @ 9:26 pm

  4. [...] the process of moving all guidelines for “Things 3to2 does not accept” (from the post of the same name and from stuff I have accidentally left in my head) to the bitbucket issues tracker.  Some will be [...]

    Pingback by bitbucket issues tracker « AirBreather’s Blog — October 21, 2009 @ 12:50 am

RSS feed for comments on this post. TrackBack URL

Leave a comment

Powered by WordPress