Joe Amenta's Blog

June 5, 2009

Milestone: print function

Filed under: 3to2,GSoC 2009,Google Summer of Code,Work — Tags: , , — AirBreather @ 2:47 am

I’m pretty pumped.  I just “finished” the print function fixer for 3to2.  I put “finished” in quotes (and only “working” instead of “complete” under its status text on the previous post) because it’s a sort of a placeholder right now: all it does is add “from __future__ import print_function” at the top of a refactored module that uses print(), so this will only successfully give you code certified for 2.6 and up, not the 2.5 branch as originally promised.  But still, this is a huge leap: now, most (if not all) of my own little py3k modules are working unmodified after running them through 3to2!  This is a good time for me to stop and explain how to get 3to2 working on your computer for those of you that do not know:

Find an SVN client (TortoiseSVN for Windows, subversion for others) , and check out (into separate folders) both http://svn.python.org/projects/sandbox/trunk/2to3 and http://svn.python.org/projects/python/trunk/.  Then, copy the folder (the entire folder!) lib2to3 (from the root of the 2to3 folder) into the Lib/ folder (from the root of the other folder).  Then, look up a great tutorial for how to build Python on your system (sorry, I’m not here to tell you how to do that… just how to get my stuff to work).

Once you’ve built your own custom Python build (with the most up-to-date version of lib2to3, that this project relies on [and I contributed a line or two of code here and there to make it work for my purposes]), then it’s time to get 3to2.  Find a Mercurial client (TortoiseHG for Windows, mercurial for others), and clone (into yet another separate folder) http://bitbucket.org/amentajo/lib3to2/.  Run “python2.7 setup.py build” (python2.7 being the path to your python2.7 executable you built earlier on), and then “python2.7 setup.py install” (you need administrator / root privileges for the latter command).  This will put “3to2″ on your sys path, and you will be able to simply run 3to2 like this:

3to2 /path/to/my_file.py # obviously, /path/to/my_file.py stands for a (relative or absolute) path to an existing py3k file that you need to refactor. (This will also work on an entire directory that contains .py files)

This will show you (in unified diff format) the changes that 3to2 will make to your file.  You can either redirect the stdout to a .patch file then apply that patch the normal way, or run 3to2 like this:

3to2 -w /path/to/my_file.py

with the -w command, and it will back up your original files, then fix them with the fixer suite.

One note: there is one fixer so far that must be run explicitly.  That is memoryview, which changes all memoryview() calls into buffer() calls.  To enable this, you must run 3to2 with the “-f memoryview” option.  There are two reasons for this.  The first is that buffer objects do not support all of the same methods that memoryview objects do, so if this fixer were enabled by default, it would fix code perfectly for some cases, and not for others.  The second reason is that memoryview was backported into python 2.7, so if the user wishes to run their fixed code on a python2.7 platform, they can benefit from keeping their memoryview() calls alone.

(Of course, if you’re using memoryview() or buffer(), you probably already know or can easily figure out that you can use -xmemoryview to turn that fixer off ;-) )

Having reached this milestone, I look both back and ahead: it feels like I’m almost all the way to a finished product.  What’s left as a high priority at this point is exception stuff, various import stuff, dict methods, the stdlib, magical metaclass mayhem, sys.intern, numliterals (0o0777, 0b011101, et al.), reduce, and set literals… plus with_statements, annotations, floor division, and some other stuff I’m too tired to think of, but was added in 3.0 that had no 2to3 fix.  Then, fix the print fixer to work for 2.5, i.e., actually perform logic and move stuff out of the function into a true print statement.  After that, I want to rework the stuff that was changed from lists into iterators in py3k to remove a direct list() call, so that list(range(…)) will (more) correctly be fixed into range(…) rather than list(xrange(…)), e.g.  After that’s done, I feel like more robust test suites are in order (for both 3to2 and 2to3).

OK, writing it all down like that in list form makes it look longer than it really is.  This doesn’t seem like it will take until the end of the Google Summer of Code program, at least not to me.  The stdlib stuff will take a good week or so, most of which will involve me learning what the modules do so that I can have a grasp on how they will be used and how they changed from 2.x to 3.x.  The print function -> statement is probably two or three days.  Exception stuff will probably take me another week, for the same reason that the stdlib stuff will: I have never used any more extended form of exception handling than “raise Exception()”, and “except Exception:”… so I don’t have much experience to go on.

So it looks like I just may get to the “Things to do if I finish early” part of my proposal.  That is, assuming I keep up the pace. :-)

No Comments »

No comments yet.

RSS feed for comments on this post. TrackBack URL

Leave a comment

Powered by WordPress