Joe Amenta's Blog

August 11, 2010

Debugging on brain

Filed under: 3to2,Google Summer of Code,GSoC 2010,Work — AirBreather @ 5:55 pm

I’ve spent the better part of today and some of Monday and Tuesday hunting down why so many tests in brain are failing.  While I haven’t completely followed the tracebacks of every single error or failure, I can say that for those that I have, the problem does not appear to be caused by lib3to2, but rather by what I can only explain as differences in the sqlite3 module between the Python3.1 version and the Python2.7 versions.  This suggests that there might still be some work to do with lib3to2.

However, I really don’t see a possible lib3to2-based solution for this.  I wouldn’t even know where to start with this, since sqlite3 is just a thin wrapper around _sqlite3, a C extension module included with Python.  I’m going to tentatively call this one finished, after I fix a bug involving print statements and parentheses.  It shows that even though lib3to2 is pretty good at syntax refactoring, there are some differences (often very subtle) between Python3 and Python2 that lib3to2 will fail to catch and will need manual refactoring (and it’s thanks to brain’s excellent test suite that I found those in the first place!).  Furthermore, lib3to2 is best used as an aid throughout the entire development process, as it will catch incompatibility problems early, before too much code starts building on incompatible code.

I’m also running 3to2’d code with Python 2.7, so there may be some additional problems with Python 2.6 and earlier.


  1. Hi ! I’m currently using your 3to2 script for a current project, this is helping me very much… I need python2.7 to use pyAMF which is not already ported to python3.

    Is there any way to NOT “translate” some parts of my code ? I’m using the struct module, and it doesn’t like the string to unicode translation (‘…’ to u’…’ I mean), because it needs real ascii string.

    I don’t have much time to read your code and understand it, because I’m already crossing the deadline of my work, so I will correct by hand the wrong parts for the moment.

    It gives me:
    ushort = Struct(u'<H')
    TypeError: Struct() argument 1 must be string, not unicode

    Thx for your work, and sorry for my english

    PS: I cooked a cludgy module to make my code work under python2 with 2to3, to use it I add the line "from compat import *" at the beginning of my modules. Here it is:

    # Compatibility for Python 2 & 3
    # open (compat_open) and re.compile (compat_regexp)

    import sys
    import re

    if sys.version_info[0] == 3:

    import builtins

    __builtin__ = builtins

    def open(filename, mode='r'):
    return, mode, encoding='utf-8', newline='')

    def re_compile(pat, flags=None):
    if flags is not None: return re._compileorig(pat, flags)
    else: return re._compileorig(pat)


    from future_builtins import *
    import codecs
    import __builtin__

    def open(filename, mode='r'):
    return, mode, encoding='utf-8')

    def re_compile(pat, flags=0):
    return re._compileorig(pat, flags|re.UNICODE)

    #__builtin__.__open_orig = = _compat_open

    re._compileorig = re.compile
    re.compile = re_compile

    Comment by anondev — August 15, 2010 @ 11:58 am

  2. I forgot to say: i’m using python2.7

    Comment by anondev — August 15, 2010 @ 12:01 pm

  3. I found! I had to rewrite (in my py3k code) ‘someasciistring’ to b’someasciistring’ and someunicodevar to str(someunicodevar).


    Comment by anondev — August 15, 2010 @ 12:14 pm

RSS feed for comments on this post. TrackBack URL

Leave a comment

Powered by WordPress