I’ve started to use the Python 3 standard library (and tests) for testing lib3to2. I like this idea, because future developments come in the Python 3 branch only (they’re backported separately).
I’ve found a few bugs and omissions this way, and I think I’ll find a good few more as I go on.
I ran out of ADD drugs (Concerta) late last week. Thanks to difficulties with insurance, I’m still out. I should have some more today, but there’s no certainty.
The last time I was able to spend most of a day working on lib3to2 was the last time I had drugs… it’s really frustrating me that I can’t sit for more than a few minutes at a time and work. I’m trying, I really am!
A solution to the class decorator problem is in the works. I’d commit what I have right now, but it’s in a halfway state that currently breaks any functionality it had, not to mention the fact that it errors out while running tests.
If I don’t have more ADD drugs by tomorrow, I don’t know what I’m going to do to keep my mind steady.
So I start with the very first pure Python 3 package I find on PyPI (stagger). 3to2 works mostly fine, except for the class decorations used in id3.py: it doesn’t add a newline to the end of the assignment statement that implements the decorations in Python 2 syntax. ”OK”, I thought, “I’ll make a bug report and add a few more test cases to the test suite for fix_classdecorator.” Except there was no test suite for fix_classdecorator.
The initial bug is simple: I forgot to (and/or simply didn’t know that I should) wrap the assignment statement in a simple_stmt with a newline afterwards. But after adding a few test cases (several, actually), it turns out that the fixer wasn’t really doing the language feature much justice: it indiscriminately threw out anything in parentheses (and the parentheses themselves) after the name in the @decorator.
*sigh*
I’m going to resolve my phone problem today and then start hunting for code for 3to2 tomorrow. Sorry about the big delay period, I’ve just been trying to keep my mind off of my issues.
So, I lost my Nexus One about a week ago, and I’ve been stressing out about it really bad. After tearing apart my house only to find out that it’s not here, I just have to assume that it’s gone. I haven’t been able to sleep right or think straight since.
I only had it for less than a month…
I think I’ve been a little too cautious about version naming. The “beta” tag is rather unnecessary at this point. I have rolled in a couple of changes and released the first non-alpha, non-beta version.
I wanted to do this before I started with bug hunting to emphasize the fact that lib3to2 is usable, though incomplete.
Today, I start hunting down native Python 3 code to test out lib3to2. It’s quite a bit earlier than the midterm evaluation, which was my target date to start this, but I don’t think that anything else I could do in the meantime would have a better impact than this.
For the next week, I will be searching for Python 3 code, running 3to2 on it and testing both versions to try and find as many lib3to2 bugs as possible. Unless the fix is obvious and quick, each bug will go into the tracker and a test case will be written. This accomplishes a few things:
As an aside, I consider anything not fixed that emits a proper, descriptive message not to be a lib3to2 bug. If a fix would be possible within the framework of the lib2to3 API, then it could be considered for an “enhancement” entry on the tracker. I’ve probably written that before, but it is relevant here again.
I’ll start with the Python 3 standard library, then work my way to PyPI, then I’ll do some Google Code Searching.
I am proud to announce the first beta version of lib3to2. Links:
3to2 for 2.7 (pypi)
3to2 for 3.1 (pypi)
Features of this release:
The issue tracker can be found on bitbucket. Please post any bugs that you find with this release.
This week hasn’t been a great week as far as coding hours go; I’ve just been obnoxiously distracted. From the compulsion to get Brawl Minus working on Dolphin to playing with my new Nexus One, I haven’t worked the amount of hours that I should be. I’ll try to do better next week.
In spite of that, I did manage to reorganize the test cases in fix_imports2 and even get some of them to pass. I also optimized the pattern to match Python code that needs fixing; this results in a huge speed increase when running fix_imports2, as evidenced when running the tests.
Specifically, fix_imports2 now fixes cases of ”from spam.ham import eggs” (no “as bacon” or multiple comma-separated modules yet) and “from spam.ham import *”. The framework is already there to extend this to the rest of the “from spam.ham import …” cases, which will possibly end up being done today, or maybe not. At this point, it looks like very reasonable to expect “from spam.ham import …” imports to be done by the end of next week.
“import spam.ham” variants along with usage will be the next subtask after that. I think that, from a pragmatic standpoint, a better approach than the one I’ve been considering would be just to change “import urllib.request” to “import urllib2, urllib”, then refactor uses of “urllib.request” on a case-by-case basis.
My previous approach was, upon encountering “import urllib.request”, to find all references to “urllib.request” and refactor them, then import “urllib2″ and/or “urllib” as used. If only names from “urllib2″ were used, then only “urllib2″ would be imported.
This way is overly complex and adds serious mental strain. I originally thought to try it this way because that’s closer to the way that Python logically ends up running such code: it sees “import urllib.request” (or “import urllib.request as billy”) and binds that name in the namespace. A change to that import statement (and the name it binds) will affect everything that references that specific name, and all references must be changed in tandem.
In theory, this is an approach suitable for handling “import urllib.request as subtle”, and I can’t think of a better way to handle that case. However, I’ve already determined that binding a builtin module to a new name adds too much complexity and/or ambiguity (if “import urllib.request as subtle” needs both “urllib” and “urllib2″, which one gets the name “subtle”? should lib3to2 just pick an arbitrary similar [should it even be similar? how similar?] name for the module that does not? why rebind the second one to begin with? follow-up: why rebind the first one if the second one doesn’t have a good reason to be? should it be acceptable to replace this with “import urllib2 as subtle” when only names from “urllib2″ are used, but “import urllib, urllib2″ in other cases? what about third-party usage of the name “subtle”?). So, that reasoning no longer applies. The same argument can be applied to “from urllib import request”, as it is essentially the same as “import urllib.request as request”. In the end, I was so committed to that idea, that I did not take seriously enough the idea of refactoring the import statement and usage separately.
I think I will approach the next part of fix_imports2 (after “from spam.ham import …” cases are done and slightly more thoroughly tested) assuming that “urllib.request.urlopen” actually means the name “urlopen” from the builtin “urllib.request” module that has already been imported, and that “import urllib.request” means to import everything that proviles “urllib.request” functionality, even if it is never referenced later.
With that finished, fix_imports2 should have the level of completion that the other fixers have.
Powered by WordPress