launchpad development
Moving to newer Python versions
The ultimate goal is to migrate Launchpad to Python 2.6. Rather than go straight to Python 2.6, we are focusing on Python 2.5 first as a stepping stone. People involved in this effort: maxb, gary, salgado, barry, simono. We are treating this work as a sprint; come join us on #launchpad-sprint on freenode.
https://bugs.launchpad.net/launchpad-foundations/+bugs?field.tag=python-upgrade
Branches
https://code.launchpad.net/~launchpad-dev/launchpad/python-migration is the master branch tracking the migration process. This was branched off of devel. You should submit merge proposals for your branches into this one. gary, salgado, and barry will review all branches and we'll use a lightweight review process to manage this (simple cover letters, 5-minute reviews). Once approved, you can land your change by just pushing it into python-migration; there is no pqm gateway.
Known/Open issues
Bustimications
The extension modules in your sourcecode directory will have been built for Python 2.4 and these will break until your 2.5 branch. The best thing to do is to move your old sourcecode aside and run utilities/rocketfuel-get to get it again with Python 2.5.
The pytz source package has a mysterious problem. The work around is to download the 2.5 egg and put that in your download-cache/dist directory.
_lsprof is in sourcecode because Python 2.4 doesn't come with it. Python 2.5 and 2.6 do though, so we'll be able to remove it from sourcecode once we upgrade.
- launchpadlib will need a new release that prints results in its tests rather than relying on the repr. This should only cause problems if we are unable to move to the newest launchpadlib code for some reason.
The Python 2.6 cgi module mixes the query string and POST values together. We need to fix zope.publisher so that it does not do this. The pertinent spot to fix is zope.publisher.browser.BrowserRequest.processInputs. The zope.publisher 3.5.7 and 3.5.8 releases (and the 3.5 branch) use an approach that we don't want: they move QUERY_STRING to another key in the environment (X-ORIGINAL-QUERY_STRING). We'd prefer a solution that keeps the query string in the right key. If we hand a QUERY_STRING-free copy of the environment to the cgi.FieldStorage machinery that should work. Note that the solution in 3.5.7 and 3.5.8 is not on trunk for that package, so it is doubly problematic.
Cleanup potential
Eventually we should get rid of all the scripts hardcoded /usr/bin/pythonX.Y and use buildout's shebang munging.
scripts/branch-rewrite.py uses /usr/bin/pythonX.Y -u. We should figure out if this is significant and why this script has to be different from other scripts.
scripts/ftpmaster-tools/_syncorigins.py uses /usr/bin/env pythonX.Y... why?!
Launchpad source dependencies (sourcecode bzr trees)
The Makefiles of pygettextpo & pygpgme make direct reference to Python 2.4, you'll need to amend them appropriately.
Baseline
Baseline failures using Karmic + devel + Python 2.4. There is no plan to fix failures in Python 2.5 for tests that are already broken in Python 2.4 as part of this effort.
Tests with failures:
lib/canonical/launchpad/pagetests/webservice/xx-wadl.txt - which appears in fact is a bug in lazr.restful, https://launchpad.net/bugs/451129
Test Status for lp:~launchpad-dev/launchpad/python-migration@9687
(Only outstanding problems shown)
(Don't worry about the launchpadlib failures; they fail on karmic + 2.4 too if you have the launchpadlib package installed in your site-packages, and Gary is working on this.)
canonical.testing.layers.LaunchpadFunctionalLayer tests: Ran 8081 tests with 6 failures and 0 errors
NEW FAILURE: lib/canonical/launchpad/ftests/../doc/hierarchical-menu.txt line 180, example 50 'NoneType' object has no attribute 'encode' calling self.text.encode('raw-unicode-escape') (fixed: salgado)
- lib/canonical/launchpad/ftests/../doc/pagetest-helpers.txt: CDATA section introduced
- lib/canonical/launchpad/ftests/../doc/unicode_csv.txt: Py2.5 behaviour change in comparison of unicode / nonunicode
- lib/canonical/launchpad/scripts/ftests/librarianformatter.txt: Spurious characters.
- lib/lp/bugs/tests/externalbugtracker-xmlrpc-transport.txt: urllib2 seems to be behaving differently concerning installed handlers
lp.soyuz.scripts.tests.test_copypackage.ReUploadFileTestCase.test_re_upload_file_does_not_leak_file_descriptors - 1 != 0
canonical.testing.layers.PageTestLayer tests: Ran 5998 tests with 4 failures and 0 errors
lib/canonical/launchpad/pagetests/webservice/xx-wadl.txt: wadl_schema.validate(...) is returning False (bug in lazr.restful)
lib/lp/answers/stories/question-add.txt: Escaping issues as above, but with < and > too.
- lib/lp/answers/stories/question-obfuscation.txt: Ditto.
- lib/lp/bugs/stories/distribution/xx-distribution-upstream-bug-report.txt: Formatting issue.
zope.testing.testrunner.layer.UnitTests tests: Ran 1412 tests with 2 failures and 0 errors
- librarianformatter_noca.txt: Py2.5 differences in traceback formatting
test_raising_with_unprintable_exception (canonical.launchpad.webapp.tests.test_errorlog.TestErrorReportingUtility): looks benign
status lp:~launchpad-dev/launchpad/python-migration@9685
(full test run on 64bit karmic 20091014T0134)
barry: These results must be merged into the above
maxb: Please don't, many are spurious-ish fails. I've edited them below removing dups with the above master list and adding comments.
lp.soyuz.tests.test_packageupload.PackageUploadTestCase.test_realiseUpload_for_delayed_copies
- Spurious fail triggered local umask being other than the Ubuntu default
lib/canonical/shipit/tests/../doc/shipit.txt (fixed: salgado)
lib/canonical/signon/stories/openid/per-version/referer-cookie.txt (fixed: salgado)
lp.codehosting.puller.tests.test_acceptance.TestBranchPuller.test_<various>
- sys.path/site-packages/eggs fubar
- gary's work will fix these
lp.services.scripts.tests.test_all_scripts.ScriptsTestCase.test_scripts
- sys.path/site-packages/eggs fubar
- gary is working on these
- gary's work will fix these
