launchpad development
Moving to newer Python versions
The ultimate goal is to migrate Launchpad to Python 2.6. Rather than go straight to Python 2.6, we are focusing on Python 2.5 first as a stepping stone. People involved in this effort: maxb, gary, salgado, barry, simono. We are treating this work as a sprint; come join us on #launchpad-sprint on freenode.
Relevant tagged bugs: https://bugs.launchpad.net/launchpad-foundations/+bugs?field.tag=python-upgrade
Branches
https://code.launchpad.net/~launchpad-dev/launchpad/python-migration is the master branch tracking the migration to Python 2.5. This was branched off of devel. You should submit merge proposals for your branches into this one. gary, salgado, and barry will review all branches and we'll use a lightweight review process to manage this (simple cover letters, 5-minute reviews). Once approved, you can land your change by just pushing it into python-migration; there is no pqm gateway.
https://code.launchpad.net/~launchpad-dev/launchpad/ztk-2.5 is the python-migration branch with the latest Zope Toolkit updates, still on Python 2.5.
https://code.launchpad.net/~launchpad-dev/launchpad/python-migration2.6 is the ztk-2.5 branch with changes to get us on Python 2.6.
https://code.edge.launchpad.net/~salgado/shipit/py2.5 is the shipit branch that make it possible to run shipit under 2.5
https://code.edge.launchpad.net/~salgado/canonical-identity-provider/ztk-2.5 has a test fix, needed when we change to ztk on 2.5
In general, we'll merge from devel -> python-migration -> ztk-2.5 -> python-migration2.6, but we're keeping them separate in case something prevents us from making it all the way to 2.6.
Things you'll need to do
The Makefiles of pygettextpo & pygpgme (in sourcecode/) make direct reference to Python 2.4, you'll need to amend them appropriately.
Current status of the Python 2.5 effort
It passes all of its tests!
Known/Open issues
Bustimications
The extension modules in your sourcecode directory will have been built for Python 2.4 and these will break until your 2.5 branch. The simple thing to do is to move your old sourcecode aside and run utilities/rocketfuel-get to get it again with Python 2.5. A more complex but useful thing to do to manage sourcecode directories for multiple python versions is:
cd ~/launchpad/lp-sourcedeps mkdir 2.5 cd 2.5 ln -s ../download-cache ln -s ../eggs mkdir sourcecode cd sourcecode for i in ../../sourcecode/*; do bzr checkout --lightweight "$i"; done
and now point utilities/link-external-sourcecode to ../../lp-sourcedeps/2.5 - if you do that you'll need to be aware that rocketfuel-get won't know about it, and your extra directories will require occasional manual updating.
The pytz source package has a mysterious problem, hence we have eggs in download-cache.
_lsprof is in sourcecode because Python 2.4 doesn't come with it. Python 2.5 and 2.6 do though, so we'll be able to remove it from sourcecode once we upgrade. Except, it's not quite as simple as that. Currently we're in a bizare mixed state where the _lsprof.so symlink has been removed, but the cProfile.py one has not been. Furthermore, pstats2.py is also provided by our lsprof, and there's no pstats2 in Python - there's a pstats, though.
Cleanup potential
Eventually we should get rid of all the scripts hardcoded /usr/bin/pythonX.Y and use buildout's shebang munging.
scripts/branch-rewrite.py uses /usr/bin/pythonX.Y -u. We should figure out if this is significant and why this script has to be different from other scripts.
scripts/ftpmaster-tools/_syncorigins.py uses /usr/bin/env pythonX.Y... why?!
cronscripts/mirror-prober.sh and cronscripts/nightly.sh are invoking python scripts with explicit interpreters. Why are they not using the shebangs? This leads to versions being defined in two places.
Python 2.5 w/Zope Toolkit
This is a branch off of python-migration which uses Python 2.5 and the Zope Toolkit.
test failures
- lib/canonical/shipit/tests/../doc/shipit.txt
- Salgado has a branch that fixes this
- lib/canonical/signon/browser/tests/loginservice-unauthorized-view.txt
- Salgado has a branch that fixes this
Python 2.6
This is the ztk-2.5 branch, converted to use Python 2.6.
Also, a multitude of modules of modules (sha, md5, sets, ...) are deprecated in Python 2.6, and many tests will fail merely because of unexpected DeprecationWarnings (but see below).
Getting started
Apply this patch and re-run bin/buildout to temporarily suppress the turning of warnings into errors. There are DeprecationErrors coming from Twisted that we can't seem to turn off any other way.
make run failures
Tests with failures: test_import_script (lp.codehosting.codeimport.tests.test_worker.TestGitImport) testLibrarianRunning (canonical.testing.ftests.test_layers.DatabaseTestCase) testLibrarianRunning (canonical.testing.ftests.test_layers.ZopelessTestCase) lib/canonical/launchpad/ftests/../doc/old-testing.txt testLaunchpadDbAvailable (canonical.testing.ftests.test_layers.FunctionalTestCase) testLibrarianRunning (canonical.testing.ftests.test_layers.FunctionalTestCase) lib/canonical/shipit/tests/../doc/shipit-process-requests.txt lib/canonical/signon/browser/tests/loginservice-unauthorized-view.txt lp.codehosting.tests.test_rewrite.TestBranchRewriterScript.test_script lib/lp/registry/browser/tests/gpg-views.txt lib/lp/registry/tests/../doc/cache-country-mirrors.txt lib/lp/registry/tests/../doc/convert-person-to-team.txt lib/lp/registry/tests/../doc/teammembership.txt BuilddSlaveTestSetup (canonical.buildd.tests.harness) make_test_tarball_2 (canonical.launchpad.tests.test_helpers) shortlist (canonical.launchpad.helpers) shortlist (canonical.launchpad.helpers) _enable_bzr_ssh_in_urlparse (canonical.launchpad.webapp.url) urlparse (canonical.launchpad.webapp.url) urlsplit (canonical.launchpad.webapp.url) /home/barry/projects/launchpad/python-migration2.6/lib/canonical/lazr/tests/../doc/timeout.txt LibrarianTestSetup (canonical.librarian.ftests.harness) ZecaTestSetup (canonical.zeca.ftests.harness) test_successful_start_then_stop (lp.codehosting.tests.test_lpserve.TestLaunchpadServe) test_unexpected_error_logs_oops (lp.codehosting.tests.test_lpserve.TestLaunchpadServe) lp.services.scripts.tests.test_all_scripts.ScriptsTestCase.test_scripts lib/lp/translations/tests/../doc/remove-upstream-translations-script.txt Total: 6208 tests, 27 failures, 27 errors in 28 minutes 46.400 seconds.
