• Immutable Page
• Info
• Attachments
• More Actions: Raw Text Print View Render as Docbook Delete Cache ------------------------ Check Spelling Like Pages Local Site Map ------------------------ Rename Page Delete Page ------------------------ Subscribe User ------------------------ Remove Spam Revert to this revision Package Pages Sync Pages ------------------------ Load Save SlideShow

Moving to newer Python versions

The ultimate goal is to migrate Launchpad to Python 2.6. Rather than go straight to Python 2.6, we are focusing on Python 2.5 first as a stepping stone. People involved in this effort: maxb, gary, salgado, barry, simono. We are treating this work as a sprint; come join us on #launchpad-sprint on freenode.

https://bugs.launchpad.net/launchpad-foundations/+bugs?field.tag=python-upgrade

Branches

• https://code.launchpad.net/~launchpad-dev/launchpad/python-migration is the master branch tracking the migration process. This was branched off of devel. You should submit merge proposals for your branches into this one. gary, salgado, and barry will review all branches and we'll use a lightweight review process to manage this (simple cover letters, 5-minute reviews). Once approved, you can land your change by just pushing it into python-migration; there is no pqm gateway.

Known/Open issues

Bustimications

• The extension modules in your sourcecode directory will have been built for Python 2.4 and these will break until your 2.5 branch. The best thing to do is to move your old sourcecode aside and run utilities/rocketfuel-get to get it again with Python 2.5.

• The pytz source package has a mysterious problem. The work around is to download the 2.5 egg and put that in your download-cache/dist directory.

• _lsprof is in sourcecode because Python 2.4 doesn't come with it. Python 2.5 and 2.6 do though, so we'll be able to remove it from sourcecode once we upgrade.

• launchpadlib will need a new release that prints results in its tests rather than relying on the repr. This should only cause problems if we are unable to move to the newest launchpadlib code for some reason.

• The Python 2.6 cgi module mixes the query string and POST values together. We need to fix zope.publisher so that it does not do this. The pertinent spot to fix is zope.publisher.browser.BrowserRequest.processInputs. The zope.publisher 3.5.7 and 3.5.8 releases (and the 3.5 branch) use an approach that we don't want: they move QUERY_STRING to another key in the environment (X-ORIGINAL-QUERY_STRING). We'd prefer a solution that keeps the query string in the right key. If we hand a QUERY_STRING-free copy of the environment to the cgi.FieldStorage machinery that should work. Note that the solution in 3.5.7 and 3.5.8 is not on trunk for that package, so it is doubly problematic.

Cleanup potential

• Eventually we should get rid of all the scripts hardcoded /usr/bin/pythonX.Y and use buildout's shebang munging.

• scripts/branch-rewrite.py uses /usr/bin/pythonX.Y -u. We should figure out if this is significant and why this script has to be different from other scripts.

• scripts/ftpmaster-tools/_syncorigins.py uses /usr/bin/env pythonX.Y... why?!

Launchpad source dependencies (sourcecode bzr trees)

The Makefiles of pygettextpo & pygpgme make direct reference to Python 2.4, you'll need to amend them appropriately.

Baseline

Baseline failures using Karmic + devel + Python 2.4. There is no plan to fix failures in Python 2.5 for tests that are already broken in Python 2.4.

Tests with errors:
   lp.translations.tests.test_autoapproval.TestCleanup.test_cleanUpInactiveProductEntries
   lp.translations.tests.test_autoapproval.TestCleanup.test_cleanUpObsoleteDistroEntries
   lp.translations.tests.test_autoapproval.TestCleanup.test_cleanUpObsoleteEntries_affected_statuses
   lp.translations.tests.test_autoapproval.TestCleanup.test_cleanUpObsoleteEntries_unaffected_statuses
   lp.translations.tests.test_autoapproval.TestGetPOFileFromLanguage.test_get_pofile_from_language_feeds_enabled_template
   lp.translations.tests.test_autoapproval.TestGetPOFileFromLanguage.test_get_pofile_from_language_starves_disabled_template
   test_CustomLanguageCodeEnablesMatch (lp.translations.tests.test_autoapproval.TestGuessPOFileCustomLanguageCode)
   test_CustomLanguageCodeHidesPOFile (lp.translations.tests.test_autoapproval.TestGuessPOFileCustomLanguageCode)
   test_CustomLanguageCodeParsesBogusLanguage (lp.translations.tests.test_autoapproval.TestGuessPOFileCustomLanguageCode)
   test_CustomLanguageCodePreventsMatch (lp.translations.tests.test_autoapproval.TestGuessPOFileCustomLanguageCode)
   test_CustomLanguageCodeRedirectsMatch (lp.translations.tests.test_autoapproval.TestGuessPOFileCustomLanguageCode)
   test_CustomLanguageCodeReplacesMatch (lp.translations.tests.test_autoapproval.TestGuessPOFileCustomLanguageCode)
   test_CustomLanguageCodesSwitchLanguages (lp.translations.tests.test_autoapproval.TestGuessPOFileCustomLanguageCode)
   test_MatchWithoutCustomLanguageCode (lp.translations.tests.test_autoapproval.TestGuessPOFileCustomLanguageCode)
   test_kde3 (lp.translations.tests.test_autoapproval.TestKdePOFileGuess)
   test_kde4 (lp.translations.tests.test_autoapproval.TestKdePOFileGuess)
   test_ByPathAndOriginDistroFromSourcePackageNoCurrentTemplate (lp.translations.tests.test_autoapproval.TestTemplateGuess)
   test_ByPathAndOriginDistroFromSourcePackageNonCurrentDuplicate (lp.translations.tests.test_autoapproval.TestTemplateGuess)
   test_ByPathAndOriginDistroNoCurrentTemplate (lp.translations.tests.test_autoapproval.TestTemplateGuess)
   test_ByPathAndOriginDistroNonCurrentDuplicate (lp.translations.tests.test_autoapproval.TestTemplateGuess)
   test_ByPathAndOriginProductNoCurrentTemplate (lp.translations.tests.test_autoapproval.TestTemplateGuess)
   test_ByPathAndOriginProductNonCurrentDuplicate (lp.translations.tests.test_autoapproval.TestTemplateGuess)
   test_ByPathAndOrigin_from_package_duplicate (lp.translations.tests.test_autoapproval.TestTemplateGuess)
   test_ByPathAndOrigin_package_duplicate (lp.translations.tests.test_autoapproval.TestTemplateGuess)
   test_ByPathAndOrigin_preferred_match (lp.translations.tests.test_autoapproval.TestTemplateGuess)
   test_ByPathAndOrigin_product_duplicate (lp.translations.tests.test_autoapproval.TestTemplateGuess)
   test_ByTranslationDomain (lp.translations.tests.test_autoapproval.TestTemplateGuess)
   test_ByTranslationDomain_duplicate (lp.translations.tests.test_autoapproval.TestTemplateGuess)
   test_ByTranslationDomain_none (lp.translations.tests.test_autoapproval.TestTemplateGuess)
   test_ClashingEntries (lp.translations.tests.test_autoapproval.TestTemplateGuess)
   lp.codehosting.puller.tests.test_scheduler.TestPullerMasterIntegration.test_mirror
   lp.codehosting.puller.tests.test_scheduler.TestPullerMasterIntegration.test_mirror_with_destination_self_locked
   lp.code.mail.tests.test_codehandler.TestCodeHandler.test_reviewer_with_diff

Tests with failures:
   lp.soyuz.tests.test_packageupload.PackageUploadTestCase.test_realiseUpload_for_delayed_copies
   lib/lp/translations/tests/../doc/sourcepackagerelease-translations.txt
   /home/barry/projects/launchpad/eggs/launchpadlib-1.5.1-py2.4.egg/launchpadlib/docs/people.txt
   /home/barry/projects/launchpad/eggs/launchpadlib-1.5.1-py2.4.egg/launchpadlib/docs/toplevel.txt
   lib/canonical/launchpad/pagetests/webservice/xx-wadl.txt
   lp.codehosting.puller.tests.test_acceptance.TestBranchPuller.test_hosted_branch_stacked_on_mirrored_branch
   lp.codehosting.puller.tests.test_acceptance.TestBranchPuller.test_manual_stacking
   lp.codehosting.puller.tests.test_acceptance.TestBranchPuller.test_mirror_hosted_branch
   lp.codehosting.puller.tests.test_acceptance.TestBranchPuller.test_mirror_hosted_loom_branch
   lp.codehosting.puller.tests.test_acceptance.TestBranchPuller.test_mirror_imported_branch
   lp.codehosting.puller.tests.test_acceptance.TestBranchPuller.test_mirror_mirrored_branch
   lp.codehosting.puller.tests.test_acceptance.TestBranchPuller.test_mirror_private_branch
   lp.codehosting.puller.tests.test_acceptance.TestBranchPuller.test_remirror_hosted_branch
   lp.codehosting.puller.tests.test_acceptance.TestBranchPuller.test_stack_mirrored_branch
   lp.codehosting.puller.tests.test_acceptance.TestBranchPuller.test_stack_mirrored_branch_onto_private
   lp.services.scripts.tests.test_all_scripts.ScriptsTestCase.test_scripts
Total: 24555 tests, 16 failures, 33 errors in 135 minutes 18.295 seconds.

test status for ~gary/launchpad/py25

Tests with errors:
   lp.codehosting.puller.tests.test_scheduler.TestPullerMasterIntegration.test_mirror
   lp.codehosting.puller.tests.test_scheduler.TestPullerMasterIntegration.test_mirror_with_destination_self_locked

Tests with failures:
   lp.scripts.utilities.tests.test_sanitizedb.SanitizeDbScriptTestCase.test_script
   lp.scripts.utilities.tests.test_sanitizedb.SanitizeDbScriptTestCase.test_script_dryrun
   lib/lp/bugs/tests/../doc/checkwatches-cli-switches.txt
   lib/lp/bugs/tests/../doc/externalbugtracker-mantis-csv.txt
   lib/lp/soyuz/tests/../doc/buildd-slavescanner.txt
   lp.soyuz.tests.test_packageupload.PackageUploadTestCase.test_realiseUpload_for_delayed_copies
   lp.translations.scripts.tests.test_reupload_translations.TestReuploadScript.test_reupload_translations
   lib/canonical/shipit/tests/../doc/shipit.txt
   lib/canonical/launchpad/ftests/../doc/pagetest-helpers.txt
   lib/canonical/launchpad/ftests/../doc/tales.txt
   lib/canonical/launchpad/ftests/../doc/unicode_csv.txt
   lib/canonical/launchpad/scripts/ftests/librarianformatter.txt
   lib/lp/bugs/tests/externalbugtracker-xmlrpc-transport.txt
   lib/lp/registry/browser/tests/person-views.txt
   lp.soyuz.scripts.tests.test_copypackage.ReUploadFileTestCase.test_re_upload_file_does_not_leak_file_descriptors
   lib/lp/soyuz/tests/../doc/distroseriesqueue-translations.txt
   lib/lp/translations/tests/../doc/poexport-language-pack.txt
   lib/lp/translations/tests/../doc/poexport-request-productseries.txt
   lib/lp/translations/tests/../doc/poexport-request.txt
   lib/lp/translations/tests/../doc/potemplate.txt
   /home/barry/projects/launchpad/eggs/launchpadlib-1.5.1-py2.5.egg/launchpadlib/docs/people.txt
   /home/barry/projects/launchpad/eggs/launchpadlib-1.5.1-py2.5.egg/launchpadlib/docs/toplevel.txt
   lib/lp/registry/tests/../doc/teammembership.txt
   lib/canonical/signon/stories/openid/per-version/referer-cookie.txt
   lib/canonical/signon/stories/openid/per-version/referer-cookie.txt
   lib/canonical/launchpad/pagetests/feeds/xx-links.txt
   lib/canonical/launchpad/pagetests/webservice/xx-wadl.txt
   lib/lp/answers/tests/../stories/question-add.txt
   lib/lp/answers/tests/../stories/question-obfuscation.txt
   lib/lp/bugs/tests/../stories/distribution/xx-distribution-upstream-bug-report.txt
   lib/lp/bugs/tests/../stories/feeds/xx-bug-atom.txt
   lib/lp/bugs/tests/../stories/feeds/xx-bug-html.txt
   lib/lp/registry/tests/../stories/foaf/xx-person-working-on.txt
   test_getAttrs (lp.codehosting.tests.test_sftp.TestSFTPFile)
   test_serverGetAttrs (lp.codehosting.tests.test_sftp.TestSFTPServer)
   lp.code.scripts.tests.test_update_preview_diffs.TestUpdatePreviewDiffs.test_update_preview_diffs
   lp.codehosting.puller.tests.test_acceptance.TestBranchPuller.test_hosted_branch_stacked_on_mirrored_branch
   lp.codehosting.puller.tests.test_acceptance.TestBranchPuller.test_manual_stacking
   lp.codehosting.puller.tests.test_acceptance.TestBranchPuller.test_mirror_hosted_branch
   lp.codehosting.puller.tests.test_acceptance.TestBranchPuller.test_mirror_hosted_loom_branch
   lp.codehosting.puller.tests.test_acceptance.TestBranchPuller.test_mirror_imported_branch
   lp.codehosting.puller.tests.test_acceptance.TestBranchPuller.test_mirror_mirrored_branch
   lp.codehosting.puller.tests.test_acceptance.TestBranchPuller.test_mirror_private_branch
   lp.codehosting.puller.tests.test_acceptance.TestBranchPuller.test_remirror_hosted_branch
   lp.codehosting.puller.tests.test_acceptance.TestBranchPuller.test_stack_mirrored_branch
   lp.codehosting.puller.tests.test_acceptance.TestBranchPuller.test_stack_mirrored_branch_onto_private
   lib/canonical/launchpad/scripts/tests/librarianformatter_noca.txt
   make_test_tarball_2 (canonical.launchpad.tests.test_helpers)
   test_raising_with_unprintable_exception (canonical.launchpad.webapp.tests.test_errorlog.TestErrorReportingUtility)
   lib/canonical/poppy/tests/filesystem.txt
   lp.services.scripts.tests.test_all_scripts.ScriptsTestCase.test_scripts
   test_EscapedQuotesPropertyFileTest (lp.translations.utilities.tests.test_xpi_properties_format.PropertyFileFormatTestCase)
   test_InvalidPropertyFileUnicodeEscape (lp.translations.utilities.tests.test_xpi_properties_format.PropertyFileFormatTestCase)
   test_MultilinePropertyFileTest (lp.translations.utilities.tests.test_xpi_properties_format.PropertyFileFormatTestCase)
   test_TrailingBackslashPropertyFileTest (lp.translations.utilities.tests.test_xpi_properties_format.PropertyFileFormatTestCase)
   test_UnicodeEscapedPropertyFileTest (lp.translations.utilities.tests.test_xpi_properties_format.PropertyFileFormatTestCase)
Total: 24511 tests, 56 failures, 2 errors in 124 minutes 53.859 seconds.

** 2 import policy violations **

There were 2 imports of names not appearing in the __all__.
You should not import domain_match from cookielib:
    canonical.launchpad.webapp.session
You should not import mktime_tz from email.Utils:
    canonical.launchpad.database.message

Test Status for lp:~maxb/launchpad/python2.5@9026 based on lp:launchpad/stable@9539

• canonical.testing.layers.BaseLayer tests: Ran 256 tests with 0 failures and 0 errors

• canonical.testing.layers.LibrarianLayer tests: Ran 8 tests with 0 failures and 0 errors

• canonical.testing.layers.DatabaseLayer tests: Ran 43 tests with 0 failures and 0 errors

• canonical.testing.layers.LaunchpadLayer tests: Ran 23 tests with 0 failures and 0 errors

• canonical.testing.layers.ZopelessLayer tests: Ran 19 tests with 0 failures and 0 errors

• canonical.testing.layers.LaunchpadScriptLayer tests: Ran 72 tests with 0 failures and 0 errors

• canonical.testing.layers.LaunchpadZopelessLayer tests: Ran 3192 tests with 2 failures and 0 errors

  1. lib/lp/bugs/doc/checkwatches-cli-switches.txt: Looks to be formatting differences in help message (fixed: barry)

  2. lib/lp/bugs/doc/externalbugtracker-mantis-csv.txt: Sort order problem. (fixed: barry)

• canonical.launchpad.ftests.test_system_documentation.ProcessMailLayer tests: Ran 269 tests with 0 failures and 0 errors

• canonical.testing.layers.FunctionalLayer tests: Ran 164 tests with 0 failures and 0 errors

• canonical.testing.layers.LaunchpadFunctionalLayer tests: Ran 8001 tests with 11 failures and 0 errors

  1. lib/canonical/launchpad/ftests/../doc/pagetest-helpers.txt: CDATA section introduced

  2. lib/canonical/launchpad/ftests/../doc/tales.txt: Exception message concerning no len() changed in Py2.5. (fixed: barry)

  3. lib/canonical/launchpad/ftests/../doc/unicode_csv.txt: Py2.5 behaviour change in comparison of unicode / nonunicode
  4. lib/canonical/launchpad/scripts/ftests/librarianformatter.txt: Spurious characters.
  5. lib/lp/bugs/tests/externalbugtracker-xmlrpc-transport.txt: urllib2 seems to be behaving differently concerning installed handlers

  6. lib/lp/registry/browser/tests/person-views.txt: bytestring vs unicode issue (fixed: barry)

  7. lp.soyuz.scripts.tests.test_copypackage.ReUploadFileTestCase.test_re_upload_file_does_not_leak_file_descriptors - 1 != 0

  8. lib/lp/translations/tests/../doc/poexport-language-pack.txt: Hints at tarfile.getnames() behaviour change as seen in unit tests too
  9. lib/lp/translations/tests/../doc/poexport-request-productseries.txt: Ditto.
  10. lib/lp/translations/tests/../doc/poexport-request.txt: Ditto.
  11. lib/lp/translations/tests/../doc/potemplate.txt: Ditto.

• canonical.testing.layers.AppServerLayer tests: Ran 72 tests with 0 failures and 0 errors

• canonical.testing.layers.DatabaseFunctionalLayer tests: Ran 3677 tests with 1 failures and 0 errors

  1. lib/lp/registry/doc/teammembership.txt: Looks like benign formatting issue.

• canonical.testing.layers.PageTestLayer tests: Ran 5965 tests with 7 failures and 0 errors

  1. lib/canonical/launchpad/pagetests/feeds/xx-links.txt: No longer replaces " with ".

  2. lib/lp/answers/stories/question-add.txt: Escaping issues as above, but with < and > too.

  3. lib/lp/answers/stories/question-obfuscation.txt: Ditto.
  4. lib/lp/bugs/stories/feeds/xx-bug-atom.txt: Looks like benign formatting issue
  5. lib/lp/bugs/stories/feeds/xx-bug-html.txt: Looks like benign formatting issue.
  6. lib/lp/bugs/stories/distribution/xx-distribution-upstream-bug-report.txt: Formatting issue.

  7. lib/lp/registry/tests/../stories/foaf/xx-person-working-on.txt: bytestring vs unicode issue (fixed: barry)

• canonical.testing.layers.TwistedLayer tests: Ran 219 tests with 0 failures and 0 errors

• canonical.testing.layers.TwistedLaunchpadZopelessLayer tests: Ran 23 tests with 0 failures and 0 errors

• canonical.testing.layers.TwistedAppServerLayer tests: Ran 5 tests with 0 failures and 0 errors

• canonical.testing.layers.ZopelessAppServerLayer tests: Ran 73 tests with 0 failures and 0 errors

• canonical.testing.layers.ZopelessDatabaseLayer tests: Ran 192 tests with 0 failures and 0 errors

• lp.codehosting.tests.test_acceptance.SSHServerLayer tests: Ran 46 tests with 0 failures and 0 errors

• zope.testing.testrunner.layer.UnitTests tests: Ran 1378 tests with 8 failures and 0 errors

  1. librarianformatter_noca.txt: Py2.5 differences in traceback formatting
  2. test_helpers.make_test_tarball_2: odd discrepancy

  3. test_raising_with_unprintable_exception (canonical.launchpad.webapp.tests.test_errorlog.TestErrorReportingUtility): looks benign

  4. test_EscapedQuotesPropertyFileTest (lp.translations.utilities.tests.test_xpi_properties_format.PropertyFileFormatTestCase)

  5. test_InvalidPropertyFileUnicodeEscape (lp.translations.utilities.tests.test_xpi_properties_format.PropertyFileFormatTestCase)

  6. test_MultilinePropertyFileTest (lp.translations.utilities.tests.test_xpi_properties_format.PropertyFileFormatTestCase)

  7. test_TrailingBackslashPropertyFileTest (lp.translations.utilities.tests.test_xpi_properties_format.PropertyFileFormatTestCase)

  8. test_UnicodeEscapedPropertyFileTest (lp.translations.utilities.tests.test_xpi_properties_format.PropertyFileFormatTestCase)