Diff for "LaunchpadOnKarmic"

Not logged in - Log In / Register

Differences between revisions 1 and 19 (spanning 18 versions)
Revision 1 as of 2009-07-27 23:40:32
Size: 1374
Editor: maxb
Comment:
Revision 19 as of 2009-08-13 11:41:22
Size: 7797
Editor: maxb
Comment:
Deletions are marked like this. Additions are marked like this.
Line 1: Line 1:
As mentioned on [[Getting]], Launchpad is not currently supported on Ubuntu Karmic Koala, primarily because it currently has a hard dependency on Python 2.4, which is no longer included in Karmic. As mentioned on [[Getting]], Launchpad is not currently supported on Ubuntu Karmic Koala.
Line 3: Line 3:
This page is a place to collect notes written for and by people living on the bleeding edge and attempting to get Launchpad working on Karmic / Python 2.5 This page is a place to collect notes written for and by people living on the bleeding edge and attempting to get Launchpad working on Karmic.
Line 5: Line 5:
== launchpad-dependencies ==
First, there is the issue that the ~launchpad PPA doesn't have any packages for Karmic. I (maxb) have tweaked the launchpad-dependencies package to have a first approximation of suitable dependencies for Python 2.5, and uploaded it to my PPA [[https://launchpad.net/~maxb/+archive/ppa]], and copied the geoip-data-city-lite package from the ~launchpad PPA there as well for convenience.
= Phase 1 - Python 2.4 =
Phase 1 is to get it working on Karmic using Python 2.4 - this requires rebuilding quite a few Python module packages, as Python 2.4 is not a supported Python version in Karmic. These packages are in my PPA for Launchpad-related packages [[https://launchpad.net/~maxb/+archive/launchpad]]. In support of phase 2, the launchpad-dependencies package in this PPA uses version-independent dependencies on python module packages.

Status: 4 test failures
 * lib/canonical/lazr/tests/../doc/timeout.txt - wgrant can't reproduce this on karmic i386.
 * lp.soyuz.tests.test_packageupload.!PackageUploadTestCase.test_realiseUpload_for_delayed_copies - wgrant can't reproduce this on karmic i386 either.
 * lib/lp/soyuz/tests/../doc/gina.txt - tar broke dpkg. dpkg was fixed. python breaks the fix. wgrant has a fix.
 * lib/lp/soyuz/tests/../doc/package-diff.txt - karmic's debdiff changed exit code semantics; wgrant has a fix.

Once it becomes possible to run the testsuite successfully with Python 2.4 on Karmic, we move on to phase 2...

= Phase 2 - Python 2.5 =

The ultimate goal is to migrate Launchpad to Python 2.6. Rather than go straight to Python 2.6, I (maxb) am focusing on Python 2.5 first as a stepping stone.
Line 12: Line 24:
 * Making buildmailman.py compatible with Python 2.5 sys.path_importer_cache behaviour
 * Fixing Launchpad's !TestCase subclass for method renames in unittest in Python 2.5
Line 14: Line 28:
https://code.launchpad.net/~maxb/zope3/launchpad-3.4-py2.5 contains backports of some changes to zope.proxy and zope.security in post-3.4 Zope releases which are required for Python 2.5 compatibility (possibly only on 64-bit architectures). Without these, Launchpad produces perplexing MemoryErrors. https://code.launchpad.net/~maxb/zope3/launchpad-3.4-py2.5 contains backports of some changes to zope.proxy and zope.security in post-3.4 Zope releases which are required for Python 2.5 compatibility (possibly only on 64-bit architectures). Without these, Launchpad produces perplexing !MemoryErrors. It also contains a patch to zope.sendmail working around a change in the ordering of atexit vs. waiting for non-main non-daemon threads to shut down which otherwise caused various Python processes to hang on shutdown.

== Launchpad source dependencies - Twisted ==
I used to have a Twisted branch mentioned here. It's now obsoleted by the above patch to zope.sendmail.

== Test Status ==
 * unit tests: Ran 1337 tests with '''9 failures''' and 0 errors
  * librarianformatter_noca.txt: Py2.5 differences in traceback formatting
  * test_helpers.make_test_tarball_2: odd discrepancy
  * test_raising_with_unprintable_exception (canonical.launchpad.webapp.tests.test_errorlog.!TestErrorReportingUtility): looks benign
  * lp.services.scripts.tests.test_all_scripts.ScriptsTestCase.test_scripts: just a local transient issue, I think
  * lib/lp/translations/utilities/tests/test_xpi_properties_format.py: several escaping issues
 * ''canonical.testing.layers.!BaseLayer tests: Ran 250 tests with 0 failures and 0 errors''
 * ''canonical.testing.layers.!LibrarianLayer tests: Ran 8 tests with 0 failures and 0 errors''
 * ''canonical.testing.layers.!DatabaseLayer tests: Ran 48 tests with 0 failures and 0 errors''
 * ''canonical.testing.layers.!LaunchpadLayer tests: Ran 23 tests with 0 failures and 0 errors''
 * ''canonical.testing.layers.!ZopelessLayer tests: Ran 17 tests with 0 failures and 0 errors''
 * canonical.testing.layers.!LaunchpadScriptLayer tests: Ran 72 tests with 0 failures and '''1 errors'''
  * test_initZopelessTwice (canonical.lp.ftests.test_zopeless.!TestInitZopeless) monkeypatches warnings.warn_explicit in way that is no longer compatible
 * canonical.testing.layers.!LaunchpadZopelessLayer tests: Ran 3105 tests with '''5 failures''' and '''9 errors'''
  * canonical.launchpad.scripts.tests.test_entitlementimport.!EntitlementImporterTestCase: multiple errors, UnicodeCSVReader not compatible with Py2.5 csv.
  * lib/lp/bugs/doc/bug-export.txt: Mysterious failure parsing doctest
  * lib/lp/bugs/doc/checkwatches-cli-switches.txt: Looks to be formatting differences in help message
  * lib/lp/bugs/doc/externalbugtracker-mantis-csv.txt: Sort order problem.
  * lib/lp/soyuz/doc/soyuz-set-of-uploads.txt: Uploads rejected?!
  * lp.soyuz.tests.test_packageupload.!PackageUploadTestCase.test_realiseUpload_for_delayed_copies: <fails with python 2.4 too>
 * ''canonical.launchpad.ftests.test_system_documentation.!ProcessMailLayer tests: Ran 269 tests with 0 failures and 0 errors''
 * ''canonical.testing.layers.!FunctionalLayer tests: Ran 164 tests with 0 failures and 0 errors''
 * canonical.testing.layers.!LaunchpadFunctionalLayer tests: Ran 8142 tests with '''11 failures''' and 0 errors
  * lib/canonical/launchpad/doc/pagetest-helpers.txt: CDATA section introduced
  * lib/canonical/launchpad/doc/tales.txt: Exception message changed in Py2.5.
  * lib/canonical/launchpad/doc/unicode_csv.txt: Failure mode changed in Py2.5. Recurrence of above UnicodeCSVReader issue.
  * lib/canonical/launchpad/scripts/ftests/librarianformatter.txt: Spurious characters.
  * lib/lp/bugs/tests/externalbugtracker-xmlrpc-transport.txt: urllib2 seems to be behaving differently
  * lib/lp/soyuz/doc/gina.txt: <fails with python 2.4 too; wgrant fix incoming>
  * lib/lp/soyuz/doc/package-diff.txt: <fails with python 2.4 too; wgrant fix incoming>
  * lib/lp/translations/doc/poexport-language-pack.txt, lib/lp/translations/doc/potemplate.txt: Hints at tarfile.getnames() behaviour change as seen in unit tests too
  * lib/lp/translations/tests/../doc/poexport-request-productseries.txt: ?
  * lib/lp/translations/tests/../doc/poexport-request.txt: ?
 * ''canonical.testing.layers.!AppServerLayer tests: Ran 128 tests with 0 failures and 0 errors''
 * canonical.testing.layers.!DatabaseFunctionalLayer tests: Ran 3098 tests with '''1 failures''' and 0 errors
  * lib/lp/registry/doc/teammembership.txt: Looks like benign formatting issue.
 * canonical.testing.layers.!PageTestLayer tests: Ran 6037 tests with '''6 failures''' and 0 errors
  * lib/canonical/launchpad/pagetests/feeds/xx-links.txt: Formatting issue
  * lib/lp/answers/stories/question-add.txt, question-obfuscation.txt: Formatting issue
  * lib/lp/bugs/stories/feeds/xx-bug-{atom,html}.txt: Looks like benign formatting issue.
  * lib/lp/bugs/stories/distribution/xx-distribution-upstream-bug-report.txt: Formatting issue.
 * ''canonical.testing.layers.!TwistedLayer tests: Ran 219 tests with 0 failures and 0 errors''
 * ''canonical.testing.layers.!TwistedLaunchpadZopelessLayer tests: Ran 23 tests with 0 failures and 0 errors''
 * ''canonical.testing.layers.!TwistedAppServerLayer tests: Ran 5 tests with 0 failures and 0 errors''
 * ''canonical.testing.layers.!ZopelessAppServerLayer tests: Ran 69 tests with 0 failures and 0 errors''
 * ''canonical.testing.layers.!ZopelessDatabaseLayer tests: Ran 145 tests with 0 failures and 0 errors''
 * ''lp.codehosting.tests.test_acceptance.SSHServerLayer tests: Ran 46 tests with 0 failures and 0 errors''

== Other Related Work ==
 * Launchpad currently depends on a monolithic Zope source distribution. Zope upstream has since restructured itself as a group of separated modular eggs. Gary Poster is working on migrating Launchpad to this new structure at https://code.launchpad.net/~gary/launchpad/zbuildout - this is a prerequisite to upgrading to newer Zope, which would then avoid the need to cherrypick backports into the existing Zope sourcedep.

As mentioned on Getting, Launchpad is not currently supported on Ubuntu Karmic Koala.

This page is a place to collect notes written for and by people living on the bleeding edge and attempting to get Launchpad working on Karmic.

Phase 1 - Python 2.4

Phase 1 is to get it working on Karmic using Python 2.4 - this requires rebuilding quite a few Python module packages, as Python 2.4 is not a supported Python version in Karmic. These packages are in my PPA for Launchpad-related packages https://launchpad.net/~maxb/+archive/launchpad. In support of phase 2, the launchpad-dependencies package in this PPA uses version-independent dependencies on python module packages.

Status: 4 test failures

  • lib/canonical/lazr/tests/../doc/timeout.txt - wgrant can't reproduce this on karmic i386.
  • lp.soyuz.tests.test_packageupload.PackageUploadTestCase.test_realiseUpload_for_delayed_copies - wgrant can't reproduce this on karmic i386 either.

  • lib/lp/soyuz/tests/../doc/gina.txt - tar broke dpkg. dpkg was fixed. python breaks the fix. wgrant has a fix.
  • lib/lp/soyuz/tests/../doc/package-diff.txt - karmic's debdiff changed exit code semantics; wgrant has a fix.

Once it becomes possible to run the testsuite successfully with Python 2.4 on Karmic, we move on to phase 2...

Phase 2 - Python 2.5

The ultimate goal is to migrate Launchpad to Python 2.6. Rather than go straight to Python 2.6, I (maxb) am focusing on Python 2.5 first as a stepping stone.

Launchpad code itself

https://code.launchpad.net/~maxb/launchpad/python2.5 contains a few fixes:

  • Adjusting the Python version in the Makefiles
  • Adjusting import statements to handle the built-in location of cElementTree in Python 2.5
  • Making buildmailman.py compatible with Python 2.5 sys.path_importer_cache behaviour
  • Fixing Launchpad's TestCase subclass for method renames in unittest in Python 2.5

Launchpad source dependencies - Zope

https://code.launchpad.net/~maxb/zope3/launchpad-3.4-py2.5 contains backports of some changes to zope.proxy and zope.security in post-3.4 Zope releases which are required for Python 2.5 compatibility (possibly only on 64-bit architectures). Without these, Launchpad produces perplexing MemoryErrors. It also contains a patch to zope.sendmail working around a change in the ordering of atexit vs. waiting for non-main non-daemon threads to shut down which otherwise caused various Python processes to hang on shutdown.

Launchpad source dependencies - Twisted

I used to have a Twisted branch mentioned here. It's now obsoleted by the above patch to zope.sendmail.

Test Status

  • unit tests: Ran 1337 tests with 9 failures and 0 errors

    • librarianformatter_noca.txt: Py2.5 differences in traceback formatting
    • test_helpers.make_test_tarball_2: odd discrepancy
    • test_raising_with_unprintable_exception (canonical.launchpad.webapp.tests.test_errorlog.TestErrorReportingUtility): looks benign

    • lp.services.scripts.tests.test_all_scripts.ScriptsTestCase.test_scripts: just a local transient issue, I think

    • lib/lp/translations/utilities/tests/test_xpi_properties_format.py: several escaping issues
  • canonical.testing.layers.BaseLayer tests: Ran 250 tests with 0 failures and 0 errors

  • canonical.testing.layers.LibrarianLayer tests: Ran 8 tests with 0 failures and 0 errors

  • canonical.testing.layers.DatabaseLayer tests: Ran 48 tests with 0 failures and 0 errors

  • canonical.testing.layers.LaunchpadLayer tests: Ran 23 tests with 0 failures and 0 errors

  • canonical.testing.layers.ZopelessLayer tests: Ran 17 tests with 0 failures and 0 errors

  • canonical.testing.layers.LaunchpadScriptLayer tests: Ran 72 tests with 0 failures and 1 errors

    • test_initZopelessTwice (canonical.lp.ftests.test_zopeless.TestInitZopeless) monkeypatches warnings.warn_explicit in way that is no longer compatible

  • canonical.testing.layers.LaunchpadZopelessLayer tests: Ran 3105 tests with 5 failures and 9 errors

    • canonical.launchpad.scripts.tests.test_entitlementimport.EntitlementImporterTestCase: multiple errors, UnicodeCSVReader not compatible with Py2.5 csv.

    • lib/lp/bugs/doc/bug-export.txt: Mysterious failure parsing doctest
    • lib/lp/bugs/doc/checkwatches-cli-switches.txt: Looks to be formatting differences in help message
    • lib/lp/bugs/doc/externalbugtracker-mantis-csv.txt: Sort order problem.
    • lib/lp/soyuz/doc/soyuz-set-of-uploads.txt: Uploads rejected?!
    • lp.soyuz.tests.test_packageupload.PackageUploadTestCase.test_realiseUpload_for_delayed_copies: <fails with python 2.4 too>

  • canonical.launchpad.ftests.test_system_documentation.ProcessMailLayer tests: Ran 269 tests with 0 failures and 0 errors

  • canonical.testing.layers.FunctionalLayer tests: Ran 164 tests with 0 failures and 0 errors

  • canonical.testing.layers.LaunchpadFunctionalLayer tests: Ran 8142 tests with 11 failures and 0 errors

    • lib/canonical/launchpad/doc/pagetest-helpers.txt: CDATA section introduced
    • lib/canonical/launchpad/doc/tales.txt: Exception message changed in Py2.5.
    • lib/canonical/launchpad/doc/unicode_csv.txt: Failure mode changed in Py2.5. Recurrence of above UnicodeCSVReader issue.
    • lib/canonical/launchpad/scripts/ftests/librarianformatter.txt: Spurious characters.
    • lib/lp/bugs/tests/externalbugtracker-xmlrpc-transport.txt: urllib2 seems to be behaving differently
    • lib/lp/soyuz/doc/gina.txt: <fails with python 2.4 too; wgrant fix incoming>

    • lib/lp/soyuz/doc/package-diff.txt: <fails with python 2.4 too; wgrant fix incoming>

    • lib/lp/translations/doc/poexport-language-pack.txt, lib/lp/translations/doc/potemplate.txt: Hints at tarfile.getnames() behaviour change as seen in unit tests too
    • lib/lp/translations/tests/../doc/poexport-request-productseries.txt: ?
    • lib/lp/translations/tests/../doc/poexport-request.txt: ?
  • canonical.testing.layers.AppServerLayer tests: Ran 128 tests with 0 failures and 0 errors

  • canonical.testing.layers.DatabaseFunctionalLayer tests: Ran 3098 tests with 1 failures and 0 errors

    • lib/lp/registry/doc/teammembership.txt: Looks like benign formatting issue.
  • canonical.testing.layers.PageTestLayer tests: Ran 6037 tests with 6 failures and 0 errors

    • lib/canonical/launchpad/pagetests/feeds/xx-links.txt: Formatting issue
    • lib/lp/answers/stories/question-add.txt, question-obfuscation.txt: Formatting issue
    • lib/lp/bugs/stories/feeds/xx-bug-{atom,html}.txt: Looks like benign formatting issue.
    • lib/lp/bugs/stories/distribution/xx-distribution-upstream-bug-report.txt: Formatting issue.
  • canonical.testing.layers.TwistedLayer tests: Ran 219 tests with 0 failures and 0 errors

  • canonical.testing.layers.TwistedLaunchpadZopelessLayer tests: Ran 23 tests with 0 failures and 0 errors

  • canonical.testing.layers.TwistedAppServerLayer tests: Ran 5 tests with 0 failures and 0 errors

  • canonical.testing.layers.ZopelessAppServerLayer tests: Ran 69 tests with 0 failures and 0 errors

  • canonical.testing.layers.ZopelessDatabaseLayer tests: Ran 145 tests with 0 failures and 0 errors

  • lp.codehosting.tests.test_acceptance.SSHServerLayer tests: Ran 46 tests with 0 failures and 0 errors

  • Launchpad currently depends on a monolithic Zope source distribution. Zope upstream has since restructured itself as a group of separated modular eggs. Gary Poster is working on migrating Launchpad to this new structure at https://code.launchpad.net/~gary/launchpad/zbuildout - this is a prerequisite to upgrading to newer Zope, which would then avoid the need to cherrypick backports into the existing Zope sourcedep.