Diff for "PythonMigrationStatus"

Not logged in - Log In / Register

Differences between revisions 1 and 115 (spanning 114 versions)
Revision 1 as of 2009-10-09 17:47:27
Size: 6491
Editor: barry
Comment:
Revision 115 as of 2010-04-07 20:42:43
Size: 29356
Editor: maxb
Comment:
Deletions are marked like this. Additions are marked like this.
Line 3: Line 3:
The ultimate goal is to migrate Launchpad to Python 2.6. Rather than go straight to Python 2.6, we are focusing on Python 2.5 first as a stepping stone. People involved in this effort: maxb, gary, salgado, barry

https://bugs.launchpad.net/launchpad-foundations/+bugs?field.tag=python-upgrade

== Interesting branches ==

 * https://code.launchpad.net/~maxb/launchpad/python2.5 tracks `lp:launchpad/stable` (or occasionally `devel`), on a manual basis, and acts as a one-stop-shop for something that people can check out and build using Python 2.5

 * https://code.launchpad.net/~gary/launchpad/py25 is gary's branch tracking the upgrade.

 * ''add yours here''

== Launchpad code itself ==

Remaining changes in `python2.5` branch:
 * Changed `PYTHON_VERSION` in Makefiles
 * Changed shebang in `cronscripts/*`, `scripts/*`,

== Known issues ==

 * The pytz source package has a [[https://launchpad.net/bugs/438634|mysterious problem]]. The work around is to [[http://pypi.python.org/packages/2.5/p/pytz/pytz-2009l-py2.5.egg#md5=b41c8cacb5f4a8a64db05afbdf3ef9ef|download the 2.5 egg]] and put that in your `download-cache/dist` directory.

== Launchpad source dependencies (sourcecode bzr trees) ==

The `Makefile`s of pygettextpo & pygpgme make direct reference to Python 2.4, you'll need to amend them appropriately.

== Test Status for lp:~maxb/launchpad/python2.5@9026 based on lp:launchpad/stable@9539 ==

 * ''canonical.testing.layers.!BaseLayer tests: Ran 256 tests with 0 failures and 0 errors''
 * ''canonical.testing.layers.!LibrarianLayer tests: Ran 8 tests with 0 failures and 0 errors''
 * ''canonical.testing.layers.!DatabaseLayer tests: Ran 43 tests with 0 failures and 0 errors''
 * ''canonical.testing.layers.!LaunchpadLayer tests: Ran 23 tests with 0 failures and 0 errors''
 * ''canonical.testing.layers.!ZopelessLayer tests: Ran 19 tests with 0 failures and 0 errors''
 * ''canonical.testing.layers.!LaunchpadScriptLayer tests: Ran 72 tests with 0 failures and 0 errors''
 * canonical.testing.layers.!LaunchpadZopelessLayer tests: Ran 3192 tests with '''2 failures''' and 0 errors
  1. lib/lp/bugs/doc/checkwatches-cli-switches.txt: Looks to be formatting differences in help message
  1. lib/lp/bugs/doc/externalbugtracker-mantis-csv.txt: Sort order problem.
 * ''canonical.launchpad.ftests.test_system_documentation.!ProcessMailLayer tests: Ran 269 tests with 0 failures and 0 errors''
 * ''canonical.testing.layers.!FunctionalLayer tests: Ran 164 tests with 0 failures and 0 errors''
 * canonical.testing.layers.!LaunchpadFunctionalLayer tests: Ran 8001 tests with '''11 failures''' and 0 errors
  1. lib/canonical/launchpad/ftests/../doc/pagetest-helpers.txt: CDATA section introduced
  1. lib/canonical/launchpad/ftests/../doc/tales.txt: Exception message concerning no len() changed in Py2.5.
  1. lib/canonical/launchpad/ftests/../doc/unicode_csv.txt: Py2.5 behaviour change in comparison of unicode / nonunicode
  1. lib/canonical/launchpad/scripts/ftests/librarianformatter.txt: Spurious characters.
  1. lib/lp/bugs/tests/externalbugtracker-xmlrpc-transport.txt: urllib2 seems to be behaving differently concerning installed handlers
  1. lib/lp/registry/browser/tests/person-views.txt: bytestring vs unicode issue
  1. lp.soyuz.scripts.tests.test_copypackage.!ReUploadFileTestCase.test_re_upload_file_does_not_leak_file_descriptors - `1 != 0`
  1. lib/lp/translations/tests/../doc/poexport-language-pack.txt: Hints at tarfile.getnames() behaviour change as seen in unit tests too
  1. lib/lp/translations/tests/../doc/poexport-request-productseries.txt: Ditto.
  1. lib/lp/translations/tests/../doc/poexport-request.txt: Ditto.
  1. lib/lp/translations/tests/../doc/potemplate.txt: Ditto.
 * ''canonical.testing.layers.!AppServerLayer tests: Ran 72 tests with 0 failures and 0 errors''
 * canonical.testing.layers.!DatabaseFunctionalLayer tests: Ran 3677 tests with '''1 failures''' and 0 errors
  1. lib/lp/registry/doc/teammembership.txt: Looks like benign formatting issue.
 * canonical.testing.layers.!PageTestLayer tests: Ran 5965 tests with '''7 failures''' and 0 errors
  1. lib/canonical/launchpad/pagetests/feeds/xx-links.txt: No longer replaces " with ".
  1. lib/lp/answers/stories/question-add.txt: Escaping issues as above, but with < and > too.
  1. lib/lp/answers/stories/question-obfuscation.txt: Ditto.
  1. lib/lp/bugs/stories/feeds/xx-bug-atom.txt: Looks like benign formatting issue
  1. lib/lp/bugs/stories/feeds/xx-bug-html.txt: Looks like benign formatting issue.
  1. lib/lp/bugs/stories/distribution/xx-distribution-upstream-bug-report.txt: Formatting issue.
  1. lib/lp/registry/tests/../stories/foaf/xx-person-working-on.txt: bytestring vs unicode issue
 * ''canonical.testing.layers.!TwistedLayer tests: Ran 219 tests with 0 failures and 0 errors''
 * ''canonical.testing.layers.!TwistedLaunchpadZopelessLayer tests: Ran 23 tests with 0 failures and 0 errors''
 * ''canonical.testing.layers.!TwistedAppServerLayer tests: Ran 5 tests with 0 failures and 0 errors''
 * ''canonical.testing.layers.!ZopelessAppServerLayer tests: Ran 73 tests with 0 failures and 0 errors''
 * ''canonical.testing.layers.!ZopelessDatabaseLayer tests: Ran 192 tests with 0 failures and 0 errors''
 * ''lp.codehosting.tests.test_acceptance.SSHServerLayer tests: Ran 46 tests with 0 failures and 0 errors''
 * zope.testing.testrunner.layer.!UnitTests tests: Ran 1378 tests with '''8 failures''' and 0 errors
  1. librarianformatter_noca.txt: Py2.5 differences in traceback formatting
  1. test_helpers.make_test_tarball_2: odd discrepancy
  1. test_raising_with_unprintable_exception (canonical.launchpad.webapp.tests.test_errorlog.!TestErrorReportingUtility): looks benign
  1. test_!EscapedQuotesPropertyFileTest (lp.translations.utilities.tests.test_xpi_properties_format.!PropertyFileFormatTestCase)
  1. test_!InvalidPropertyFileUnicodeEscape (lp.translations.utilities.tests.test_xpi_properties_format.!PropertyFileFormatTestCase)
  1. test_!MultilinePropertyFileTest (lp.translations.utilities.tests.test_xpi_properties_format.!PropertyFileFormatTestCase)
  1. test_!TrailingBackslashPropertyFileTest (lp.translations.utilities.tests.test_xpi_properties_format.!PropertyFileFormatTestCase)
  1. test_!UnicodeEscapedPropertyFileTest (lp.translations.utilities.tests.test_xpi_properties_format.!PropertyFileFormatTestCase)
<<TableOfContents>>

With Lucid fast approaching, efforts to actually migrate Launchpad to Python 2.6 are getting under way.

Relevant tagged bugs: https://bugs.launchpad.net/launchpad-foundations/+bugs?field.tag=python-upgrade

== Branches ==

=== Obsolete branches from the last push, which got us to 2.5 ===
 * https://code.launchpad.net/~launchpad-committers/launchpad/python-migration2.6 is the incomplete work done on Python 2.6 compatibility mixed with ZTK upgrade work since cherrypicked and landed.
 * https://code.launchpad.net/~launchpad-committers/launchpad/python-migration2.6-rebased is the above rebased on recent devel dropping then-empty revisions.

Remaining useful content of these branches:
 * shebang updates / Makefile PYTHON_VERSION updates all over the place -- ''redone in salgado's branch below''
 * Deprecation warning suppression. -- ''cherrypicked into salgado's branch below''
 * versions.cfg change: lazr.smtptest = 1.2 -- ''disregarded, there is no known reason why this was Python 2.6 related''

=== The new effort ===
 * https://code.launchpad.net/~salgado/launchpad/real-python-2.6 -- the most current work so far.

== Known/Open issues ==

'''Today I've tried building a python-migration2.6-rebased branch (after merging devel) but it failed with a max recursion depth exceeded, very similar to what's described in http://bitbucket.org/tarek/distribute/issue/63/distribute-leaves-system-setuptools-in-inconsistent (although that's supposed to be fixed in our version of buildout). http://paste.ubuntu.com/409662/ is the actual error that I got.''' -- GuilhermeSalgado

=== Setup caveat ===
 * The extension modules in your `sourcecode` directory will have been built for Python 2.5 and these will break for your 2.6 branch. The simple thing to do is to move your old `sourcecode` aside and run `utilities/rocketfuel-get` to get it again with Python 2.6. A more complex but useful thing to do to manage `sourcecode` directories for multiple python versions is: {{{
cd ~/launchpad/lp-sourcedeps
mkdir 2.6
cd 2.6
ln -s ../download-cache
ln -s ../eggs
mkdir sourcecode
cd sourcecode
for i in ../../sourcecode/*; do bzr checkout --lightweight "$i"; done
}}} and now point `utilities/link-external-sourcecode` to `../../lp-sourcedeps/2.6` - if you do that you'll need to be aware that `rocketfuel-get` won't know about it, and your extra directories will require occasional manual updating.

=== Cleanup potential ===
 * Is there any reason we need ipython-0.9.1-py2.5.egg in egg, not tarball, form?
  * Presumably we could bin the wadllib 0.1 eggs too.
 * Eventually we should get rid of all the scripts hardcoded `/usr/bin/pythonX.Y` and use buildout's shebang munging.
 * `scripts/branch-rewrite.py` uses `/usr/bin/pythonX.Y -u`. We should figure out if this is significant and why this script has to be different from other scripts.
 * `scripts/ftpmaster-tools/_syncorigins.py` uses `/usr/bin/env pythonX.Y`... why?!
 * `cronscripts/mirror-prober.sh` and `cronscripts/nightly.sh` are invoking python scripts with explicit interpreters. Why are they not using the shebangs? This leads to versions being defined in two places.

=== Situation ===

A multitude of modules of modules (sha, md5, sets, ...) are deprecated in Python 2.6, and many tests would fail merely because of unexpected !DeprecationWarnings, so we've had to jump through multiple hoops all over the place to suppress these. Ultimately we need a plan for removing these suppressions, because we ''want'' to know about deprecation long-term.

== Known test failures and notes thereon ==

 * Running canonical.testing.layers.BaseLayer tests:
  * test_import (lp.codehosting.codeimport.tests.test_worker.TestGitImport) {{{
Error in test test_import (lp.codehosting.codeimport.tests.test_worker.TestGitImport)
Traceback (most recent call last):
  File "/home/barry/projects/launchpad/eggs/bzr-2.0.0_lp_1-py2.6-linux-x86_64.egg/bzrlib/tests/__init__.py", line 1470, in run
    testMethod()
  File "/home/barry/projects/launchpad/python-migration2.6/lib/lp/codehosting/codeimport/tests/test_worker.py", line 641, in test_import
    worker.run()
  File "/home/barry/projects/launchpad/python-migration2.6/lib/lp/codehosting/codeimport/worker.py", line 398, in run
    self._doImport()
  File "/home/barry/projects/launchpad/python-migration2.6/lib/lp/codehosting/codeimport/worker.py", line 495, in _doImport
    self.pushBazaarWorkingTree(bazaar_tree)
  File "/home/barry/projects/launchpad/python-migration2.6/lib/lp/codehosting/codeimport/worker.py", line 525, in pushBazaarWorkingTree
    'git.db', bazaar_tree.branch.repository._transport)
  File "/home/barry/projects/launchpad/python-migration2.6/lib/lp/codehosting/codeimport/worker.py", line 254, in put
    local_file = source_transport.get(filename)
  File "/home/barry/projects/launchpad/eggs/bzr-2.0.0_lp_1-py2.6-linux-x86_64.egg/bzrlib/transport/local.py", line 167, in get
    self._translate_error(e, path)
  File "/home/barry/projects/launchpad/eggs/bzr-2.0.0_lp_1-py2.6-linux-x86_64.egg/bzrlib/transport/__init__.py", line 306, in _translate_error
    raise errors.NoSuchFile(path, extra=e)
NoSuchFile: No such file: u'/var/tmp/codeimport/data/worker-for-branch-1/bzr_working_tree/.bzr/repository/git.db': [Errno 2] No such file or directory: u'/var/tmp/codeimport/data/worker-for-branch-1/bzr_working_tree/.bzr/repository/git.db'
}}}
  * test_import_script (lp.codehosting.codeimport.tests.test_worker.TestGitImport) {{{
Failure in test test_import_script (lp.codehosting.codeimport.tests.test_worker.TestGitImport)
Traceback (most recent call last):
  File "/home/barry/projects/launchpad/eggs/bzr-2.0.0_lp_1-py2.6-linux-x86_64.egg/bzrlib/tests/__init__.py", line 1470, in run
    testMethod()
  File "/home/barry/projects/launchpad/python-migration2.6/lib/lp/codehosting/codeimport/tests/test_worker.py", line 688, in test_import_script
    self.assertEqual(retcode, 0)
  File "/home/barry/projects/launchpad/eggs/bzr-2.0.0_lp_1-py2.6-linux-x86_64.egg/bzrlib/tests/__init__.py", line 958, in assertEqual
    pformat(a), pformat(b)))
AssertionError: not equal:
a = 1
b = 0
}}}
  * test_sync (lp.codehosting.codeimport.tests.test_worker.TestGitImport) {{{
Error in test test_sync (lp.codehosting.codeimport.tests.test_worker.TestGitImport)
Traceback (most recent call last):
  File "/home/barry/projects/launchpad/eggs/bzr-2.0.0_lp_1-py2.6-linux-x86_64.egg/bzrlib/tests/__init__.py", line 1470, in run
    testMethod()
  File "/home/barry/projects/launchpad/python-migration2.6/lib/lp/codehosting/codeimport/tests/test_worker.py", line 649, in test_sync
    worker.run()
  File "/home/barry/projects/launchpad/python-migration2.6/lib/lp/codehosting/codeimport/worker.py", line 398, in run
    self._doImport()
  File "/home/barry/projects/launchpad/python-migration2.6/lib/lp/codehosting/codeimport/worker.py", line 495, in _doImport
    self.pushBazaarWorkingTree(bazaar_tree)
  File "/home/barry/projects/launchpad/python-migration2.6/lib/lp/codehosting/codeimport/worker.py", line 525, in pushBazaarWorkingTree
    'git.db', bazaar_tree.branch.repository._transport)
  File "/home/barry/projects/launchpad/python-migration2.6/lib/lp/codehosting/codeimport/worker.py", line 254, in put
    local_file = source_transport.get(filename)
  File "/home/barry/projects/launchpad/eggs/bzr-2.0.0_lp_1-py2.6-linux-x86_64.egg/bzrlib/transport/local.py", line 167, in get
    self._translate_error(e, path)
  File "/home/barry/projects/launchpad/eggs/bzr-2.0.0_lp_1-py2.6-linux-x86_64.egg/bzrlib/transport/__init__.py", line 306, in _translate_error
    raise errors.NoSuchFile(path, extra=e)
NoSuchFile: No such file: u'/var/tmp/codeimport/data/worker-for-branch-1/bzr_working_tree/.bzr/repository/git.db': [Errno 2] No such file or directory: u'/var/tmp/codeimport/data/worker-for-branch-1/bzr_working_tree/.bzr/repository/git.db'
}}}
 * Running canonical.testing.layers.LaunchpadScriptLayer tests:
  * test_initZopelessTwice (canonical.lp.ftests.test_zopeless.TestInitZopeless) {{{
Error in test test_initZopelessTwice (canonical.lp.ftests.test_zopeless.TestInitZopeless)
Traceback (most recent call last):
  File "/usr/lib/python2.6/unittest.py", line 279, in run
    testMethod()
  File "/home/barry/projects/launchpad/python-migration2.6/lib/canonical/lp/ftests/test_zopeless.py", line 45, in test_initZopelessTwice
    dbuser='launchpad')
  File "/home/barry/projects/launchpad/python-migration2.6/lib/canonical/lp/__init__.py", line 84, in initZopeless
    dbname=dbname, dbhost=dbhost, dbuser=dbuser, isolation=isolation)
  File "/home/barry/projects/launchpad/python-migration2.6/lib/canonical/database/sqlbase.py", line 328, in initZopeless
    warnings.warn(alreadyInstalledMsg, stacklevel=3)
UserWarning: A ZopelessTransactionManager with these settings is already installed. This is probably caused by calling initZopeless twice.
}}}
 * Running canonical.testing.layers.LaunchpadZopelessLayer tests:
  * lib/lp/soyuz/tests/../doc/buildd-slave.txt {{{
Failure in test lib/lp/soyuz/tests/../doc/buildd-slave.txt
Failed doctest test for buildd-slave.txt
  File "lib/lp/soyuz/tests/../doc/buildd-slave.txt", line 0

----------------------------------------------------------------------
File "lib/lp/soyuz/tests/../doc/buildd-slave.txt", line 102, in buildd-slave.txt
Failed example:
    s.info()
Differences (ndiff with -expected +actual):
      Traceback (most recent call last):
    - ...
    + File "/home/barry/projects/launchpad/eggs/zope.testing-3.8.1-py2.6.egg/zope/testing/doctest.py", line 1361, in __run
    + compileflags, 1) in test.globs
    + File "<doctest buildd-slave.txt[line 102, example 38]>", line 1, in <module>
    + File "/usr/lib/python2.6/xmlrpclib.py", line 1199, in __call__
    + return self.__send(self.__name, args)
    + File "/usr/lib/python2.6/xmlrpclib.py", line 1489, in __request
    + verbose=self.__verbose
    + File "/usr/lib/python2.6/xmlrpclib.py", line 1235, in request
    + self.send_content(h, request_body)
    + File "/usr/lib/python2.6/xmlrpclib.py", line 1349, in send_content
    + connection.endheaders()
    + File "/usr/lib/python2.6/httplib.py", line 892, in endheaders
    + self._send_output()
    + File "/usr/lib/python2.6/httplib.py", line 764, in _send_output
    + self.send(msg)
    + File "/usr/lib/python2.6/httplib.py", line 723, in send
    + self.connect()
    + File "/usr/lib/python2.6/httplib.py", line 704, in connect
    + self.timeout)
    + File "/usr/lib/python2.6/socket.py", line 514, in create_connection
    + raise error, msg
    - error: (111, 'Connection refused')
    ? ^ ^^^ --
    + error: [Errno 111] Connection refused
    ? ^^^^^^^ ^^
}}}
  * lib/lp/soyuz/tests/../doc/manage-chroot.txt {{{
Failure in test lib/lp/soyuz/tests/../doc/manage-chroot.txt
Failed doctest test for manage-chroot.txt
  File "lib/lp/soyuz/tests/../doc/manage-chroot.txt", line 0

----------------------------------------------------------------------
File "lib/lp/soyuz/tests/../doc/manage-chroot.txt", line 189, in manage-chroot.txt
Failed example:
    try:
         manage_chroot.mainTask()
    except SoyuzScriptError, info:
        print info
    else:
        print "Did not get expected exception"
Differences (ndiff with -expected +actual):
    - DEBUG Initialising ChrootManager for 'The Hoary Hedgehog Release for i386 (x86)'
    ? --
    + DEBUG Initialising ChrootManager for 'The Hoary Hedgehog Release for i386 (x86)'
    - Librarian upload failed: [...]: (111, 'Connection refused')
    ? ^^^ ^ ^^^ --
    + Librarian upload failed: [localhost:57930]: [Errno 111] Connection refused
    ? ^^^^^^^^^^^^^^^ ^^^^^^^ ^^
}}}
  * lib/canonical/launchpad/ftests/../doc/incomingmail.txt {{{
Failure in test lib/canonical/launchpad/ftests/../doc/incomingmail.txt
Failed doctest test for incomingmail.txt
  File "lib/canonical/launchpad/ftests/../doc/incomingmail.txt", line 0

----------------------------------------------------------------------
File "lib/canonical/launchpad/ftests/../doc/incomingmail.txt", line 377, in incomingmail.txt
Failed example:
    handleMail(transaction)
Differences (ndiff with -expected +actual):
    - ERROR:...:Upload to Librarian failed...
    ? ^^^ ---
    + ERROR:process-mail:Upload to Librarian failed
    ? ^^^^^^^^^^^^
    - ...
    - UploadFailed: [...]: (111, 'Connection refused')
    + Traceback (most recent call last):
    + File "/home/barry/projects/launchpad/python-migration2.6/lib/canonical/launchpad/mail/incoming.py", line 202, in handleMail
    + cStringIO(raw_mail), 'message/rfc822')
    + File "/home/barry/projects/launchpad/python-migration2.6/lib/canonical/launchpad/database/librarian.py", line 210, in create
    + fid = client.addFile(name, size, file, contentType, expires, debugID)
    + File "/home/barry/projects/launchpad/python-migration2.6/lib/canonical/librarian/client.py", line 104, in addFile
    + self._connect()
    + File "/home/barry/projects/launchpad/python-migration2.6/lib/canonical/librarian/client.py", line 56, in _connect
    + '[%s:%s]: %s' % (self.upload_host, self.upload_port, x))
    + UploadFailed: [localhost:57930]: [Errno 111] Connection refused
    + ERROR:process-mail:Upload to Librarian failed
    + Traceback (most recent call last):
    + File "/home/barry/projects/launchpad/python-migration2.6/lib/canonical/launchpad/mail/incoming.py", line 202, in handleMail
    + cStringIO(raw_mail), 'message/rfc822')
    + File "/home/barry/projects/launchpad/python-migration2.6/lib/canonical/launchpad/database/librarian.py", line 210, in create
    + fid = client.addFile(name, size, file, contentType, expires, debugID)
    + File "/home/barry/projects/launchpad/python-migration2.6/lib/canonical/librarian/client.py", line 104, in addFile
    + self._connect()
    + File "/home/barry/projects/launchpad/python-migration2.6/lib/canonical/librarian/client.py", line 56, in _connect
    + '[%s:%s]: %s' % (self.upload_host, self.upload_port, x))
    + UploadFailed: [localhost:57930]: [Errno 111] Connection refused
}}}
 * Running canonical.testing.layers.LaunchpadFunctionalLayer tests:
  * Set up canonical.testing.layers.GoogleServiceLayer {{{
Traceback (most recent call last):
  File "/home/barry/projects/launchpad/eggs/zope.testing-3.8.1-py2.6.egg/zope/testing/testrunner/runner.py", line 364, in run_layer
    setup_layer(options, layer, setup_layers)
  File "/home/barry/projects/launchpad/eggs/zope.testing-3.8.1-py2.6.egg/zope/testing/testrunner/runner.py", line 624, in setup_layer
    setup_layer(options, base, setup_layers)
  File "/home/barry/projects/launchpad/eggs/zope.testing-3.8.1-py2.6.egg/zope/testing/testrunner/runner.py", line 629, in setup_layer
    layer.setUp()
  File "/home/barry/projects/launchpad/python-migration2.6/lib/canonical/testing/layers.py", line 1009, in setUp
    google.setUp()
  File "/home/barry/projects/launchpad/python-migration2.6/lib/canonical/launchpad/testing/tests/googleserviceharness.py", line 76, in setUp
    self.startService()
  File "/home/barry/projects/launchpad/python-migration2.6/lib/canonical/launchpad/testing/tests/googleserviceharness.py", line 88, in startService
    googletestservice.wait_for_service()
  File "/home/barry/projects/launchpad/python-migration2.6/lib/canonical/launchpad/testing/googletestservice.py", line 126, in wait_for_service
    sock.connect((host, port))
  File "<string>", line 1, in connect
error: [Errno 103] Software caused connection abort
}}}
 * Running canonical.testing.layers.AppServerLayer tests:
  * Same layer failure as above
 * Running canonical.testing.layers.DatabaseFunctionalLayer tests:
  * lib/canonical/shipit/tests/../doc/shipit-process-requests.txt {{{

Failure in test lib/canonical/shipit/tests/../doc/shipit-process-requests.txt
Failed doctest test for shipit-process-requests.txt
  File "lib/canonical/shipit/tests/../doc/shipit-process-requests.txt", line 0

----------------------------------------------------------------------
File "lib/canonical/shipit/tests/../doc/shipit-process-requests.txt", line 96, in shipit-process-requests.txt
Failed example:
    out, err
Differences (ndiff with -expected +actual):
    - ('', '')
    + ('', '/home/barry/projects/launchpad/python-migration2.6/lib/cscvs/dircompare/dirsums.py:18: DeprecationWarning: the md5 module is deprecated; use hashlib instead\n import md5\n/home/barry/projects/launchpad/python-migration2.6/lib/canonical/uuid.py:10: DeprecationWarning: the sha module is deprecated; use the hashlib module instead\n import time, random, subprocess, thread, sha\n')
}}}
  * canonical.signon.browser.tests.test_openidserver.OpenIDMixin_checkTeamMembership_TestCase.test_full_fledged_account {{{
Error in test canonical.signon.browser.tests.test_openidserver.OpenIDMixin_checkTeamMembership_TestCase.test_full_fledged_account
Traceback (most recent call last):
  File "/home/barry/projects/launchpad/python-migration2.6/lib/lp/testing/__init__.py", line 390, in run
    self.setUp()
  File "/home/barry/projects/launchpad/python-migration2.6/lib/canonical/signon/browser/tests/test_openidserver.py", line 268, in setUp
    self.openid_mixin = CheckTeamMembershipTest(None, self.request)
  File "/home/barry/projects/launchpad/python-migration2.6/lib/canonical/signon/browser/openidserver.py", line 101, in __init__
    super(OpenIDMixin, self).__init__(context, request)
TypeError: object.__init__() takes no parameters
}}}
  * canonical.signon.browser.tests.test_openidserver.OpenIDMixin_checkTeamMembership_TestCase.test_personless_account {{{
Error in test canonical.signon.browser.tests.test_openidserver.OpenIDMixin_checkTeamMembership_TestCase.test_personless_account
Traceback (most recent call last):
  File "/home/barry/projects/launchpad/python-migration2.6/lib/lp/testing/__init__.py", line 390, in run
    self.setUp()
  File "/home/barry/projects/launchpad/python-migration2.6/lib/canonical/signon/browser/tests/test_openidserver.py", line 268, in setUp
    self.openid_mixin = CheckTeamMembershipTest(None, self.request)
  File "/home/barry/projects/launchpad/python-migration2.6/lib/canonical/signon/browser/openidserver.py", line 101, in __init__
    super(OpenIDMixin, self).__init__(context, request)
TypeError: object.__init__() takes no parameters
}}}
  * lots more of those
  * lib/canonical/signon/browser/tests/loginservice-unauthorized-view.txt {{{
Failure in test lib/canonical/signon/browser/tests/loginservice-unauthorized-view.txt
Failed doctest test for loginservice-unauthorized-view.txt
  File "lib/canonical/signon/browser/tests/loginservice-unauthorized-view.txt", line 0

----------------------------------------------------------------------
File "lib/canonical/signon/browser/tests/loginservice-unauthorized-view.txt", line 39, in loginservice-unauthorized-view.txt
Failed example:
    browser.open('http://openid.launchpad.dev/+edit')
Differences (ndiff with -expected +actual):
    - ERROR:...
}}}
 * Running canonical.testing.layers.PageTestLayer tests:
  * Google service layer failure again
 * Running canonical.testing.layers.TwistedLayer tests:
  * test_run_proces_with_timeout_success (canonical.twistedsupport.tests.test_processmonitor.TestRunProcessWithTimeout) {{{
Error in test test_run_proces_with_timeout_success (canonical.twistedsupport.tests.test_processmonitor.TestRunProcessWithTimeout)
Traceback (most recent call last):
  File "/home/barry/projects/launchpad/python-migration2.6/lib/twisted/trial/unittest.py", line 1250, in run
    warnings.warn_explicit(**w)
PotentialZombieWarning: spawnProcess called, but the SIGCHLD handler is not installed. This probably means you have not yet called reactor.run, or called reactor.run(installSignalHandler=0). You will probably never see this process finish, and it may become a zombie process.
}}}
  * more of the same
 * Running canonical.testing.layers.TwistedLaunchpadZopelessLayer tests:
  * test_import_git (lp.codehosting.codeimport.tests.test_workermonitor.TestWorkerMonitorIntegration) {{{
Failure in test test_import_git (lp.codehosting.codeimport.tests.test_workermonitor.TestWorkerMonitorIntegration)
Traceback (most recent call last):
  File "/home/barry/projects/launchpad/python-migration2.6/lib/twisted/internet/defer.py", line 323, in _runCallbacks
    self.result = callback(self.result, *args, **kw)
  File "/home/barry/projects/launchpad/python-migration2.6/lib/lp/codehosting/codeimport/workermonitor.py", line 124, in transacted
    return function(*args, **kwargs)
  File "/home/barry/projects/launchpad/python-migration2.6/lib/lp/codehosting/codeimport/tests/test_workermonitor.py", line 531, in assertImported
    self.assertCodeImportResultCreated(code_import)
  File "/home/barry/projects/launchpad/python-migration2.6/lib/lp/codehosting/codeimport/tests/test_workermonitor.py", line 513, in assertCodeImportResultCreated
    self.assertEqual(result.status, CodeImportResultStatus.SUCCESS)
  File "/home/barry/projects/launchpad/python-migration2.6/lib/twisted/trial/unittest.py", line 290, in failUnlessEqual
    % (msg, pformat(first), pformat(second)))
FailTest: not equal:
a = <DBItem CodeImportResultStatus.FAILURE, (200) Failure>
b = <DBItem CodeImportResultStatus.SUCCESS, (100) Success>
}}}
  * test_import_git (lp.codehosting.codeimport.tests.test_workermonitor.TestWorkerMonitorIntegrationScript) {{{
Failure in test test_import_git (lp.codehosting.codeimport.tests.test_workermonitor.TestWorkerMonitorIntegrationScript)
Traceback (most recent call last):
  File "/home/barry/projects/launchpad/python-migration2.6/lib/twisted/internet/defer.py", line 323, in _runCallbacks
    self.result = callback(self.result, *args, **kw)
  File "/home/barry/projects/launchpad/python-migration2.6/lib/lp/codehosting/codeimport/workermonitor.py", line 124, in transacted
    return function(*args, **kwargs)
  File "/home/barry/projects/launchpad/python-migration2.6/lib/lp/codehosting/codeimport/tests/test_workermonitor.py", line 531, in assertImported
    self.assertCodeImportResultCreated(code_import)
  File "/home/barry/projects/launchpad/python-migration2.6/lib/lp/codehosting/codeimport/tests/test_workermonitor.py", line 513, in assertCodeImportResultCreated
    self.assertEqual(result.status, CodeImportResultStatus.SUCCESS)
  File "/home/barry/projects/launchpad/python-migration2.6/lib/twisted/trial/unittest.py", line 290, in failUnlessEqual
    % (msg, pformat(first), pformat(second)))
FailTest: not equal:
a = <DBItem CodeImportResultStatus.FAILURE, (200) Failure>
b = <DBItem CodeImportResultStatus.SUCCESS, (100) Success>

Error in test test_import_git (lp.codehosting.codeimport.tests.test_workermonitor.TestWorkerMonitorIntegrationScript)
Traceback (most recent call last):
  File "/home/barry/projects/launchpad/python-migration2.6/lib/twisted/trial/unittest.py", line 1250, in run
    warnings.warn_explicit(**w)
PotentialZombieWarning: spawnProcess called, but the SIGCHLD handler is not installed. This probably means you have not yet called reactor.run, or called reactor.run(installSignalHandler=0). You will probably never see this process finish, and it may become a zombie process.
}}}
 * Running canonical.testing.layers.TwistedAppServerLayer tests:
  * lp.codehosting.puller.tests.test_scheduler.TestPullerMasterIntegration.test_lock_with_magic_id {{{
/home/barry/projects/launchpad/python-migration2.6/lib/cscvs/dircompare/dirsums.py:18: DeprecationWarning: the md5 module is deprecated; use hashlib instead
  import md5



Error in test lp.codehosting.puller.tests.test_scheduler.TestPullerMasterIntegration.test_lock_with_magic_id
Traceback (most recent call last):
UnexpectedStderr: Unexpected standard error from subprocess: import md5



Error in test lp.codehosting.puller.tests.test_scheduler.TestPullerMasterIntegration.test_lock_with_magic_id
Traceback (most recent call last):
  File "/home/barry/projects/launchpad/python-migration2.6/lib/twisted/trial/unittest.py", line 1250, in run
    warnings.warn_explicit(**w)
PotentialZombieWarning: spawnProcess called, but the SIGCHLD handler is not installed. This probably means you have not yet called reactor.run, or called reactor.run(installSignalHandler=0). You will probably never see this process finish, and it may become a zombie process.
}}}
  * lp.codehosting.puller.tests.test_scheduler.TestPullerMasterIntegration.test_mirror {{{
Error in test lp.codehosting.puller.tests.test_scheduler.TestPullerMasterIntegration.test_mirror
Traceback (most recent call last):
  File "/home/barry/projects/launchpad/python-migration2.6/lib/twisted/trial/unittest.py", line 1250, in run
    warnings.warn_explicit(**w)
PotentialZombieWarning: spawnProcess called, but the SIGCHLD handler is not installed. This probably means you have not yet called reactor.run, or called reactor.run(installSignalHandler=0). You will probably never see this process finish, and it may become a zombie process.


 lp.codehosting.puller.tests.test_scheduler.TestPullerMasterIntegration.test_mirror_with_destination_locked_by_another/home/barry/projects/launchpad/python-migration2.6/lib/cscvs/dircompare/dirsums.py:18: DeprecationWarning: the md5 module is deprecated; use hashlib instead
  import md5



Error in test lp.codehosting.puller.tests.test_scheduler.TestPullerMasterIntegration.test_mirror_with_destination_locked_by_another
Traceback (most recent call last):
UnexpectedStderr: Unexpected standard error from subprocess: import md5



Error in test lp.codehosting.puller.tests.test_scheduler.TestPullerMasterIntegration.test_mirror_with_destination_locked_by_another
Traceback (most recent call last):
  File "/home/barry/projects/launchpad/python-migration2.6/lib/twisted/trial/unittest.py", line 1250, in run
    warnings.warn_explicit(**w)
PotentialZombieWarning: spawnProcess called, but the SIGCHLD handler is not installed. This probably means you have not yet called reactor.run, or called reactor.run(installSignalHandler=0). You will probably never see this process finish, and it may become a zombie process.
}}}
  * lp.codehosting.puller.tests.test_scheduler.TestPullerMasterIntegration.test_mirror_with_destination_self_locked {{{

Error in test lp.codehosting.puller.tests.test_scheduler.TestPullerMasterIntegration.test_mirror_with_destination_self_locked
Traceback (most recent call last):
  File "/home/barry/projects/launchpad/python-migration2.6/lib/twisted/trial/unittest.py", line 1250, in run
    warnings.warn_explicit(**w)
PotentialZombieWarning: spawnProcess called, but the SIGCHLD handler is not installed. This probably means you have not yet called reactor.run, or called reactor.run(installSignalHandler=0). You will probably never see this process finish, and it may become a zombie process.
}}}
  * lp.codehosting.puller.tests.test_scheduler.TestPullerMasterIntegration.test_stderrLoggedToOOPS {{{
Error in test lp.codehosting.puller.tests.test_scheduler.TestPullerMasterIntegration.test_stderrLoggedToOOPS
Traceback (most recent call last):
  File "/home/barry/projects/launchpad/python-migration2.6/lib/twisted/trial/unittest.py", line 1250, in run
    warnings.warn_explicit(**w)
PotentialZombieWarning: spawnProcess called, but the SIGCHLD handler is not installed. This probably means you have not yet called reactor.run, or called reactor.run(installSignalHandler=0). You will probably never see this process finish, and it may become a zombie process.
}}}
 * Running canonical.testing.layers.ZopelessAppServerLayer tests:
  * lp.codehosting.puller.tests.test_acceptance.TestBranchPuller.test_mirror_hosted_loom_branch {{{
  C-c C-cTraceback (most recent call last):
  File "bin/test", line 246, in <module>
    result = testrunner.run([])
  File "/home/barry/projects/launchpad/eggs/zope.testing-3.8.1-py2.6.egg/zope/testing/testrunner/__init__.py", line 32, in run
    failed = run_internal(defaults, args, script_parts=script_parts)
  File "/home/barry/projects/launchpad/eggs/zope.testing-3.8.1-py2.6.egg/zope/testing/testrunner/__init__.py", line 45, in run_internal
    runner.run()
  File "/home/barry/projects/launchpad/eggs/zope.testing-3.8.1-py2.6.egg/zope/testing/testrunner/runner.py", line 136, in run
    self.run_tests()
  File "/home/barry/projects/launchpad/eggs/zope.testing-3.8.1-py2.6.egg/zope/testing/testrunner/runner.py", line 235, in run_tests
    layers_to_run, self.failures, self.errors)
  File "/home/barry/projects/launchpad/eggs/zope.testing-3.8.1-py2.6.egg/zope/testing/testrunner/runner.py", line 584, in resume_tests
    time.sleep(0.01) # Keep the loop from being too tight.
KeyboardInterrupt
}}}
(test run hung and aborted incomplete)

Moving to newer Python versions

With Lucid fast approaching, efforts to actually migrate Launchpad to Python 2.6 are getting under way.

Relevant tagged bugs: https://bugs.launchpad.net/launchpad-foundations/+bugs?field.tag=python-upgrade

Branches

Obsolete branches from the last push, which got us to 2.5

Remaining useful content of these branches:

  • shebang updates / Makefile PYTHON_VERSION updates all over the place -- redone in salgado's branch below

  • Deprecation warning suppression. -- cherrypicked into salgado's branch below

  • versions.cfg change: lazr.smtptest = 1.2 -- disregarded, there is no known reason why this was Python 2.6 related

The new effort

Known/Open issues

Today I've tried building a python-migration2.6-rebased branch (after merging devel) but it failed with a max recursion depth exceeded, very similar to what's described in http://bitbucket.org/tarek/distribute/issue/63/distribute-leaves-system-setuptools-in-inconsistent (although that's supposed to be fixed in our version of buildout). http://paste.ubuntu.com/409662/ is the actual error that I got. -- GuilhermeSalgado

Setup caveat

  • The extension modules in your sourcecode directory will have been built for Python 2.5 and these will break for your 2.6 branch. The simple thing to do is to move your old sourcecode aside and run utilities/rocketfuel-get to get it again with Python 2.6. A more complex but useful thing to do to manage sourcecode directories for multiple python versions is:

    cd ~/launchpad/lp-sourcedeps
    mkdir 2.6
    cd 2.6
    ln -s ../download-cache
    ln -s ../eggs
    mkdir sourcecode
    cd sourcecode
    for i in ../../sourcecode/*; do bzr checkout --lightweight "$i"; done

    and now point utilities/link-external-sourcecode to ../../lp-sourcedeps/2.6 - if you do that you'll need to be aware that rocketfuel-get won't know about it, and your extra directories will require occasional manual updating.

Cleanup potential

  • Is there any reason we need ipython-0.9.1-py2.5.egg in egg, not tarball, form?
    • Presumably we could bin the wadllib 0.1 eggs too.
  • Eventually we should get rid of all the scripts hardcoded /usr/bin/pythonX.Y and use buildout's shebang munging.

  • scripts/branch-rewrite.py uses /usr/bin/pythonX.Y -u. We should figure out if this is significant and why this script has to be different from other scripts.

  • scripts/ftpmaster-tools/_syncorigins.py uses /usr/bin/env pythonX.Y... why?!

  • cronscripts/mirror-prober.sh and cronscripts/nightly.sh are invoking python scripts with explicit interpreters. Why are they not using the shebangs? This leads to versions being defined in two places.

Situation

A multitude of modules of modules (sha, md5, sets, ...) are deprecated in Python 2.6, and many tests would fail merely because of unexpected DeprecationWarnings, so we've had to jump through multiple hoops all over the place to suppress these. Ultimately we need a plan for removing these suppressions, because we want to know about deprecation long-term.

Known test failures and notes thereon

  • Running canonical.testing.layers.BaseLayer tests:

    • test_import (lp.codehosting.codeimport.tests.test_worker.TestGitImport)

      Error in test test_import (lp.codehosting.codeimport.tests.test_worker.TestGitImport)
      Traceback (most recent call last):
        File "/home/barry/projects/launchpad/eggs/bzr-2.0.0_lp_1-py2.6-linux-x86_64.egg/bzrlib/tests/__init__.py", line 1470, in run
          testMethod()
        File "/home/barry/projects/launchpad/python-migration2.6/lib/lp/codehosting/codeimport/tests/test_worker.py", line 641, in test_import
          worker.run()
        File "/home/barry/projects/launchpad/python-migration2.6/lib/lp/codehosting/codeimport/worker.py", line 398, in run
          self._doImport()
        File "/home/barry/projects/launchpad/python-migration2.6/lib/lp/codehosting/codeimport/worker.py", line 495, in _doImport
          self.pushBazaarWorkingTree(bazaar_tree)
        File "/home/barry/projects/launchpad/python-migration2.6/lib/lp/codehosting/codeimport/worker.py", line 525, in pushBazaarWorkingTree
          'git.db', bazaar_tree.branch.repository._transport)
        File "/home/barry/projects/launchpad/python-migration2.6/lib/lp/codehosting/codeimport/worker.py", line 254, in put
          local_file = source_transport.get(filename)
        File "/home/barry/projects/launchpad/eggs/bzr-2.0.0_lp_1-py2.6-linux-x86_64.egg/bzrlib/transport/local.py", line 167, in get
          self._translate_error(e, path)
        File "/home/barry/projects/launchpad/eggs/bzr-2.0.0_lp_1-py2.6-linux-x86_64.egg/bzrlib/transport/__init__.py", line 306, in _translate_error
          raise errors.NoSuchFile(path, extra=e)
      NoSuchFile: No such file: u'/var/tmp/codeimport/data/worker-for-branch-1/bzr_working_tree/.bzr/repository/git.db': [Errno 2] No such file or directory: u'/var/tmp/codeimport/data/worker-for-branch-1/bzr_working_tree/.bzr/repository/git.db'
    • test_import_script (lp.codehosting.codeimport.tests.test_worker.TestGitImport)

      Failure in test test_import_script (lp.codehosting.codeimport.tests.test_worker.TestGitImport)
      Traceback (most recent call last):
        File "/home/barry/projects/launchpad/eggs/bzr-2.0.0_lp_1-py2.6-linux-x86_64.egg/bzrlib/tests/__init__.py", line 1470, in run
          testMethod()
        File "/home/barry/projects/launchpad/python-migration2.6/lib/lp/codehosting/codeimport/tests/test_worker.py", line 688, in test_import_script
          self.assertEqual(retcode, 0)
        File "/home/barry/projects/launchpad/eggs/bzr-2.0.0_lp_1-py2.6-linux-x86_64.egg/bzrlib/tests/__init__.py", line 958, in assertEqual
          pformat(a), pformat(b)))
      AssertionError: not equal:
      a = 1
      b = 0
    • test_sync (lp.codehosting.codeimport.tests.test_worker.TestGitImport)

      Error in test test_sync (lp.codehosting.codeimport.tests.test_worker.TestGitImport)
      Traceback (most recent call last):
        File "/home/barry/projects/launchpad/eggs/bzr-2.0.0_lp_1-py2.6-linux-x86_64.egg/bzrlib/tests/__init__.py", line 1470, in run
          testMethod()
        File "/home/barry/projects/launchpad/python-migration2.6/lib/lp/codehosting/codeimport/tests/test_worker.py", line 649, in test_sync
          worker.run()
        File "/home/barry/projects/launchpad/python-migration2.6/lib/lp/codehosting/codeimport/worker.py", line 398, in run
          self._doImport()
        File "/home/barry/projects/launchpad/python-migration2.6/lib/lp/codehosting/codeimport/worker.py", line 495, in _doImport
          self.pushBazaarWorkingTree(bazaar_tree)
        File "/home/barry/projects/launchpad/python-migration2.6/lib/lp/codehosting/codeimport/worker.py", line 525, in pushBazaarWorkingTree
          'git.db', bazaar_tree.branch.repository._transport)
        File "/home/barry/projects/launchpad/python-migration2.6/lib/lp/codehosting/codeimport/worker.py", line 254, in put
          local_file = source_transport.get(filename)
        File "/home/barry/projects/launchpad/eggs/bzr-2.0.0_lp_1-py2.6-linux-x86_64.egg/bzrlib/transport/local.py", line 167, in get
          self._translate_error(e, path)
        File "/home/barry/projects/launchpad/eggs/bzr-2.0.0_lp_1-py2.6-linux-x86_64.egg/bzrlib/transport/__init__.py", line 306, in _translate_error
          raise errors.NoSuchFile(path, extra=e)
      NoSuchFile: No such file: u'/var/tmp/codeimport/data/worker-for-branch-1/bzr_working_tree/.bzr/repository/git.db': [Errno 2] No such file or directory: u'/var/tmp/codeimport/data/worker-for-branch-1/bzr_working_tree/.bzr/repository/git.db'
  • Running canonical.testing.layers.LaunchpadScriptLayer tests:

    • test_initZopelessTwice (canonical.lp.ftests.test_zopeless.TestInitZopeless)

      Error in test test_initZopelessTwice (canonical.lp.ftests.test_zopeless.TestInitZopeless)
      Traceback (most recent call last):
        File "/usr/lib/python2.6/unittest.py", line 279, in run
          testMethod()
        File "/home/barry/projects/launchpad/python-migration2.6/lib/canonical/lp/ftests/test_zopeless.py", line 45, in test_initZopelessTwice
          dbuser='launchpad')
        File "/home/barry/projects/launchpad/python-migration2.6/lib/canonical/lp/__init__.py", line 84, in initZopeless
          dbname=dbname, dbhost=dbhost, dbuser=dbuser, isolation=isolation)
        File "/home/barry/projects/launchpad/python-migration2.6/lib/canonical/database/sqlbase.py", line 328, in initZopeless
          warnings.warn(alreadyInstalledMsg, stacklevel=3)
      UserWarning: A ZopelessTransactionManager with these settings is already installed.  This is probably caused by calling initZopeless twice.
  • Running canonical.testing.layers.LaunchpadZopelessLayer tests:

    • lib/lp/soyuz/tests/../doc/buildd-slave.txt

      Failure in test lib/lp/soyuz/tests/../doc/buildd-slave.txt
      Failed doctest test for buildd-slave.txt
        File "lib/lp/soyuz/tests/../doc/buildd-slave.txt", line 0
      
      ----------------------------------------------------------------------
      File "lib/lp/soyuz/tests/../doc/buildd-slave.txt", line 102, in buildd-slave.txt
      Failed example:
          s.info()
      Differences (ndiff with -expected +actual):
            Traceback (most recent call last):
          - ...
          +   File "/home/barry/projects/launchpad/eggs/zope.testing-3.8.1-py2.6.egg/zope/testing/doctest.py", line 1361, in __run
          +     compileflags, 1) in test.globs
          +   File "<doctest buildd-slave.txt[line 102, example 38]>", line 1, in <module>
          +   File "/usr/lib/python2.6/xmlrpclib.py", line 1199, in __call__
          +     return self.__send(self.__name, args)
          +   File "/usr/lib/python2.6/xmlrpclib.py", line 1489, in __request
          +     verbose=self.__verbose
          +   File "/usr/lib/python2.6/xmlrpclib.py", line 1235, in request
          +     self.send_content(h, request_body)
          +   File "/usr/lib/python2.6/xmlrpclib.py", line 1349, in send_content
          +     connection.endheaders()
          +   File "/usr/lib/python2.6/httplib.py", line 892, in endheaders
          +     self._send_output()
          +   File "/usr/lib/python2.6/httplib.py", line 764, in _send_output
          +     self.send(msg)
          +   File "/usr/lib/python2.6/httplib.py", line 723, in send
          +     self.connect()
          +   File "/usr/lib/python2.6/httplib.py", line 704, in connect
          +     self.timeout)
          +   File "/usr/lib/python2.6/socket.py", line 514, in create_connection
          +     raise error, msg
          - error: (111, 'Connection refused')
          ?        ^   ^^^                  --
          + error: [Errno 111] Connection refused
          ?        ^^^^^^^   ^^
    • lib/lp/soyuz/tests/../doc/manage-chroot.txt

      Failure in test lib/lp/soyuz/tests/../doc/manage-chroot.txt
      Failed doctest test for manage-chroot.txt
        File "lib/lp/soyuz/tests/../doc/manage-chroot.txt", line 0
      
      ----------------------------------------------------------------------
      File "lib/lp/soyuz/tests/../doc/manage-chroot.txt", line 189, in manage-chroot.txt
      Failed example:
          try:
               manage_chroot.mainTask()
          except SoyuzScriptError, info:
              print info
          else:
              print "Did not get expected exception"
      Differences (ndiff with -expected +actual):
          - DEBUG   Initialising ChrootManager for 'The Hoary Hedgehog Release for i386 (x86)'
          ?      --
          + DEBUG Initialising ChrootManager for 'The Hoary Hedgehog Release for i386 (x86)'
          - Librarian upload failed: [...]: (111, 'Connection refused')
          ?                           ^^^   ^   ^^^                  --
          + Librarian upload failed: [localhost:57930]: [Errno 111] Connection refused
          ?                           ^^^^^^^^^^^^^^^   ^^^^^^^   ^^
    • lib/canonical/launchpad/ftests/../doc/incomingmail.txt

      Failure in test lib/canonical/launchpad/ftests/../doc/incomingmail.txt
      Failed doctest test for incomingmail.txt
        File "lib/canonical/launchpad/ftests/../doc/incomingmail.txt", line 0
      
      ----------------------------------------------------------------------
      File "lib/canonical/launchpad/ftests/../doc/incomingmail.txt", line 377, in incomingmail.txt
      Failed example:
          handleMail(transaction)
      Differences (ndiff with -expected +actual):
          - ERROR:...:Upload to Librarian failed...
          ?       ^^^                           ---
          + ERROR:process-mail:Upload to Librarian failed
          ?       ^^^^^^^^^^^^
          - ...
          - UploadFailed: [...]: (111, 'Connection refused')
          + Traceback (most recent call last):
          +   File "/home/barry/projects/launchpad/python-migration2.6/lib/canonical/launchpad/mail/incoming.py", line 202, in handleMail
          +     cStringIO(raw_mail), 'message/rfc822')
          +   File "/home/barry/projects/launchpad/python-migration2.6/lib/canonical/launchpad/database/librarian.py", line 210, in create
          +     fid = client.addFile(name, size, file, contentType, expires, debugID)
          +   File "/home/barry/projects/launchpad/python-migration2.6/lib/canonical/librarian/client.py", line 104, in addFile
          +     self._connect()
          +   File "/home/barry/projects/launchpad/python-migration2.6/lib/canonical/librarian/client.py", line 56, in _connect
          +     '[%s:%s]: %s' % (self.upload_host, self.upload_port, x))
          + UploadFailed: [localhost:57930]: [Errno 111] Connection refused
          + ERROR:process-mail:Upload to Librarian failed
          + Traceback (most recent call last):
          +   File "/home/barry/projects/launchpad/python-migration2.6/lib/canonical/launchpad/mail/incoming.py", line 202, in handleMail
          +     cStringIO(raw_mail), 'message/rfc822')
          +   File "/home/barry/projects/launchpad/python-migration2.6/lib/canonical/launchpad/database/librarian.py", line 210, in create
          +     fid = client.addFile(name, size, file, contentType, expires, debugID)
          +   File "/home/barry/projects/launchpad/python-migration2.6/lib/canonical/librarian/client.py", line 104, in addFile
          +     self._connect()
          +   File "/home/barry/projects/launchpad/python-migration2.6/lib/canonical/librarian/client.py", line 56, in _connect
          +     '[%s:%s]: %s' % (self.upload_host, self.upload_port, x))
          + UploadFailed: [localhost:57930]: [Errno 111] Connection refused
  • Running canonical.testing.layers.LaunchpadFunctionalLayer tests:

    • Set up canonical.testing.layers.GoogleServiceLayer

      Traceback (most recent call last):
        File "/home/barry/projects/launchpad/eggs/zope.testing-3.8.1-py2.6.egg/zope/testing/testrunner/runner.py", line 364, in run_layer
          setup_layer(options, layer, setup_layers)
        File "/home/barry/projects/launchpad/eggs/zope.testing-3.8.1-py2.6.egg/zope/testing/testrunner/runner.py", line 624, in setup_layer
          setup_layer(options, base, setup_layers)
        File "/home/barry/projects/launchpad/eggs/zope.testing-3.8.1-py2.6.egg/zope/testing/testrunner/runner.py", line 629, in setup_layer
          layer.setUp()
        File "/home/barry/projects/launchpad/python-migration2.6/lib/canonical/testing/layers.py", line 1009, in setUp
          google.setUp()
        File "/home/barry/projects/launchpad/python-migration2.6/lib/canonical/launchpad/testing/tests/googleserviceharness.py", line 76, in setUp
          self.startService()
        File "/home/barry/projects/launchpad/python-migration2.6/lib/canonical/launchpad/testing/tests/googleserviceharness.py", line 88, in startService
          googletestservice.wait_for_service()
        File "/home/barry/projects/launchpad/python-migration2.6/lib/canonical/launchpad/testing/googletestservice.py", line 126, in wait_for_service
          sock.connect((host, port))
        File "<string>", line 1, in connect
      error: [Errno 103] Software caused connection abort
  • Running canonical.testing.layers.AppServerLayer tests:

    • Same layer failure as above
  • Running canonical.testing.layers.DatabaseFunctionalLayer tests:

    • lib/canonical/shipit/tests/../doc/shipit-process-requests.txt

      Failure in test lib/canonical/shipit/tests/../doc/shipit-process-requests.txt
      Failed doctest test for shipit-process-requests.txt
        File "lib/canonical/shipit/tests/../doc/shipit-process-requests.txt", line 0
      
      ----------------------------------------------------------------------
      File "lib/canonical/shipit/tests/../doc/shipit-process-requests.txt", line 96, in shipit-process-requests.txt
      Failed example:
          out, err
      Differences (ndiff with -expected +actual):
          - ('', '')
          + ('', '/home/barry/projects/launchpad/python-migration2.6/lib/cscvs/dircompare/dirsums.py:18: DeprecationWarning: the md5 module is deprecated; use hashlib instead\n  import md5\n/home/barry/projects/launchpad/python-migration2.6/lib/canonical/uuid.py:10: DeprecationWarning: the sha module is deprecated; use the hashlib module instead\n  import time, random, subprocess, thread, sha\n')
    • canonical.signon.browser.tests.test_openidserver.OpenIDMixin_checkTeamMembership_TestCase.test_full_fledged_account

      Error in test canonical.signon.browser.tests.test_openidserver.OpenIDMixin_checkTeamMembership_TestCase.test_full_fledged_account
      Traceback (most recent call last):
        File "/home/barry/projects/launchpad/python-migration2.6/lib/lp/testing/__init__.py", line 390, in run
          self.setUp()
        File "/home/barry/projects/launchpad/python-migration2.6/lib/canonical/signon/browser/tests/test_openidserver.py", line 268, in setUp
          self.openid_mixin = CheckTeamMembershipTest(None, self.request)
        File "/home/barry/projects/launchpad/python-migration2.6/lib/canonical/signon/browser/openidserver.py", line 101, in __init__
          super(OpenIDMixin, self).__init__(context, request)
      TypeError: object.__init__() takes no parameters
    • canonical.signon.browser.tests.test_openidserver.OpenIDMixin_checkTeamMembership_TestCase.test_personless_account

      Error in test canonical.signon.browser.tests.test_openidserver.OpenIDMixin_checkTeamMembership_TestCase.test_personless_account
      Traceback (most recent call last):
        File "/home/barry/projects/launchpad/python-migration2.6/lib/lp/testing/__init__.py", line 390, in run
          self.setUp()
        File "/home/barry/projects/launchpad/python-migration2.6/lib/canonical/signon/browser/tests/test_openidserver.py", line 268, in setUp
          self.openid_mixin = CheckTeamMembershipTest(None, self.request)
        File "/home/barry/projects/launchpad/python-migration2.6/lib/canonical/signon/browser/openidserver.py", line 101, in __init__
          super(OpenIDMixin, self).__init__(context, request)
      TypeError: object.__init__() takes no parameters
    • lots more of those
    • lib/canonical/signon/browser/tests/loginservice-unauthorized-view.txt

      Failure in test lib/canonical/signon/browser/tests/loginservice-unauthorized-view.txt
      Failed doctest test for loginservice-unauthorized-view.txt
        File "lib/canonical/signon/browser/tests/loginservice-unauthorized-view.txt", line 0
      
      ----------------------------------------------------------------------
      File "lib/canonical/signon/browser/tests/loginservice-unauthorized-view.txt", line 39, in loginservice-unauthorized-view.txt
      Failed example:
          browser.open('http://openid.launchpad.dev/+edit')
      Differences (ndiff with -expected +actual):
          - ERROR:...
  • Running canonical.testing.layers.PageTestLayer tests:

    • Google service layer failure again
  • Running canonical.testing.layers.TwistedLayer tests:

    • test_run_proces_with_timeout_success (canonical.twistedsupport.tests.test_processmonitor.TestRunProcessWithTimeout)

      Error in test test_run_proces_with_timeout_success (canonical.twistedsupport.tests.test_processmonitor.TestRunProcessWithTimeout)
      Traceback (most recent call last):
        File "/home/barry/projects/launchpad/python-migration2.6/lib/twisted/trial/unittest.py", line 1250, in run
          warnings.warn_explicit(**w)
      PotentialZombieWarning: spawnProcess called, but the SIGCHLD handler is not installed. This probably means you have not yet called reactor.run, or called reactor.run(installSignalHandler=0). You will probably never see this process finish, and it may become a zombie process.
    • more of the same
  • Running canonical.testing.layers.TwistedLaunchpadZopelessLayer tests:

    • test_import_git (lp.codehosting.codeimport.tests.test_workermonitor.TestWorkerMonitorIntegration)

      Failure in test test_import_git (lp.codehosting.codeimport.tests.test_workermonitor.TestWorkerMonitorIntegration)
      Traceback (most recent call last):
        File "/home/barry/projects/launchpad/python-migration2.6/lib/twisted/internet/defer.py", line 323, in _runCallbacks
          self.result = callback(self.result, *args, **kw)
        File "/home/barry/projects/launchpad/python-migration2.6/lib/lp/codehosting/codeimport/workermonitor.py", line 124, in transacted
          return function(*args, **kwargs)
        File "/home/barry/projects/launchpad/python-migration2.6/lib/lp/codehosting/codeimport/tests/test_workermonitor.py", line 531, in assertImported
          self.assertCodeImportResultCreated(code_import)
        File "/home/barry/projects/launchpad/python-migration2.6/lib/lp/codehosting/codeimport/tests/test_workermonitor.py", line 513, in assertCodeImportResultCreated
          self.assertEqual(result.status, CodeImportResultStatus.SUCCESS)
        File "/home/barry/projects/launchpad/python-migration2.6/lib/twisted/trial/unittest.py", line 290, in failUnlessEqual
          % (msg, pformat(first), pformat(second)))
      FailTest: not equal:
      a = <DBItem CodeImportResultStatus.FAILURE, (200) Failure>
      b = <DBItem CodeImportResultStatus.SUCCESS, (100) Success>
    • test_import_git (lp.codehosting.codeimport.tests.test_workermonitor.TestWorkerMonitorIntegrationScript)

      Failure in test test_import_git (lp.codehosting.codeimport.tests.test_workermonitor.TestWorkerMonitorIntegrationScript)
      Traceback (most recent call last):
        File "/home/barry/projects/launchpad/python-migration2.6/lib/twisted/internet/defer.py", line 323, in _runCallbacks
          self.result = callback(self.result, *args, **kw)
        File "/home/barry/projects/launchpad/python-migration2.6/lib/lp/codehosting/codeimport/workermonitor.py", line 124, in transacted
          return function(*args, **kwargs)
        File "/home/barry/projects/launchpad/python-migration2.6/lib/lp/codehosting/codeimport/tests/test_workermonitor.py", line 531, in assertImported
          self.assertCodeImportResultCreated(code_import)
        File "/home/barry/projects/launchpad/python-migration2.6/lib/lp/codehosting/codeimport/tests/test_workermonitor.py", line 513, in assertCodeImportResultCreated
          self.assertEqual(result.status, CodeImportResultStatus.SUCCESS)
        File "/home/barry/projects/launchpad/python-migration2.6/lib/twisted/trial/unittest.py", line 290, in failUnlessEqual
          % (msg, pformat(first), pformat(second)))
      FailTest: not equal:
      a = <DBItem CodeImportResultStatus.FAILURE, (200) Failure>
      b = <DBItem CodeImportResultStatus.SUCCESS, (100) Success>
      
      Error in test test_import_git (lp.codehosting.codeimport.tests.test_workermonitor.TestWorkerMonitorIntegrationScript)
      Traceback (most recent call last):
        File "/home/barry/projects/launchpad/python-migration2.6/lib/twisted/trial/unittest.py", line 1250, in run
          warnings.warn_explicit(**w)
      PotentialZombieWarning: spawnProcess called, but the SIGCHLD handler is not installed. This probably means you have not yet called reactor.run, or called reactor.run(installSignalHandler=0). You will probably never see this process finish, and it may become a zombie process.
  • Running canonical.testing.layers.TwistedAppServerLayer tests:

    • lp.codehosting.puller.tests.test_scheduler.TestPullerMasterIntegration.test_lock_with_magic_id

      /home/barry/projects/launchpad/python-migration2.6/lib/cscvs/dircompare/dirsums.py:18: DeprecationWarning: the md5 module is deprecated; use hashlib instead
        import md5
      
      
      
      Error in test lp.codehosting.puller.tests.test_scheduler.TestPullerMasterIntegration.test_lock_with_magic_id
      Traceback (most recent call last):
      UnexpectedStderr: Unexpected standard error from subprocess:   import md5
      
      
      
      Error in test lp.codehosting.puller.tests.test_scheduler.TestPullerMasterIntegration.test_lock_with_magic_id
      Traceback (most recent call last):
        File "/home/barry/projects/launchpad/python-migration2.6/lib/twisted/trial/unittest.py", line 1250, in run
          warnings.warn_explicit(**w)
      PotentialZombieWarning: spawnProcess called, but the SIGCHLD handler is not installed. This probably means you have not yet called reactor.run, or called reactor.run(installSignalHandler=0). You will probably never see this process finish, and it may become a zombie process.
    • lp.codehosting.puller.tests.test_scheduler.TestPullerMasterIntegration.test_mirror

      Error in test lp.codehosting.puller.tests.test_scheduler.TestPullerMasterIntegration.test_mirror
      Traceback (most recent call last):
        File "/home/barry/projects/launchpad/python-migration2.6/lib/twisted/trial/unittest.py", line 1250, in run
          warnings.warn_explicit(**w)
      PotentialZombieWarning: spawnProcess called, but the SIGCHLD handler is not installed. This probably means you have not yet called reactor.run, or called reactor.run(installSignalHandler=0). You will probably never see this process finish, and it may become a zombie process.
      
      
       lp.codehosting.puller.tests.test_scheduler.TestPullerMasterIntegration.test_mirror_with_destination_locked_by_another/home/barry/projects/launchpad/python-migration2.6/lib/cscvs/dircompare/dirsums.py:18: DeprecationWarning: the md5 module is deprecated; use hashlib instead
        import md5
      
      
      
      Error in test lp.codehosting.puller.tests.test_scheduler.TestPullerMasterIntegration.test_mirror_with_destination_locked_by_another
      Traceback (most recent call last):
      UnexpectedStderr: Unexpected standard error from subprocess:   import md5
      
      
      
      Error in test lp.codehosting.puller.tests.test_scheduler.TestPullerMasterIntegration.test_mirror_with_destination_locked_by_another
      Traceback (most recent call last):
        File "/home/barry/projects/launchpad/python-migration2.6/lib/twisted/trial/unittest.py", line 1250, in run
          warnings.warn_explicit(**w)
      PotentialZombieWarning: spawnProcess called, but the SIGCHLD handler is not installed. This probably means you have not yet called reactor.run, or called reactor.run(installSignalHandler=0). You will probably never see this process finish, and it may become a zombie process.
    • lp.codehosting.puller.tests.test_scheduler.TestPullerMasterIntegration.test_mirror_with_destination_self_locked

      Error in test lp.codehosting.puller.tests.test_scheduler.TestPullerMasterIntegration.test_mirror_with_destination_self_locked
      Traceback (most recent call last):
        File "/home/barry/projects/launchpad/python-migration2.6/lib/twisted/trial/unittest.py", line 1250, in run
          warnings.warn_explicit(**w)
      PotentialZombieWarning: spawnProcess called, but the SIGCHLD handler is not installed. This probably means you have not yet called reactor.run, or called reactor.run(installSignalHandler=0). You will probably never see this process finish, and it may become a zombie process.
    • lp.codehosting.puller.tests.test_scheduler.TestPullerMasterIntegration.test_stderrLoggedToOOPS

      Error in test lp.codehosting.puller.tests.test_scheduler.TestPullerMasterIntegration.test_stderrLoggedToOOPS
      Traceback (most recent call last):
        File "/home/barry/projects/launchpad/python-migration2.6/lib/twisted/trial/unittest.py", line 1250, in run
          warnings.warn_explicit(**w)
      PotentialZombieWarning: spawnProcess called, but the SIGCHLD handler is not installed. This probably means you have not yet called reactor.run, or called reactor.run(installSignalHandler=0). You will probably never see this process finish, and it may become a zombie process.
  • Running canonical.testing.layers.ZopelessAppServerLayer tests:

    • lp.codehosting.puller.tests.test_acceptance.TestBranchPuller.test_mirror_hosted_loom_branch

        C-c C-cTraceback (most recent call last):
        File "bin/test", line 246, in <module>
          result = testrunner.run([])
        File "/home/barry/projects/launchpad/eggs/zope.testing-3.8.1-py2.6.egg/zope/testing/testrunner/__init__.py", line 32, in run
          failed = run_internal(defaults, args, script_parts=script_parts)
        File "/home/barry/projects/launchpad/eggs/zope.testing-3.8.1-py2.6.egg/zope/testing/testrunner/__init__.py", line 45, in run_internal
          runner.run()
        File "/home/barry/projects/launchpad/eggs/zope.testing-3.8.1-py2.6.egg/zope/testing/testrunner/runner.py", line 136, in run
          self.run_tests()
        File "/home/barry/projects/launchpad/eggs/zope.testing-3.8.1-py2.6.egg/zope/testing/testrunner/runner.py", line 235, in run_tests
          layers_to_run, self.failures, self.errors)
        File "/home/barry/projects/launchpad/eggs/zope.testing-3.8.1-py2.6.egg/zope/testing/testrunner/runner.py", line 584, in resume_tests
          time.sleep(0.01) # Keep the loop from being too tight.
      KeyboardInterrupt

(test run hung and aborted incomplete)