Overview

Parallel testing would be nice. Theres a bunch of things to do to make it work. See the LEP for constraints/goals/resourcing.

Known bugs/issues: parallel test bugs

Design sketch

• bin/test --parallel runs N test runners with subunit, where N is the number of cores, and the tests are partitioned across runners. (implemented)
• Layers dynamically allocate/deallocate resources such as:
  • dbnames (implemented)
  • config files (implemented)
  • librarian work dir
  • librarian ports
  • keyserver work area
  • soyuz work area

Things that need specialist knowledge:

• dynamically allocating ports for zope - port 8085 and 9025 specifically, which can then be fed back into e.g. zcml files/launchpad.conf.
• Buildmaster slave tests hard code the xmlrpc port to 8221 everywhere.

LXC containers and parallel testing

LXC containers combined with aufs offer a pretty cheap way to get solid isolation - a great big hammer of a workaround for our existing globals (shared work dirs etc). William has put together a proof of concept, and Robert has made that generic. That combined with an updated .testr.conf (a TODO is to offer profiles for testr) like:

[DEFAULT]
test_command=lxc-start-aufs $LP_LXC_BASE $PWD xvfb-run $PWD/bin/test --subunit $IDOPTION $LISTOPT
test_id_option=--load-list $IDFILE
test_list_option=--list

will let testr run tests in a temporary container. (e.g. testr -- -t stories/gpg will fire up an aufs container and run the stories/gpg tests inside it). We need a new bin/test though, as --list --subunit currently outputs subunit rather than just a list of tests, which isn't what we need..

Be sure to export LP_LXC_BASE with the name of your lxc base container.

Caveats

• If the base container is running it will be a disaster. Don't try.
• aufs does not seem to permit deletes in some circumstances, so test fixtures which start by deleting a directory tree will fail if the directory tree exists. Known cases:
  • /var/tmp/testkeyserver.test

Workflow

• You need a temp directory in your source tree to workaround bug 808557

mkdir temp

• Edit outside the container in your normal work area
• Start the base container to do maintenance: make schema, bin/buildout

lxc-start -n $basename -d

• ssh to it

• and within it - sudo poweroff -n

• Run tests with testr:

All tests

TEMP=$(pwd)/temp testr run --parallel

Some tests

TEMP=$(pwd)/temp testr run --parallel -- -t stories/gpg