2562
Comment: fix
|
2697
more tweaks
|
Deletions are marked like this. | Additions are marked like this. |
Line 39: | Line 39: |
* You need a temp directory in your source tree to workaround bug Bug:808557 {{{ mkdir temp }}} |
|
Line 49: | Line 53: |
testr run --parallel | TEMP=$(pwd)/temp testr run --parallel |
Line 53: | Line 57: |
testr run --parallel -- -t stories/gpg | TEMP=$(pwd)/temp testr run --parallel -- -t stories/gpg |
Overview
Parallel testing would be nice. Theres a bunch of things to do to make it work. See the LEP for constraints/goals/resourcing.
Known bugs/issues: parallel test bugs
Design sketch
- bin/test --parallel runs N test runners with subunit, where N is the number of cores, and the tests are partitioned across runners. (implemented)
- Layers dynamically allocate/deallocate resources such as:
- dbnames (implemented)
- config files (implemented)
- librarian work dir
- librarian ports
- keyserver work area
- soyuz work area
Things that need specialist knowledge:
- dynamically allocating ports for zope - port 8085 and 9025 specifically, which can then be fed back into e.g. zcml files/launchpad.conf.
- Buildmaster slave tests hard code the xmlrpc port to 8221 everywhere.
LXC containers and parallel testing
LXC containers combined with aufs offer a pretty cheap way to get solid isolation - a great big hammer of a workaround for our existing globals (shared work dirs etc). William has put together a proof of concept, and Robert has made that generic. That combined with an updated .testr.conf (a TODO is to offer profiles for testr) like:
[DEFAULT] test_command=lxc-start-aufs $LP_LXC_BASE $PWD xvfb-run $PWD/bin/test --subunit $IDOPTION $LISTOPT test_id_option=--load-list $IDFILE test_list_option=--list
will let testr run tests in a temporary container. (e.g. testr -- -t stories/gpg will fire up an aufs container and run the stories/gpg tests inside it). We need a new bin/test though, as --list --subunit currently outputs subunit rather than just a list of tests, which isn't what we need..
Caveats
- If the base container is running it will be a disaster. Don't try.
- aufs does not seem to permit deletes in some circumstances, so test fixtures which start by deleting a directory tree will fail if the directory tree exists. Known cases:
- /var/tmp/testkeyserver.test
Workflow
You need a temp directory in your source tree to workaround bug 808557
mkdir temp
- Edit outside the container in your normal work area
- Start the base container to do maintenance: make schema, bin/buildout
lxc-start -n $basename -d
- ssh to it
and within it - sudo poweroff -n
- Run tests with testr:
All tests
TEMP=$(pwd)/temp testr run --parallel
Some tests
TEMP=$(pwd)/temp testr run --parallel -- -t stories/gpg