launchpad development|
Size: 2185
Comment: extend
|
Size: 2774
Comment: fix env variable name
|
| Deletions are marked like this. | Additions are marked like this. |
| Line 29: | Line 29: |
| will let testr run tests in a temporary container. (e.g. testr -- -t stories/gpg will fire up an aufs container and run the stories/gpg tests inside it). We need a new bin/test though, as --list --subunit currently outputs subunit rather than just a list of tests, which isn't what we need.. | will let testr run tests in a temporary container. (e.g. testr -- -t stories/gpg will fire up an aufs container and run the stories/gpg tests inside it). We need a new bin/test though, as --list --subunit currently outputs subunit rather than just a list of tests, which isn't what we need.. Be sure to export LP_LXC_BASE with the name of your lxc base container. |
| Line 36: | Line 39: |
== Workflow == * You need a temp directory in your source tree to workaround bug Bug:808557 {{{ mkdir temp }}} * Edit outside the container in your normal work area * Start the base container to do maintenance: make schema, bin/buildout {{{ lxc-start -n $basename -d }}} * ssh to it * and within it - {{{sudo poweroff -n}}} * Run tests with testr: All tests {{{ TEMP=$(pwd)/temp testr run --parallel }}} Some tests {{{ TEMP=$(pwd)/temp testr run --parallel -- -t stories/gpg }}} |
Overview
Parallel testing would be nice. Theres a bunch of things to do to make it work. See the LEP for constraints/goals/resourcing.
Known bugs/issues: parallel test bugs
Design sketch
- bin/test --parallel runs N test runners with subunit, where N is the number of cores, and the tests are partitioned across runners. (implemented)
- Layers dynamically allocate/deallocate resources such as:
- dbnames (implemented)
- config files (implemented)
- librarian work dir
- librarian ports
- keyserver work area
- soyuz work area
Things that need specialist knowledge:
- dynamically allocating ports for zope - port 8085 and 9025 specifically, which can then be fed back into e.g. zcml files/launchpad.conf.
- Buildmaster slave tests hard code the xmlrpc port to 8221 everywhere.
LXC containers and parallel testing
LXC containers combined with aufs offer a pretty cheap way to get solid isolation - a great big hammer of a workaround for our existing globals (shared work dirs etc). William has put together a proof of concept, and Robert has made that generic. That combined with an updated .testr.conf (a TODO is to offer profiles for testr) like:
[DEFAULT] test_command=lxc-start-aufs $LP_LXC_BASE $PWD xvfb-run $PWD/bin/test --subunit $IDOPTION $LISTOPT test_id_option=--load-list $IDFILE test_list_option=--list
will let testr run tests in a temporary container. (e.g. testr -- -t stories/gpg will fire up an aufs container and run the stories/gpg tests inside it). We need a new bin/test though, as --list --subunit currently outputs subunit rather than just a list of tests, which isn't what we need..
Be sure to export LP_LXC_BASE with the name of your lxc base container.
Caveats
- If the base container is running it will be a disaster. Don't try.
- aufs does not seem to permit deletes in some circumstances, so test fixtures which start by deleting a directory tree will fail if the directory tree exists. Known cases:
- /var/tmp/testkeyserver.test
Workflow
You need a temp directory in your source tree to workaround bug 808557
mkdir temp
- Edit outside the container in your normal work area
- Start the base container to do maintenance: make schema, bin/buildout
lxc-start -n $basename -d
- ssh to it
and within it - sudo poweroff -n
- Run tests with testr:
All tests
TEMP=$(pwd)/temp testr run --parallel
Some tests
TEMP=$(pwd)/temp testr run --parallel -- -t stories/gpg
