launchpad development|
Size: 822
Comment:
|
Size: 2697
Comment: more tweaks
|
| Deletions are marked like this. | Additions are marked like this. |
| Line 1: | Line 1: |
| = Overview = | |
| Line 6: | Line 7: |
| Design sketch: * bin/test --parallel runs N test runners with subunit, where N is the number of cores, and the tests are partitioned across runners. |
= Design sketch = * bin/test --parallel runs N test runners with subunit, where N is the number of cores, and the tests are partitioned across runners. (implemented) |
| Line 9: | Line 10: |
| * dbnames * config files |
* dbnames (implemented) * config files (implemented) |
| Line 13: | Line 14: |
| * keyserver work area * soyuz work area |
|
| Line 17: | Line 20: |
= LXC containers and parallel testing = LXC containers combined with aufs offer a pretty cheap way to get solid isolation - a great big hammer of a workaround for our existing globals (shared work dirs etc). William has put together a proof of concept, and Robert has made that [[https://bugs.launchpad.net/ubuntu/+source/lxc/+bug/807351/+attachment/2196930/+files/lxc-start-aufs|generic]]. That combined with an updated .testr.conf (a TODO is to offer profiles for testr) like: {{{ [DEFAULT] test_command=lxc-start-aufs $LP_LXC_BASE $PWD xvfb-run $PWD/bin/test --subunit $IDOPTION $LISTOPT test_id_option=--load-list $IDFILE test_list_option=--list }}} will let testr run tests in a temporary container. (e.g. testr -- -t stories/gpg will fire up an aufs container and run the stories/gpg tests inside it). We need a new bin/test though, as --list --subunit currently outputs subunit rather than just a list of tests, which isn't what we need.. == Caveats == * If the base container is running it will be a disaster. Don't try. * aufs does not seem to permit deletes in some circumstances, so test fixtures which start by deleting a directory tree will fail if the directory tree exists. Known cases: * /var/tmp/testkeyserver.test == Workflow == * You need a temp directory in your source tree to workaround bug Bug:808557 {{{ mkdir temp }}} * Edit outside the container in your normal work area * Start the base container to do maintenance: make schema, bin/buildout {{{ lxc-start -n $basename -d }}} * ssh to it * and within it - {{{sudo poweroff -n}}} * Run tests with testr: All tests {{{ TEMP=$(pwd)/temp testr run --parallel }}} Some tests {{{ TEMP=$(pwd)/temp testr run --parallel -- -t stories/gpg }}} |
Overview
Parallel testing would be nice. Theres a bunch of things to do to make it work. See the LEP for constraints/goals/resourcing.
Known bugs/issues: parallel test bugs
Design sketch
- bin/test --parallel runs N test runners with subunit, where N is the number of cores, and the tests are partitioned across runners. (implemented)
- Layers dynamically allocate/deallocate resources such as:
- dbnames (implemented)
- config files (implemented)
- librarian work dir
- librarian ports
- keyserver work area
- soyuz work area
Things that need specialist knowledge:
- dynamically allocating ports for zope - port 8085 and 9025 specifically, which can then be fed back into e.g. zcml files/launchpad.conf.
- Buildmaster slave tests hard code the xmlrpc port to 8221 everywhere.
LXC containers and parallel testing
LXC containers combined with aufs offer a pretty cheap way to get solid isolation - a great big hammer of a workaround for our existing globals (shared work dirs etc). William has put together a proof of concept, and Robert has made that generic. That combined with an updated .testr.conf (a TODO is to offer profiles for testr) like:
[DEFAULT] test_command=lxc-start-aufs $LP_LXC_BASE $PWD xvfb-run $PWD/bin/test --subunit $IDOPTION $LISTOPT test_id_option=--load-list $IDFILE test_list_option=--list
will let testr run tests in a temporary container. (e.g. testr -- -t stories/gpg will fire up an aufs container and run the stories/gpg tests inside it). We need a new bin/test though, as --list --subunit currently outputs subunit rather than just a list of tests, which isn't what we need..
Caveats
- If the base container is running it will be a disaster. Don't try.
- aufs does not seem to permit deletes in some circumstances, so test fixtures which start by deleting a directory tree will fail if the directory tree exists. Known cases:
- /var/tmp/testkeyserver.test
Workflow
You need a temp directory in your source tree to workaround bug 808557
mkdir temp
- Edit outside the container in your normal work area
- Start the base container to do maintenance: make schema, bin/buildout
lxc-start -n $basename -d
- ssh to it
and within it - sudo poweroff -n
- Run tests with testr:
All tests
TEMP=$(pwd)/temp testr run --parallel
Some tests
TEMP=$(pwd)/temp testr run --parallel -- -t stories/gpg
