= Logs of successful/near successful runs = || Who || Log file || Date || Launchpad revno || Manual tweaks || Known issues || New issues || Subunit || Build (test) time in minutes:seconds || || bac || [[http://paste.ubuntu.com/932523/| paste 932523]] || 2012-04-16 || r15102 || [[https://bugs.launchpad.net/launchpad/+bug/974584/comments/1|fstab tweak for bug 974584]] || Bug:974585 || || || || || frankban || [[http://pastebin.ubuntu.com/934203/| paste 934203]] || 2012-04-17 || r15108 || [[https://bugs.launchpad.net/launchpad/+bug/974584/comments/1|fstab tweak for bug 974584]] || Bug:974585 || Bug:984312 Bug:849056 Bug:974617 || || || || gary_poster || [[http://pastebin.ubuntu.com/936255/| paste 936255]] || 2012-04-18 || r15120 || - || Bug:974585 Bug:849056 || Bug:985338 || || || || bac || [[http://pastebin.ubuntu.com/937602/|paste 937602]] || 2012-04-19 || ?r15127? || - || Bug:849056 || || [[http://dl.dropbox.com/u/420990/testrepo-0|subunit]] || || || gary_poster || [[http://pastebin.ubuntu.com/942814/|pastebin]] || 2012-04-23 || r15138 || - || - || Bug:987441 Bug:987442 Bug:987444 (setup) || [[http://ubuntuone.com/2ToMVXFzwiYP2LUNG2CGbN|subunit]] || || || gary_poster || [[http://pastebin.ubuntu.com/943063/|pastebin]] || 2012-04-23 || r15139 || - || Bug:849056 || Bug:987499 || [[http://ubuntuone.com/2rMfcHi2RuUXJGRSGZZfzS|subunit]] || || || bac || [[http://ubuntuone.com/4MVqmWWoYUcD87uPXzqSKa|0.txt]] || 2012-04-23 || [[https://code.launchpad.net/~bac/launchpad/bug-974617-2 | bug-974617-2 r15142]] || - || Bug:849056 || ||[[http://ubuntuone.com/3p3PnFv2ypEhk6vMnRfoTO| U1 link]] || || || bac || [[http://ubuntuone.com/0vEmgzhRffrkwCYs7m0sAz|1.txt]] || 2012-04-23 || [[https://code.launchpad.net/~bac/launchpad/bug-974617-2 | bug-974617-2 r15142]] || - || || Bug:987898 ||[[http://ubuntuone.com/0orhaXlGtO7WaZSf4ImKwf| U1 link]] || || || bac || [[http://ubuntuone.com/4DHw1kNXhfYsg6T9XAMv2a|2.txt]] || 2012-04-23 || [[https://code.launchpad.net/~bac/launchpad/bug-974617-2 | bug-974617-2 r15142]] || - || Bug:987898|| ||[[http://ubuntuone.com/4NmxmxUhKvrzxe41K2hEnA| U1 link]] || || || bac || [[http://ubuntuone.com/5bh7zSW1m4JSPBGXjdrBSW|3.txt]] || 2012-04-23 || [[https://code.launchpad.net/~bac/launchpad/bug-974617-2 | bug-974617-2 r15142]] || - || || Bug:987903, Bug:987904 || [[http://ubuntuone.com/6ZuX4zRyZELzmGvpGk8Bln| U1 link]] || || || bac || [[http://ubuntuone.com/2ERGegrtSCcNth17wL34SF|4.txt]] || 2012-04-23 || [[https://code.launchpad.net/~bac/launchpad/bug-974617-2 | bug-974617-2 r15142]] || - || || \0/ - clean run! ||[[http://ubuntuone.com/6ZuX4zRyZELzmGvpGk8Bln| U1 link]] || || || benji || - || 2012-04-27 || - || - || || clean run! || || || || benji || - || 2012-04-27 || - || - || || another clean run || || || || gary_poster || || 2012-04-30 || r15176 || || || clean run || || || || gary_poster || || 2012-04-30 || r15177 || || || Bug:992184 || (worker test list is in bug) || || || gary_poster || || 2012-04-30 || r15178 || || || clean run || || || || gary_poster || || 2012-04-30 || r15179 || || || clean run || || || || gary_poster || || 2012-04-30 || r15180 || || || clean run || || || || gary_poster || || 2012-04-30 || r15181 || || || --(Bug:992692)-- || (worker test list is in bug) || || || gary_poster || || 2012-04-30 || r15181 || || || clean run || || || || gary_poster || || 2012-05-01 || r15184 || || --(Bug:992692)-- || || || || || gary_poster || || 2012-05-01 || r15184 || || || clean run || || || || gary_poster || || 2012-05-01 || r15184 || || || clean run || || || || gary_poster || || 2012-05-01 || r15185 || || || Bug:992814 || || || || gary_poster || || 2012-05-01 || r15185 || || || clean run || || || || gary_poster || || 2012-05-01 || r15185 || || || clean run || || || || gary_poster || || 2012-05-01 || r15185 || || || --(Bug:992958)-- || || || || gary_poster || || 2012-05-02 || r15188 || || || clean run || || || || gary_poster || || 2012-05-02 || r15188 || || || clean run || || || || gary_poster || || 2012-05-02 || r15189 || || --(Bug:992692)-- || || || || || gary_poster || || 2012-05-02 || r15189 || || --(Bug:992692)-- || || || || || gary_poster || || 2012-05-02 || r15189 || || || clean run? Only 4038 tests! || Lost subunit output :-( || || || gary_poster || || 2012-05-03 || r15195 || || || --(Bug:994158)-- || || || || gary_poster || || 2012-05-03 || r15195 || || Bug:992184 --(Bug:987499)-- || || || || || gary_poster || || 2012-05-03 || r15195 || || || clean run || || || || gary_poster || || 2012-05-03 || r15195 || || Bug:992184 || || || || || gary_poster || || 2012-05-03 || r15196 || || || clean run || || || || gary_poster || || 2012-05-04 || r15206 || || || --(Bug:994602)-- (Bug:994752) || || || || gary_poster || || 2012-05-04 || r15207 || || --(Bug:994602)-- || || || || || gary_poster || || 2012-05-04 || r15207 || || --(Bug:994602)-- || || || || || gary_poster || || 2012-05-04 || r15208 || || || --(Bug:994752)-- Bug:1014916 || || || || gary_poster || || 2012-05-04 || r15208 || Increased lxc-start-ephemeral timeout to get IP from 30 to 60 for Bug:994752 || || clean run || || || || gary_poster || || 2012-05-04 || r15208 || Increased lxc-start-ephemeral timeout || || Bug:994777 || || || || gary_poster || || 2012-05-04 || r15208 || Increased lxc-start-ephemeral timeout || || clean run || || || || gary_poster || || 2012-05-04 || r15209 || Increased lxc-start-ephemeral timeout || || clean run || || || || gary_poster || || 2012-05-07 || r15210 || Increased lxc-start-ephemeral timeout, PG 9.1 || || clean run || || || || gary_poster || || 2012-05-07 || r15211 || Increased lxc-start-ephemeral timeout, PG 9.1 || || clean run || || || || gary_poster || || 2012-05-07 || r15211 || Increased lxc-start-ephemeral timeout, PG 9.1 || || clean run || || || || gary_poster || || 2012-05-07 || r15211 || Increased lxc-start-ephemeral timeout, PG 9.1 || || clean run || || || || gary_poster || || 2012-05-07 || r15212 || Increased lxc-start-ephemeral timeout, PG 9.1 || || clean run || || || || gary_poster || || 2012-05-07 || r15212 || Increased lxc-start-ephemeral timeout, PG 9.1 || || clean run || || || || gary_poster || || 2012-05-07 || r15212 || Increased lxc-start-ephemeral timeout, PG 9.1 || || clean run || || || || gary_poster || || 2012-05-07 || r15212 || Increased lxc-start-ephemeral timeout, PG 9.1 || || clean run || || || || gary_poster || || 2012-05-07 || r15212 || Increased lxc-start-ephemeral timeout, PG 9.1 || || clean run || || || || gary_poster || || 2012-05-08 || r15213 || Increased lxc-start-ephemeral timeout, PG 9.1 || || clean run (12 in a row) || || || || gary_poster || || 2012-05-08 || r15213 || Increased lxc-start-ephemeral timeout, PG 9.1 || Bug:974617 || || || || || gary_poster || || 2012-05-08 || r15213 || Increased lxc-start-ephemeral timeout, PG 9.1 || || Bug:996729 (subunit stream bug: Bug:996773) || || || || gary_poster || || 2012-05-08 || r15213 || Increased lxc-start-ephemeral timeout, PG 9.1 || || clean run || || || || frankban || || 2012-05-08 || r15213 || lpsetup (includes PG 9.1) || || clean run || || || || frankban || || 2012-05-08 || r15213 || lpsetup (includes PG 9.1) || || clean run || || || || frankban || || 2012-05-08 || r15214 || lpsetup (includes PG 9.1) || || Bug:996720 || || || || frankban || || 2012-05-08 || r15214 || lpsetup (includes PG 9.1) || || clean run || || || || gary_poster || || 2012-05-08 || r15214 || Increased lxc-start-ephemeral timeout, PG 9.1 || || clean run || || || || gary_poster || || 2012-05-08 || r15215 || Increased lxc-start-ephemeral timeout, PG 9.1 || || clean run || || || || gary_poster || || 2012-05-08 || r15215 || Increased lxc-start-ephemeral timeout, PG 9.1 || Bug:996729 || || || || || frankban || || 2012-05-09 || r15218 || lpsetup (includes PG 9.1) || Bug:994777 || || || || || frankban || || 2012-05-09 || r15218 || lpsetup (includes PG 9.1) || || clean run || || || || frankban || || 2012-05-09 || r15218 || lpsetup (includes PG 9.1), lxc-start-ephemeral uses lp-lxc-ip || || clean run || || || || frankban || || 2012-05-09 || r15219 || lpsetup (includes PG 9.1), lxc-start-ephemeral uses lp-lxc-ip || || clean run || || || || frankban || || 2012-05-09 || r15220 || lpsetup (includes PG 9.1), lxc-start-ephemeral uses lp-lxc-ip || || clean run || || || || gary_poster || || 2012-05-09 || r15219 || PG 9.1 || Bug:996773 || || || || || gary_poster || || 2012-05-09 || r15220 || PG 9.1 || Bug:974617 || || || || || gary_poster || || 2012-05-09 || r15220 || PG 9.1 || || clean run || || || || frankban || || 2012-05-10 || r15225 || lpsetup landed (PG 9.1 is the default from now on) || || clean run || || || || frankban || || 2012-05-10 || r15225 || patched lxc-start-ephemeral (lp-lxc-ip, timeout) || Bug:992184 || || || || || frankban || || 2012-05-10 || r15225 || patched lxc-start-ephemeral (lp-lxc-ip, timeout) || || clean run || || || || frankban || || 2012-05-10 || r15225 || patched lxc-start-ephemeral (lp-lxc-ip, timeout) || || clean run || || || || frankban || || 2012-05-10 || r15225 || patched lxc-start-ephemeral (lp-lxc-ip, timeout) || || clean run || || || || gary_poster || || 2012-05-10 || r15226 || || || clean run || || || || gary_poster || || 2012-05-10 || r15227 || || || clean run || || || || gary_poster || || 2012-05-10 || r15229 || || || clean run || || || || gary_poster || || 2012-05-10 || r15230 || || || clean run || || || || gary_poster || || 2012-05-11 || r15231 || || || Bug:998040 || || || || frankban || || 2012-05-11 || r15232 || patched lxc-start-ephemeral (lp-lxc-ip, timeout) || || clean run || || || || frankban || || 2012-05-11 || r15233 || patched lxc-start-ephemeral (lp-lxc-ip, timeout) || || clean run || || || || gary_poster || || 2012-05-11 || r15233 || || || clean run || || || || gary_poster || || 2012-05-11 || r15233 || || Bug:992184 || || || || || frankban || || 2012-05-11 || r15233 || patched lxc-start-ephemeral (lp-lxc-ip, timeout) || || clean run || || || || frankban || || 2012-05-11 || r15235 || patched lxc-start-ephemeral (lp-lxc-ip, timeout) || || clean run || || || || gary_poster || || 2012-05-11 || r15234 || || || clean run || || || || gary_poster || || 2012-05-11 || r15235 || || || clean run || || || || gary_poster || || 2012-05-11 || r15235 || || Bug:987444 || || || || || gary_poster || || 2012-05-11 || r15235 || || Bug:996773 || || || || || gary_poster || || 2012-05-11 || r15236 || || Bug:996773 || || || || || gary_poster || || 2012-05-11 || r15236 || || Bug:992184 || || || || || gary_poster || || 2012-05-11 || r15236 || || Bug:996773 || || || || || gary_poster || || 2012-05-15 || r15247 || || || clean run in data center! || || || || gary_poster || || 2012-05-15 || r15247 || || || clean run || || || || gary_poster || || 2012-05-15 || r15247 || || || Bug:999770 || || || || gary_poster || || 2012-05-15 || r15247 || || --(Bug:994752)-- Bug:1014916 (incomplete test run) || || || || || gary_poster || || 2012-05-15 || r15248 || || || clean run || || || || gary_poster || || 2012-05-15 || r15250 || || Bug:974617 || || || || || gary_poster || || 2012-05-15 || r15250 || || || clean run || || || || gary_poster || || 2012-05-15 || r15251 || || || clean run || || || || gary_poster || || 2012-05-15 || r15252 || || Bug:992184 || || || || || frankban || || 2012-05-16 || r15258 || || || clean run || || || || gary_poster || || 2012-05-18 || r15273 || || || clean run || || || || gary_poster || || 2012-05-18 || r15273 || || || clean run || || || || gary_poster || || 2012-05-18 || r11606 dbdevel || || || clean run || || || || gary_poster || || 2012-05-18 || r15274 || || || clean run || || || || gary_poster || || 2012-05-18 || r15275 || || || clean run || || || || gary_poster || || 2012-05-18 || r11607 dbdevel || || Bug:996729 || (Bug:1001520 triggered Bug:996729) || || || || gary_poster || || 2012-05-21 || r15280 || || Bug:996729 || || || || || gary_poster || || 2012-05-21 || r15280 || || || clean run (same failure as lpbuildbot) || || || || gary_poster || || 2012-05-21 || r11611 dbdevel || || || clean run || || || || gary_poster || || 2012-05-21 || r15280 || || || clean run (same failure as lpbuildbot) || || || || gary_poster || || 2012-05-21 || r11611 dbdevel || || || clean run || || || || gary_poster || || 2012-05-21 || r15280 || || || clean run (same failure as lpbuildbot) || || || || gary_poster || || 2012-05-21 || r11611 dbdevel || || || clean run || || || || gary_poster || || 2012-05-21 || r15280 || || || clean run (same failure as lpbuildbot) || || || || gary_poster || || 2012-05-21 || r11611 dbdevel || || || clean run || || || || gary_poster || || 2012-05-21 || r15280 || || Bug:998040 || Bug:1002796 || || || || gary_poster || || 2012-05-21 || r11612 dbdevel || || || clean run || || || || gary_poster || || 2012-05-21 || r11613 dbdevel || || || clean run || || || || frankban || || 2012-05-22 || r15282 || patched lxc-start-ephemeral (lp-lxc-ip, timeout) || Bug:996729 Bug:1002820 || || || || || gary_poster || || 2012-05-22 || r15282 || || Bug:993510 || || || || || gary_poster || || 2012-05-22 || r11613 dbdevel || || || Bug:1003044 || || || || gary_poster || || 2012-05-22 || r15282 || || || Bug:1003040 || || || || gary_poster || || 2012-05-22 || r11614 dbdevel || || Bug:996729 || Bug:1003200 || || || || gary_poster || || 2012-05-22 || r11614 dbdevel || || || Bug:1003206 || || || |||||||||||||||||| We had a number of completely broken runs. We are about out of that hole now, it seems... || |||||||||||||||||| Tests above this break were divided into 8 ephemeral LXC containers. Below this line they are on 32. ({{{juju deploy --config=$REPO/$SERIES/buildbot-slave/examples/lpbuildbot.yaml --repository=$REPO local:buildbot-slave --constraints "instance-type=cc2.8xlarge"}}} or see [[http://pastebin.ubuntu.com/1017947/|this pastebin for a script that starts the entire juju instance from scratch]]) || || gary_poster || || 2012-06-01 || r15349 || configuration A || Bug:1007111 Bug:994752 || || || || || gary_poster || || 2012-06-01 || r15349 || configuration B || Bug:1007111 || || || || || gary_poster || || 2012-06-01 || r11641 dbdevel || configuration B || Bug:1007111 Bug:1002820 || Bug:1007576 || || || || gary_poster || || 2012-06-01 || r15349 || configuration B || Bug:1007111 Bug:1007576 || Bug:1007579 Bug:1007592 || || || || gary_poster || || 2012-06-01 || r15350 || configuration B || Bug:1007576 || || || || || gary_poster || || 2012-06-01 || r15350 || configuration B || Bug:1007576 || || || || || gary_poster || || 2012-06-04 || r15354 || configuration B || Bug:1007576 || || || || || gary_poster || || 2012-06-04 || r15354 || configuration B || Bug:1007576 || || || || || gary_poster || || 2012-06-04 || r15354 || configuration B + lp:~gary/launchpad/bug1007576 || Bug:996729 || || || || || gary_poster || || 2012-06-04 || r15355 || configuration B || Bug:1007576 || || || || || gary_poster || || 2012-06-04 || r15354 || configuration B + lp:~gary/launchpad/bug1007576 || || '''GREEN''' || || || || gary_poster || || 2012-06-04 || r15356 || configuration B || || '''GREEN''' || || || || gary_poster || || 2012-06-04 || r15356 || configuration B || || '''GREEN''' || || || || gary_poster || || 2012-06-05 || r15362 || configuration B || || '''GREEN''' || || || || gary_poster || || 2012-06-05 || r15363 || configuration B || Bug:1007579 || || || || || gary_poster || || 2012-06-05 || r15363 || configuration B || || '''GREEN''' || || || || gary_poster || || 2012-06-05 || r15363 || configuration B || || '''GREEN''' || || || || gary_poster || || 2012-06-05 || r11652 dbdevel || configuration B || Bug:1002820 || || || || || gary_poster || || 2012-06-05 || r15363 || configuration B || || '''GREEN''' || || || || gary_poster || || 2012-06-05 || r15363 || configuration B || --(Bug:994752)-- Bug:1014916 || || || || || gary_poster || || 2012-06-05 || r15363 || configuration C || || '''GREEN''' || || || || gary_poster || || 2012-06-05 || r15363 || configuration C || || '''GREEN''' || || || || gary_poster || || 2012-06-05 || r15363 || configuration C || Bug:996729 || || || || || gary_poster || || 2012-06-05 || r15363 || configuration C || || '''GREEN''' || || || || gary_poster || || 2012-06-05 || r15363 || configuration C || Bug:996729 || || || || || gary_poster || || 2012-06-05 || r15363 || configuration C || || '''GREEN''' || || || || gary_poster || || 2012-06-05 || r15363 || configuration C || || '''GREEN''' || || || || gary_poster || || 2012-06-06 || r15369 || configuration C || || '''GREEN''' || || || || gary_poster || || 2012-06-06 || r15370 || configuration C || || '''GREEN''' || || || || gary_poster || || 2012-06-06 || r11655 dbdevel || configuration C || || '''GREEN''' || || || || gary_poster || || 2012-06-06 || r15370 || configuration C || || '''GREEN''' || || || || gary_poster || || 2012-06-06 || r15371 || configuration C || || Bug:1009712 || || || || gary_poster || || 2012-06-06 || r15371 || configuration C || || '''GREEN''' || || || || gary_poster || || 2012-06-06 || r15371 || configuration C || || '''GREEN''' || || || || gary_poster || || 2012-06-06 || r15371 || configuration C || Bug:1003040 || || || || || gary_poster || || 2012-06-06 || r11656 dbdevel || configuration C || || '''GREEN''' || || || || gary_poster || || 2012-06-06 || r11657 dbdevel || configuration C || Bug:1002820 || || || || || gary_poster || || 2012-06-06 || r15375 || configuration C || || '''GREEN''' || || || || gary_poster || || 2012-06-06 || r15376 || configuration C || || '''GREEN''' || || || || gary_poster || || 2012-06-07 || r15376 || configuration C || || '''GREEN''' || || || || gary_poster || || 2012-06-07 || r11659 dbdevel || configuration C || || '''GREEN''' || || || || gary_poster || || 2012-06-07 || r15376 || configuration C || || '''GREEN''' || || || || gary_poster || || 2012-06-07 || r15376 || configuration C || || '''GREEN''' || || || || gary_poster || || 2012-06-07 || r15376 || configuration C || || Bug:1010251 || || || || gary_poster || || 2012-06-07 || r11659 dbdevel || configuration C || || '''GREEN''' || || || || gary_poster || || 2012-06-07 || r15376 || configuration C || || '''GREEN''' || || || || gary_poster || || 2012-06-07 || r11659 dbdevel || configuration C || || '''GREEN''' || || || || gary_poster || || 2012-06-07 || r15376 || configuration C || || '''GREEN''' || || || |||||||||||||||| We lost a number of results form 2012-06-08 because of a mistake. || || || gary_poster || || 2012-06-11 || r15389 || configuration D || || '''GREEN''' || || || || gary_poster || || 2012-06-11 || r15389 || configuration D || || '''GREEN''' || || || || gary_poster || || 2012-06-11 || r11668 dbdevel || configuration D || Bug:996729 || || || || || gary_poster || || 2012-06-11 || r15389 || configuration D || Bug:996729 || || || || || gary_poster || || 2012-06-11 || r15390 || configuration D || || '''GREEN''' || || || || gary_poster || || 2012-06-11 || r15347 lp:~benji/launchpad/bug-996729 || configuration D || || Bug:1011793 || || || || gary_poster || || 2012-06-11 || r15348 lp:~benji/launchpad/bug-996729 || configuration D || || Bug:1011847 || || || || gary_poster || || 2012-06-11 || r15349 lp:~benji/launchpad/bug-996729 || configuration D || || '''GREEN''' || || || || gary_poster || || 2012-06-12 || r15395 || configuration E || || Bug:1012166 || || || || gary_poster || || 2012-06-12 || r15395 || configuration F || || '''GREEN''' || || || || gary_poster || || 2012-06-12 || r15396 || configuration F || Bug:992184 || || || 36:10 (33:00) || || gary_poster || || 2012-06-12 || r15396 || configuration F || Bug:996729 || || || || || gary_poster || || 2012-06-12 || r15400 || configuration F || || '''GREEN''' tainted: see comment 3 of Bug:996729 || || 35:20 (31:52) || || gary_poster || || 2012-06-12 || r15400 || configuration G || Bug:1011793 || Bug:1012314 (which is sort of a repeat of Bug:721166 but not exactly) || || 40:22 (37:11) || || gary_poster || || 2012-06-12 || r15400 || configuration G || || '''GREEN''' tainted: see comment 3 of Bug:996729 || || 38:14 (35:08) || || gary_poster || || 2012-06-12 || r15400 || configuration G || || '''GREEN''' tainted: see comment 3 of Bug:996729 || || 36:27 (33:24) || || gary_poster || || 2012-06-12 || r15400 || configuration G || || '''GREEN''' tainted: see comment 3 of Bug:996729 || || 36:40 (33:27) || || gary_poster || || 2012-06-12 || r15401 || configuration G || || '''GREEN''' tainted: see comment 3 of Bug:996729 || || 36:46 (33:33) || || gary_poster || || 2012-06-12 || r15401 || configuration G || || '''GREEN''' tainted: see comment 3 of Bug:996729 || || 35:35 (32:27) || || gary_poster || || 2012-06-13 || r15405 || configuration H || || '''GREEN''' tainted: see comment 3 of Bug:996729 || || 38:32 (35:34) || || gary_poster || || 2012-06-13 || r15350 p:~benji/launchpad/bug-996729 (like devel r15405 but with zope.testing p6) || configuration H || || '''GREEN''' || || 36:11 (32:34) || || gary_poster || || 2012-06-13 || r15406 || configuration H || || Bug:1012799 || || 38:32 (35:34) || || gary_poster || || 2012-06-13 || r11675 dbdevel || configuration H || --(Bug:994752)-- Bug:1014916 || || || XXX || || gary_poster || || 2012-06-13 || r15354 lp:~benji/launchpad/bug-996729 (like devel 15405 but with a fix for comment 3 of Bug:996729) || configuration H || Bug:994752 || || || XXX || || gary_poster || || 2012-06-13 || r15355 lp:~benji/launchpad/bug-996729 (like devel 15405 but with a fix for comment 3 of Bug:996729) || configuration H || || '''GREEN''' || || 37:37 (33:56) || || gary_poster || || 2012-06-13 || r15355 lp:~bac/launchpad/bug-682772 (like devel 15409 but with a fix for Bug:682772) || configuration H || || '''GREEN''' || || 37:39 (34:04) || || gary_poster || || 2012-06-13 || r15411 || configuration H || || '''GREEN''' || || 37:14 (33:55) || || gary_poster || || 2012-06-13 || r15411 || configuration H || || '''GREEN''' || || 36:36 (33:28) || || gary_poster || || 2012-06-13 || r11676 dbdevel || configuration H || || '''GREEN''' || || 36:54 (33:46) || || gary_poster || || 2012-06-14 || r15418 || configuration H || || '''GREEN''' || || 38:41 (35:36) || || gary_poster || || 2012-06-14 || r15418 || configuration H || || '''GREEN''' || || 38:08 (35:02) || || gary_poster || || 2012-06-14 || r15419 || configuration H || || '''GREEN''' || || 40:14 (36:58) || || gary_poster || || 2012-06-14 || r11680 dbdevel || configuration H || || '''GREEN''' || || 37:15 (34:00) || || gary_poster || || 2012-06-14 || lp:~gmb/launchpad/zope.testing-p13 r15421 (devel r15419 plus fixes for Bug:682711) || configuration H || || '''GREEN''' || || 37:46 (34:16) || || gary_poster || || 2012-06-14 || r15420 || configuration H || || '''GREEN''' || || 37:20 (34:04) || || gary_poster || || 2012-06-14 || r15421 || configuration H || || '''GREEN''' || || 38:13 (35:02) || || gary_poster || || 2012-06-14 || r15421 || configuration H || || '''GREEN''' (new record, 13 in a row) || || 37:29 (34:21) || || gary_poster || || 2012-06-14 || r15422 || configuration H || Bug:1011793 Bug:1002820 || || || || || gary_poster || || 2012-06-14 || r15422 || configuration H || --(Bug:994752)-- Bug:1014916 || || || || || gary_poster || || 2012-06-15 || r15432 || configuration I || || '''GREEN''' || || about 37 or 38 minutes, very similar to 32 simultaneous LXCs || || gary_poster || || 2012-06-15 || ?? || configuration I || || '''GREEN''' || || || || gary_poster || || 2012-06-15 || ?? || configuration I || || '''GREEN''' || || || || gary_poster || || 2012-06-15 || ?? || configuration I || --(Bug:994752)-- Bug:1014916 || || || || || gary_poster || || 2012-06-15 || ?? || configuration I || Bug:1002320 || || || || || gary_poster || || 2012-06-15 || ?? || configuration J || || '''GREEN''' || || about 40 minutes || || gary_poster || || 2012-06-15 || ?? || configuration J || || '''GREEN''' || || about 42 minutes || || gary_poster || || 2012-06-18 || r15345 || configuration K || || '''GREEN''' || || 38:35 (35:31) || || gary_poster || || 2012-06-18 || r15345 || configuration K || || '''GREEN''' || || 34:56 (31:53) || || gary_poster || || 2012-06-18 || r15345 || configuration K || || '''GREEN''' || || 35:10 (32:00) || || gary_poster || || 2012-06-18 || r15345 dbdevel || configuration K || || Bug:1014907 || || 34:33 (31:33) || || gary_poster || || 2012-06-18 || r15345 || configuration K || || '''GREEN''' || || 34:09 (30:59) || || gary_poster || || 2012-06-18 || r15346 || configuration K || || '''GREEN''' || || 37:17 (33:56) || || gary_poster || || 2012-06-19 || r15440 || configuration K || || '''GREEN''' || || 36:19 (33:22) || || gary_poster || || 2012-06-19 || r15441 || configuration K || || '''GREEN''' || || 35:11 (31:59) || || gary_poster || || 2012-06-19 || r15441 || configuration K || || '''GREEN''' || || 35:11 (31:59) || || gary_poster || || 2012-06-19 || r11692 dbdevel || configuration K || || '''GREEN''' || || 36:01 (32:52) || || gary_poster || || 2012-06-19 || r15443 || configuration K || || '''GREEN''' || || 35:05 (31:47) || || gary_poster || || 2012-06-19 || r15444 || configuration K || || "'''GREEN'''" (failures were from a branch reverted later) || || 37:06 (33:54) || || gary_poster || || 2012-06-19 || r11692 dbdevel || configuration K || || '''GREEN''' || || 36:56 (33:43) || || gary_poster || || 2012-06-19 || r15447 || configuration L || || '''GREEN''' || || 33:08 (29:56) || || gary_poster || || 2012-06-19 || r15447 || configuration L || || '''GREEN''' || || 35:00 (31:55) || || gary_poster || || 2012-06-19 || r15447 || configuration L || || '''GREEN''' || || 33:28 (30:21) || || gary_poster || || 2012-06-19 || r15447 || configuration L || || '''GREEN''' || || 35:39 (32:32) || || gary_poster || || 2012-06-19 || r11693 dbdevel || configuration L || || Bug:1015318 || || 35:09 (32:01) || || gary_poster || || 2012-06-20 || r15457 || configuration M || || "'''GREEN'''" (failures were from a branch reverted later) || || 34:15 (31:09) || || gary_poster || || 2012-06-20 || r11696 dbdevel || configuration M || || "'''GREEN'''" (failures were from a branch reverted later) || || 34:19 (31:06) || || gary_poster || || 2012-06-20 || r15459 || configuration M || || '''GREEN''' || || 35:10 (32:06) || || gary_poster || || 2012-06-20 || r11696 dbdevel || configuration M || || "'''GREEN'''" (failures were from a branch reverted later) || || 34:51 (31:43) || || gary_poster || || 2012-06-20 || r15459 || configuration M || || '''GREEN''' || || 35:28 (32:17) || || gary_poster || || 2012-06-20 || r15459 || configuration N || || '''GREEN''' || || 35:57 (32:48) || || gary_poster || || 2012-06-20 || r15460 || configuration N || || '''GREEN''' || || 33:53 (30:39) || || gary_poster || || 2012-06-20 || r15460 || configuration N || || '''GREEN''' || || 33:40 (30:32) || || gary_poster || || 2012-06-21 || r15465 || configuration N || Bug:1011847 || || || 34:38 (31:29) || || gary_poster || || 2012-06-21 || r15465 || configuration N || Bug:974617 || || || 34:20 (31:14) || || gary_poster || || 2012-06-21 || r11700 dbdevel || configuration N || Bug:974617 || || || 34:21 (31:11) || || gary_poster || || 2012-06-21 || r15466 || configuration O || || '''GREEN''' || || 35:27 (32:12) || || gary_poster || || 2012-06-21 || r15466 || configuration O || || '''GREEN''' || || 34:58 (31:55) || || gary_poster || || 2012-06-21 || r15466 || configuration O || Bug:1002820 || || || 34:35 (31:31) || || gary_poster || || 2012-06-22 || r15473 || configuration O || || '''GREEN''' || || 35:25 (32:21) || || gary_poster || || 2012-06-22 || r15473 || configuration O || || '''GREEN''' || || 34:08 (31:00) || || gary_poster || || 2012-06-22 || r15474 || configuration O || || '''GREEN''' || || 34:12 (31:04) || || gary_poster || || 2012-06-22 || r15474 || configuration O || || '''GREEN''' || || 35:03 (31:34) || || gary_poster || || 2012-06-22 || r15475 || configuration O || || '''GREEN''' || || 34:25 (31:08) || || gary_poster || || 2012-06-22 || r15475 || configuration O || || '''GREEN''' || || 33:33 (30:25) || || gary_poster || || 2012-06-22 || r15476 || configuration O || || '''GREEN''' || || 35:25 (32:14) || || gary_poster || || 2012-06-22 || r15477 || configuration O || || '''GREEN''' || || 36:21 (33:10) || || gary_poster || || 2012-06-22 || r15477 || configuration O || || '''GREEN''' || || 33:54 (30:40) || || gary_poster || || 2012-06-22 || r15477 || configuration O || || '''GREEN''' || || 35:18 (32:13) || || gary_poster || || 2012-06-22 || r15477 || configuration O || || '''GREEN''' || || 35:02 (31:54) || || gary_poster || || 2012-06-22 || r15477 || configuration O || || '''GREEN''' || || 35:12 (32:03) || || gary_poster || || 2012-06-25 || r15485 || configuration O || || '''GREEN''' || || 35:21 (32:15) || || gary_poster || || 2012-06-25 || r15486 || configuration O || || '''GREEN''' || || 33:43 (30:32) || || gary_poster || || 2012-06-25 || r15487 || configuration O || Bug:974617 || || || 35:28 (32:17) || || gary_poster || || 2012-06-25 || r11710 dbdevel || configuration O || || '''GREEN''' || || 34:50 (31:33) || || gary_poster || || 2012-06-25 || r15487 || configuration O || || '''GREEN''' || || 35:50 (32:41) || || gary_poster || || 2012-06-25 || r15488 || configuration O || || '''GREEN''' || || 35:15 (32:06) || || gary_poster || || 2012-06-25 || r15488 || configuration O || || '''GREEN''' || || 35:52 (32:45) || || gary_poster || || 2012-06-25 || r15489 || configuration O || || '''GREEN''' || || 34:58 (31:45) || || gary_poster || || 2012-06-26 || r15496 || configuration O || || '''GREEN''' || || : (:) || || gary_poster || || 2012-06-25 || r11713 dbdevel || configuration O || || '''GREEN''' || || : (:) || || gary_poster || || 2012-06-26 || r15496 || configuration O || || '''GREEN''' || || : (:) || || gary_poster || || 2012-06-26 || r15496 || configuration O || || '''GREEN''' || || : (:) || || gary_poster || || 2012-06-26 || r15496 || configuration O || || '''GREEN''' || || : (:) || || gary_poster || || 2012-06-26 || r15496 || configuration O || || '''GREEN''' || || : (:) || || gary_poster || || 2012-06-26 || r15496 || configuration O || || '''GREEN''' || || : (:) || || gary_poster || || 2012-06-26 || r15496 || configuration O || || '''GREEN''' || || : (:) || || gary_poster || || 2012-06-25 || r11714 dbdevel || configuration O || || '''GREEN''' || || : (:) || || gary_poster || || 2012-06-26 || r15497 || configuration O || || '''GREEN''' || || : (:) || || gary_poster || || 2012-06-26 || r15500 || configuration O || || '''GREEN''' || || : (:) || || gary_poster || || 2012-06-26 || r15501 || configuration O || || '''GREEN''' || || : (:) || || gary_poster || || 2012-06-27 || r15505 || configuration O || Bug:974617 || || || : (:) || || gary_poster || || 2012-06-27 || r15505 || configuration O || || '''GREEN''' || || : (:) || || gary_poster || || 2012-06-27 || r15505 || configuration O || || '''GREEN''' || || : (:) || || gary_poster || || 2012-06-27 || r15505 || configuration O || || '''GREEN''' || || : (:) || || gary_poster || || 2012-06-27 || r15506 || configuration O || || '''GREEN''' || || : (:) || || gary_poster || || 2012-06-27 || r15506 || configuration O || || '''GREEN''' || || : (:) || || gary_poster || || 2012-06-27 || r15507 || configuration O || || '''GREEN''' || || : (:) || || gary_poster || || 2012-06-28 || - || configuration O || || '''GREEN''' || || : (:) || || gary_poster || || 2012-06-28 || - || configuration O || || '''GREEN''' || || : (:) || || gary_poster || || 2012-06-28 || - || configuration O || || '''GREEN''' || || : (:) || || gary_poster || || 2012-06-28 || - || configuration O || || '''GREEN''' || || : (:) || || gary_poster || || 2012-06-28 || - || configuration O || || '''GREEN''' || || : (:) || || gary_poster || || 2012-06-28 || - || configuration O || || '''GREEN''' || || : (:) || || gary_poster || || 2012-06-28 || - || configuration O || || '''GREEN''' || || : (:) || || gary_poster || || 2012-06-28 || - || configuration O || || '''GREEN''' || || : (:) || || gary_poster || || 2012-06-28 || - || configuration O || || '''GREEN''' || || : (:) || || gary_poster || || 2012-06-28 || - || configuration O || Bug:1014916 || || || : (:) || || gary_poster || || 2012-06-29 || - || configuration O || Bug:1002820 || || || : (:) || || gary_poster || || 2012-06-29 || - || configuration O || || '''GREEN''' || || : (:) || || gary_poster || || 2012-06-29 || - || configuration O || || '''GREEN''' || || : (:) || || gary_poster || || 2012-06-29 || - || configuration O || || '''GREEN''' || || : (:) || || gary_poster || || 2012-06-29 || - || configuration O || || '''GREEN''' || || : (:) || || gary_poster || || 2012-06-29 || - || configuration O || Bug:974617 || || || : (:) || || gary_poster || || 2012-07-03 || - || configuration O || || '''GREEN''' || || : (:) || || gary_poster || || 2012-07-03 || - || configuration O || Bug:974617 || || || : (:) || || gary_poster || || 2012-07-03 || - || configuration O || || '''GREEN''' || || : (:) || || gary_poster || || 2012-07-03 || - || configuration O || || '''GREEN''' || || : (:) || || gary_poster || || 2012-07-03 || - || configuration O || || '''GREEN''' || || : (:) || || gary_poster || || 2012-07-03 || - || configuration O || || '''GREEN''' || || : (:) || || gary_poster || || 2012-07-03 || - || configuration O || || '''GREEN''' || || : (:) || || gary_poster || || 2012-07-03 || - || configuration O || || '''GREEN''' || || : (:) || || gary_poster || || 2012-07-03 || - || configuration O || || '''GREEN''' || || : (:) || || gary_poster || || 2012-07-03 || - || configuration O || Bug:1002820 || || || : (:) || || gary_poster || || 2012-07-04 || - || configuration O || || '''GREEN''' || || : (:) || || gary_poster || || 2012-07-04 || - || configuration O || Bug:1014916 || || || : (:) || || gary_poster || || 2012-07-04 || - || configuration O || || '''GREEN''' || || : (:) || || gary_poster || || 2012-07-04 || - || configuration O || || '''GREEN''' || || : (:) || || gary_poster || || 2012-07-04 || - || configuration O || || '''GREEN''' || || : (:) || || gary_poster || || 2012-07-04 || - || configuration O || || '''GREEN''' || || : (:) || || gary_poster || || 2012-07-04 || - || configuration O || Bug:1002820 || || || : (:) || || gary_poster || || 2012-07-04 || - || configuration O || Bug:1002820 || || || : (:) || || gary_poster || || 2012-07-04 || - || configuration O || || '''GREEN''' || || : (:) || || gary_poster || || 2012-07-04 || - || configuration O || || '''GREEN''' || || : (:) || || gary_poster || || 2012-07-04 || - || configuration O || || '''GREEN''' || || : (:) || || gary_poster || || 2012-07-04 || - || configuration O || Bug:974617 || || || : (:) || || gary_poster || || 2012-07-05 || - || configuration O || || '''GREEN''' || || : (:) || || gary_poster || || 2012-07-05 || - || configuration O || || '''GREEN''' || || : (:) || || gary_poster || || 2012-07-05 || - || configuration O || || '''GREEN''' || || : (:) || || gary_poster || || 2012-07-05 || - || configuration O || || '''GREEN''' || || : (:) || || gary_poster || || 2012-07-05 || - || configuration O || || '''GREEN''' || || : (:) || || gary_poster || || 2012-07-05 || - || configuration O || || '''GREEN''' || || : (:) || || gary_poster || || 2012-07-05 || - || configuration O || || '''GREEN''' || || : (:) || || gary_poster || || 2012-07-05 || - || configuration O || Bug:1014916 || || || : (:) || || gary_poster || || 2012-07-05 || - || configuration O || Bug:974617 || || || : (:) || || gary_poster || || 2012-07-06 || - || configuration O || || '''GREEN''' || || : (:) || || gary_poster || || 2012-07-06 || - || configuration O || || '''GREEN''' || || : (:) || || gary_poster || || 2012-07-06 || - || configuration O || || '''GREEN''' || || : (:) || || gary_poster || || 2012-07-06 || - || configuration O || || '''GREEN''' || || : (:) || || gary_poster || || 2012-07-06 || - || configuration O || || Bug:1021940 || || : (:) || || gary_poster || || 2012-07-06 || - || configuration O || Bug:1021940 || || || : (:) || || gary_poster || || 2012-07-06 || - || configuration O || Bug:974617 || || || : (:) || || gary_poster || || 2012-07-09 || - || configuration P || || '''GREEN''' || || : (:) || || gary_poster || || 2012-07-09 || - || configuration P || Bug:1014916 || || || : (:) || || gary_poster || || 2012-07-09 || - || configuration P || || '''GREEN''' || || : (:) || || gary_poster || || 2012-07-09 || - || configuration P || || '''GREEN''' || || : (:) || || gary_poster || || 2012-07-09 || - || configuration P || Bug:974617 || || || : (:) || || gary_poster || || 2012-07-09 || - || configuration P || || '''GREEN''' || || : (:) || || gary_poster || || 2012-07-09 || - || configuration P || || '''GREEN''' || || : (:) || || gary_poster || || 2012-07-09 || - || configuration P || || '''GREEN''' || || : (:) || || gary_poster || || 2012-07-09 || - || configuration P || || '''GREEN''' || || : (:) || || gary_poster || || 2012-07-09 || - || configuration P || || '''GREEN''' || || : (:) || || gary_poster || || 2012-07-10 || - || configuration O || || '''GREEN''' || || : (:) || || gary_poster || || 2012-07-10 || - || configuration O || || '''GREEN''' || || : (:) || || gary_poster || || 2012-07-10 || - || configuration O || || '''GREEN''' || || : (:) || || gary_poster || || 2012-07-10 || - || configuration O || || '''GREEN''' || || : (:) || || gary_poster || || 2012-07-10 || - || configuration O || || '''GREEN''' || || : (:) || || gary_poster || || 2012-07-10 || - || configuration O || || '''GREEN''' || || : (:) || || gary_poster || || 2012-07-10 || - || configuration O || || '''GREEN''' || || : (:) || || gary_poster || || 2012-07-10 || - || configuration O || || '''GREEN''' || || : (:) || || gary_poster || || 2012-07-10 || - || configuration O || || '''GREEN''' || || : (:) || || gary_poster || || 2012-07-10 || - || configuration O || || '''GREEN''' || || : (:) || || gary_poster || || 2012-07-10 || - || configuration O || || '''GREEN''' || || : (:) || || gary_poster || || 2012-07-10 || - || configuration O || || '''GREEN''' || || : (:) || || gary_poster || || 2012-07-11 || - || configuration O || || '''GREEN''' || || : (:) || || gary_poster || || 2012-07-11 || - || configuration O || || '''GREEN''' || || : (:) || || gary_poster || || 2012-07-11 || - || configuration O || || '''GREEN''' || || : (:) || || gary_poster || || 2012-07-11 || - || configuration O || || '''GREEN''' || || : (:) || || gary_poster || || 2012-07-11 || - || configuration O || || '''GREEN''' || || : (:) || || gary_poster || || 2012-07-11 || - || configuration O || || '''GREEN''' || || : (:) || || gary_poster || || 2012-07-12 || - || configuration O || || '''GREEN''' || || : (:) || || gary_poster || || 2012-07-12 || - || configuration O || || '''GREEN''' || || : (:) || || gary_poster || || 2012-07-12 || - || configuration O || || '''GREEN''' || || : (:) || || gary_poster || || 2012-07-12 || - || configuration O || || '''GREEN''' || || : (:) || || gary_poster || || 2012-07-12 || - || configuration O || || '''GREEN''' || || : (:) || || gary_poster || || 2012-07-12 || - || configuration O || || '''GREEN''' || || : (:) || || gary_poster || || 2012-07-13 || - || configuration O || || '''GREEN''' || || : (:) || || gary_poster || || 2012-07-13 || - || configuration O || || '''GREEN''' || || : (:) || || gary_poster || || 2012-07-13 || - || configuration O || || '''GREEN''' || || : (:) || || gary_poster || || 2012-07-13 || - || configuration O || || '''GREEN''' || || : (:) || || gary_poster || || 2012-07-13 || - || configuration O || || '''GREEN''' || || : (:) || || gary_poster || || 2012-07-13 || - || configuration O || || '''GREEN''' || || : (:) || || gary_poster || || 2012-07-13 || - || configuration O || || '''GREEN''' || || : (:) || || gary_poster || || 2012-07-13 || - || configuration O || || '''GREEN''' || || : (:) || || gary_poster || || 2012-07-13 || - || configuration O || || '''GREEN''' || || : (:) || || gary_poster || || 2012-07-13 || - || configuration O || || '''GREEN''' || || : (:) || || gary_poster || || 2012-07-13 || - || configuration O || || '''GREEN''' || || : (:) || || gary_poster || || 2012-07-13 || - || configuration O || || '''GREEN''' || || : (:) || || gary_poster || || 2012-07-16 || - || configuration O || || '''GREEN''' || || : (:) || || gary_poster || || 2012-07-16 || - || configuration O || || '''GREEN''' || || : (:) || || gary_poster || || 2012-07-16 || - || configuration O || Bug:974617 || || || : (:) || || gary_poster || || 2012-07-16 || - || configuration O || || '''GREEN''' || || : (:) || || gary_poster || || 2012-07-16 || - || configuration O || || '''GREEN''' || || : (:) || || gary_poster || || 2012-07-16 || - || configuration O || Bug:1014916 || || || : (:) || || gary_poster || || 2012-07-16 || - || configuration O || || '''GREEN''' || || : (:) || || gary_poster || || 2012-07-24 || - || configuration O || || '''GREEN''' || || : (:) || || gary_poster || || 2012-07-24 || - || configuration O || || '''GREEN''' || || : (:) || || gary_poster || || 2012-07-24 || - || configuration O || || '''GREEN''' || || : (:) || || gary_poster || || 2012-07-24 || - || configuration O || || '''GREEN''' || || : (:) || || gary_poster || || 2012-07-24 || - || configuration O || || '''GREEN''' || || : (:) || || gary_poster || || 2012-07-24 || - || configuration O || Bug:1002820 || || || : (:) || || gary_poster || || 2012-07-24 || - || configuration O || || '''GREEN''' || || : (:) || || gary_poster || || 2012-07-24 || - || configuration O || Bug:1002820 || || || : (:) || || gary_poster || || 2012-07-24 || - || configuration O || || '''GREEN''' || || : (:) || || gary_poster || || 2012-07-24 || - || configuration O || || '''GREEN''' || || : (:) || || gary_poster || || 2012-07-24 || - || configuration O || Bug:1002820 || || || : (:) || || gary_poster || || 2012-07-24 || - || configuration O || || '''GREEN''' || || : (:) || || gary_poster || || 2012-07-24 || - || configuration O || || '''GREEN''' || || : (:) || || gary_poster || || 2012-07-24 || - || configuration O || || '''GREEN''' || || : (:) || || gary_poster || || 2012-07-24 || - || configuration O || || '''GREEN''' || || : (:) || || gary_poster || || 2012-07-24 || - || configuration O || || '''GREEN''' || || : (:) || || gary_poster || || 2012-07-24 || - || configuration O || || '''GREEN''' || || : (:) || || gary_poster || || 2012-07-24 || - || configuration O || || '''GREEN''' || || : (:) || || gary_poster || || 2012-07-24 || - || configuration O || || '''GREEN''' || || : (:) || || gary_poster || || 2012-07-24 || - || configuration O || || '''GREEN''' || || : (:) || || gary_poster || || 2012-07-25 || - || || || '''GREEN''' || || : (:) || || gary_poster || || 2012-07-25 || - || || || '''GREEN''' || || : (:) || || gary_poster || || 2012-07-25 || - || || || '''GREEN''' || || : (:) || || gary_poster || || 2012-07-25 || - || || Bug:974617 || || || : (:) || || gary_poster || || 2012-07-25 || - || || || '''GREEN''' || || : (:) || || gary_poster || || 2012-07-25 || - || || || '''GREEN''' || || : (:) || || gary_poster || || 2012-07-25 || - || || Bug:974617 || || || : (:) || || gary_poster || || 2012-07-25 || - || || || '''GREEN''' || || : (:) || || gary_poster || || 2012-07-25 || - || || || '''GREEN''' || || : (:) || || gary_poster || || 2012-07-25 || - || || Bug:1002820 || || || : (:) || || gary_poster || || 2012-07-26 || - || || Bug:974617 Bug:1014916 || || || : (:) || || gary_poster || || 2012-07-26 || - || || Bug:1014916 || || || : (:) || || gary_poster || || 2012-07-26 || - || || || '''GREEN''' || || : (:) || || gary_poster || || 2012-07-26 || - || || || '''GREEN''' || || : (:) || || gary_poster || || 2012-07-26 || - || || || '''GREEN''' || || : (:) || || gary_poster || || 2012-07-26 || - || || || '''GREEN''' || || : (:) || || gary_poster || || 2012-07-26 || - || || || '''GREEN''' || || : (:) || || gary_poster || || 2012-07-26 || - || || || '''GREEN''' || || : (:) || || gary_poster || || 2012-07-26 || - || || || '''GREEN''' || || : (:) || || gary_poster || || 2012-07-26 || - || || || '''GREEN''' || || : (:) || || gary_poster || || 2012-07-27 || - || || || '''GREEN''' || || : (:) || || gary_poster || || 2012-07-27 || - || || || '''GREEN''' || || : (:) || || gary_poster || || 2012-07-27 || - || || Bug:974617 || || || : (:) || || gary_poster || || 2012-07-27 || - || || || '''GREEN''' || || : (:) || || gary_poster || || 2012-07-27 || - || || || '''GREEN''' || || : (:) || || gary_poster || || 2012-07-27 || - || || || '''GREEN''' || || : (:) || || gary_poster || || 2012-07-27 || - || || Bug:974617 || || || : (:) || || gary_poster || || 2012-07-27 || - || || || Bug:1030148 || || : (:) || || gary_poster || || 2012-07-27 || - || || || '''GREEN''' || || : (:) || || gary_poster || || 2012-07-27 || - || || || '''GREEN''' || || : (:) || = Configurations = This is for configurations that are bigger than fit in the table above. To make a change in the container, do the following. * Make sure no tests are running. Stopping the master might be a good idea. * On the slave, set the password ({{{passwd}}}) for root. * Look in /etc/shadow for the hashed password. Copy that line. * Modify /var/lib/lxc/lptests/rootfs/etc/shadow to replace the existing root hash (empty) with the one you made from the host. * {{{lxc-start -n lptests}}} and log in as root. * Make your changes. * {{{poweroff}}} == A == * Get new lxc [[https://wiki.ubuntu.com/Testing/EnableProposed|from proposed]] for --(Bug:994752)-- Bug:1014916 '''in the host''' * Get stub's Postgres 9.1 for Bug:992184 '''in the container''' * {{{add-apt-repository ppa:stub/junk}}} * {{{apt-get update}}} * {{{apt-get upgrade}}} (or {{{apt-get install postgres-9.1}}}?) == B == All of A, plus * '''in the host''', set the TRIES in lxc-start-ephemeral ({{{TRIES=60}}}) to 300 (--(Bug:994752)-- Bug:1014916) == C == All of A, plus * '''in the host''', set the TRIES in lxc-start-ephemeral ({{{TRIES=60}}}) to 600 (--(Bug:994752)-- Bug:1014916) == D == (This is all of C except Precise's postgres now has the fix that stub's Postgres 9.1 package has, so stub's package should no longer be necessary per his comment to Bug:992184.) * Get new lxc [[https://wiki.ubuntu.com/Testing/EnableProposed|from proposed]] for --(Bug:994752)-- Bug:1014916 '''in the host''' * '''in the host''', set the TRIES in lxc-start-ephemeral ({{{TRIES=60}}}) to 600 (--(Bug:994752)-- Bug:1014916) == E == (This is all of D except the new lxc is now part of precise, so you don't need to install it from proposed.) * '''in the host''', set the TRIES in lxc-start-ephemeral ({{{TRIES=60}}}) to 600 (--(Bug:994752)-- Bug:1014916) == F == All of E plus change lp-setup-lxc-test as described in Bug:1012166 == G == All of F plus ''in the container'' run the following for Bug:992184 (until postgresql-9.1 9.1.4-1~lucid4 or higher is available by default). {{{ add-apt-repository ppa:pitti/postgresql apt-get update apt-get install postgresql-9.1 }}} == H == (We have resolutions for F and G, so this is back to E.) * '''in the host''', set the TRIES in lxc-start-ephemeral ({{{TRIES=60}}}) to 600 (--(Bug:994752)-- Bug:1014916) == I == (very large timeouts are proving to be insufficient for --(Bug:994752)-- Bug:1014916 and Bug:1002820 so I am trying an adjusted approach as an experiment) * '''in the host''', set the TRIES in lxc-start-ephemeral ({{{TRIES=60}}}) to 180 (--(Bug:994752)-- Bug:1014916 ) * '''in the host''', change /usr/lib/python2.7/dist-packages/testrepository/testcommand.py so that TestListingFixture.local_concurrency returns 24, hard coded, rather than determining for /proc/cpuinfo to return 32. 24 is arbitrary: I'm not sure how to determine this value other than experimentally, and I'm not sure what to measure reliably other than the effect on these test failures. It is 75% of 32. (--(Bug:994752)-- Bug:1014916, Bug:1011793 and Bug:1002820) == J == (more experiments for --(Bug:994752)-- Bug:1014916 and Bug:1002820) * '''in the host''', set the TRIES in lxc-start-ephemeral ({{{TRIES=60}}}) to 180 (--(Bug:994752)-- Bug:1014916 ) * '''in the host''', change /usr/lib/python2.7/dist-packages/testrepository/testcommand.py so that TestListingFixture.local_concurrency returns 16, hard coded. == K == (more experiments for --(Bug:994752)-- Bug:1014916) * '''in the host''', set the TRIES in lxc-start-ephemeral ({{{TRIES=60}}}) to 180 (--(Bug:994752)-- Bug:1014916 ) * '''in the host''', change /usr/lib/python2.7/dist-packages/testrepository/testcommand.py so that TestListingFixture.local_concurrency returns 20, hard coded. == L == (more experiments for --(Bug:994752)-- Bug:1014916) * '''in the host''', set the TRIES in lxc-start-ephemeral ({{{TRIES=60}}}) to 180 (--(Bug:994752)-- Bug:1014916 ) * '''in the host''', change /usr/lib/python2.7/dist-packages/testrepository/testcommand.py so that TestListingFixture.local_concurrency returns 20, hard coded. * Change /var/lib/lxc/lptests/rootfs/etc/init/networking.conf to not wait for udev (that is, use {{{start on (local-filesystems)}}}) * '''in the host''' remove the container's /etc/init/udevtrigger.conf ({{{rm /var/lib/lxc/lptests/rootfs/etc/init/udevtrigger.conf}}} == M == (Workarounds for Bug:1014916) * '''in the host''', change /usr/lib/python2.7/dist-packages/testrepository/testcommand.py so that TestListingFixture.local_concurrency returns 20, hard coded. * Change /var/lib/lxc/lptests/rootfs/etc/init/networking.conf to not wait for udev (that is, use {{{start on (local-filesystems)}}}) * '''in the host''' remove the container's /etc/init/udevtrigger.conf ({{{rm /var/lib/lxc/lptests/rootfs/etc/init/udevtrigger.conf}}} == N == (Workarounds for Bug:1014916. The local_concurrency hack is no longer necessary because we use the new --concurrency flag on testrepository) * Change /var/lib/lxc/lptests/rootfs/etc/init/networking.conf to not wait for udev (that is, use {{{start on (local-filesystems)}}}) * '''in the host''' remove the container's /etc/init/udevtrigger.conf ({{{rm /var/lib/lxc/lptests/rootfs/etc/init/udevtrigger.conf}}} == O == All of N plus this for Bug:1013921: * On the slave, in the host, modify /usr/local/bin/lp-setup-lxc-test so that it passes -vvv to bin/test. == P == All of O plus * '''in the host''', set the TRIES in lxc-start-ephemeral ({{{TRIES=60}}}) to 180 (--(Bug:994752)-- Bug:1014916 ) = Analyzing subunit output = '''Most of the info below is now unnecessary except for the very earliest runs. Instead, buildbot includes the data we need already pre-processed.''' == How do I find out what worker ran a particular test? == Given a full or partial test name and a subunit output, you should be able to do something like {{{subunit-filter -s --with 'NAME OF TEST'}}}. After that, look in the output for the {{{test: NAME OF TEST}}} line. The next line or so should have a "tags:" prefix. Look there for the name of the worker (of the form "worker-N"). == How do I get the list of tests run on a particular process? == Unfortunately, there's not a simple answer. * Install [[https://launchpad.net/~testing-cabal/+archive/archive|the Testing Cabal's PPA]] or [[https://launchpad.net/~yellow/+archive/ppa|the Yellow Squad's PPA]] on Precise and get or update subunit. * Download the desired subunit output from the links above. * Now run a command that looks something like this: {{{$ cat ~/Downloads/testrepo-0.txt | env PYTHONPATH=python ./filters/subunit-filter -s --with-tag='worker-0' --without=":setUp$" --without=":tearDown$" --without-tag="zope:info_suboptimal" --no-passthrough | subunit-ls > worker0.txt}}} The two {{{--without=":setUp$" --without=":tearDown$"}}} options should be replaceable with {{{--without-tag='zope:layer'}}} (see Bug:986429) but we are not using that yet--tags are still a bit hosed in other ways so we don't completely trust them yet. == How do I run the tests in a given order, to mimic the order that tests ran in a particular process? == 1. Create a test list (one test per line) to be fed to {{{--load-list}}} from your failed test run. Let's assume you save the file as {{{worker0.txt}}}. 1. Find the layer where the bad test is run and the first test in that layer: {{{bin/test --load-list worker0.txt --list-tests > layered.txt}}} 1. In {{{layered.text}}} find the test of interest and then search backwards for the layer marker. Note the first test in that layer. In {{{worker-0.txt}}}, copy and paste the set of tests starting with the first in the layer to the bad test and save as {{{worker0-trimmed.txt}}}. (This change has to be made the original file, not the one produced by {{{--list-tests}}}, as it names some tests in a way that is not compatible with {{{--load-list}}}.) 1. Now you can run just the tests that are candidates for causing the isolation error: {{{/bin/test -vv --load-list worker0-trimmed.txt}}}