launchpad development|
Size: 613
Comment:
|
Size: 4522
Comment:
|
| Deletions are marked like this. | Additions are marked like this. |
| Line 3: | Line 3: |
| This page aims to explain the mechanisms that make the wonderful diagrams on the [[Trunk|"where is trunk?"]] page work. | This page aims to explain the mechanisms that make the wonderful diagrams on the [[Trunk|"where is trunk?"]] page work. You'll need to read that page for this one to make any sense at all. |
| Line 7: | Line 7: |
| * buildbot -- this is what runs the tests. * PQM -- this is how changes get into branches. |
* PQM -- this is how changes get into the devel and db-devel branches * [[https://lpbuildbot.canonical.com/|buildbot]] -- this is what runs the tests. |
| Line 11: | Line 11: |
| NOTE: I don't yet completely understand how a commit to a Bazaar branch on Launhpad ends up triggering a build in buildbot. | There is a (currently private) [[https://code.edge.launchpad.net/~launchpad-pqm/lpbuildbot/trunk|branch on launchpad]] that contains the buildbot config and the buildbot-poll.py script. The buildbot UI is currently also private, but this will change, hopefully soon. == PQM == PQM is a robot that processes requests to merge branches into either devel or db-devel. For devel, all the requests come from developers. For db-devel, many requests are generated by buildbot-poll.py merging stable into db-devel. PQM checks that the commit message matches the required regexp, that the branch merges cleanly into the target branch, and runs "make check_merge" for landings to devel and "make check_db_merge" for landings to db-devel. The only difference is that "make check_merge" tries to prevent database changes landing. PQM has two configs: the default config and the 'testfix' config. The difference is that the 'testfix' config requires the commit message start with '[testfix]'. Sadly, there is no way of telling which config PQM is using at present, other than submitting merge requests (see [[https://bugs.edge.launchpad.net/launchpad-foundations/+bug/424060|bug 422964]]). The buildbot-poll.py script is responsible for switching between the configs. |
| Line 15: | Line 31: |
| == PQM == | Buildbot is our continuous integration tool of choice. At one level, it's fairly simple: when a change is detected on the devel or db-devel branches on launchpad, the tests are run on that branch and their success or failure noted. Buildbot also runs some non-trunk builds -- for example, testing devel against bzr.dev -- but these are ignored for the purposes of this page. In general in buildbot, a "change source" produces "changes" which are fed to a "scheduler" which can examines the change to determine whether to trigger a build. The change source we use is the "Bzr``Poller" in bzrbuildbot/poller.py in the lpbuildbot branch. It is configured with a list of URLs to watch and when it sees a new revision in one of these branches, it feeds it to buildbot. The scheduler we use for the two trunk builders is "Aggregating``Testfix". An Aggregating``Testfix scheduler is configured with a branch and watches for changes that affect this branch. When it sees a change that affect its branch, it checks to see if the last build succeeded or failed. If it failed, then it only starts a new build if the commit message contains '[testfix]'. We have a custom web UI that adds: * a page that lets you force a build, and * an XML-RPC method used by buildbot-poll.py. (Builbot's built-in "Force Build" button doesn't work for us for rather boring reasons). Builds forced using our page have a distinctive "reason" attribute that the buildbot-poll.py script looks for using our custom XML-RPC method. |
| Line 18: | Line 53: |
This script checks the status of the builds (via XML-RPC) on buildbot and does two main things depending on what it finds: 1. it swaps PQM in and out of 'testfix' mode. 1. it may push to the stable and/or db-stable branches. PQM is put in testfix mode if either branch is unclean, where "unclean" means "the last test run failed and we haven't had a testfix revision or a forced build yet". We check the branch itself for the testfix revision, so buildbot doesn't have to have seen it before we go out of testfix. If a build is forced in the UI, buildbot-poll.py can't tell until the build has started and this doesn't happen until the EC2 instance has booted which can take a few minutes. For each development branch (i.e. devel or db-devel), if the most recent build succeeded the script pushes the revision of the development branch that was tested into the corresponding stable branch (i.e. stable or db-stable). It runs out of cron every 5 minutes, on the PQM box (it has to run on the PQM box, because it switches in and out of testfix mode by moving config files around). |
How the 4 trunk branches and buildbot work together
This page aims to explain the mechanisms that make the wonderful diagrams on the "where is trunk?" page work. You'll need to read that page for this one to make any sense at all.
There are three main components:
- PQM -- this is how changes get into the devel and db-devel branches
buildbot -- this is what runs the tests.
- buildbot-poll.py -- this script monitors the branches and builds on buildbot and implements much of our policy.
There is a (currently private) branch on launchpad that contains the buildbot config and the buildbot-poll.py script.
The buildbot UI is currently also private, but this will change, hopefully soon.
PQM
PQM is a robot that processes requests to merge branches into either devel or db-devel.
For devel, all the requests come from developers.
For db-devel, many requests are generated by buildbot-poll.py merging stable into db-devel.
PQM checks that the commit message matches the required regexp, that the branch merges cleanly into the target branch, and runs "make check_merge" for landings to devel and "make check_db_merge" for landings to db-devel. The only difference is that "make check_merge" tries to prevent database changes landing.
PQM has two configs: the default config and the 'testfix' config. The difference is that the 'testfix' config requires the commit message start with '[testfix]'. Sadly, there is no way of telling which config PQM is using at present, other than submitting merge requests (see bug 422964).
The buildbot-poll.py script is responsible for switching between the configs.
Buildbot
Buildbot is our continuous integration tool of choice.
At one level, it's fairly simple: when a change is detected on the devel or db-devel branches on launchpad, the tests are run on that branch and their success or failure noted.
Buildbot also runs some non-trunk builds -- for example, testing devel against bzr.dev -- but these are ignored for the purposes of this page.
In general in buildbot, a "change source" produces "changes" which are fed to a "scheduler" which can examines the change to determine whether to trigger a build.
The change source we use is the "BzrPoller" in bzrbuildbot/poller.py in the lpbuildbot branch. It is configured with a list of URLs to watch and when it sees a new revision in one of these branches, it feeds it to buildbot.
The scheduler we use for the two trunk builders is "AggregatingTestfix". An AggregatingTestfix scheduler is configured with a branch and watches for changes that affect this branch. When it sees a change that affect its branch, it checks to see if the last build succeeded or failed. If it failed, then it only starts a new build if the commit message contains '[testfix]'.
We have a custom web UI that adds:
- a page that lets you force a build, and
- an XML-RPC method used by buildbot-poll.py.
(Builbot's built-in "Force Build" button doesn't work for us for rather boring reasons).
Builds forced using our page have a distinctive "reason" attribute that the buildbot-poll.py script looks for using our custom XML-RPC method.
buildbot-poll.py
This script checks the status of the builds (via XML-RPC) on buildbot and does two main things depending on what it finds:
- it swaps PQM in and out of 'testfix' mode.
- it may push to the stable and/or db-stable branches.
PQM is put in testfix mode if either branch is unclean, where "unclean" means "the last test run failed and we haven't had a testfix revision or a forced build yet". We check the branch itself for the testfix revision, so buildbot doesn't have to have seen it before we go out of testfix. If a build is forced in the UI, buildbot-poll.py can't tell until the build has started and this doesn't happen until the EC2 instance has booted which can take a few minutes.
For each development branch (i.e. devel or db-devel), if the most recent build succeeded the script pushes the revision of the development branch that was tested into the corresponding stable branch (i.e. stable or db-stable).
It runs out of cron every 5 minutes, on the PQM box (it has to run on the PQM box, because it switches in and out of testfix mode by moving config files around).
