Launchpad has a build farm capable of building different types of builds, such as Bazaar source recipes and translations files, but here we will concentrate on the Soyuz aspects: turning a source package into a binary package (or packages).
There are two types of builder, virtual and non-virtual. The non-virtual builders are only used for a small number of architectures where we do not yet have adequate virtualisation infrastructure. All other builds are done on virtual builders.
The buildd-manager is a Twisted daemon that is responsible for dispatching builds to the builders, monitoring the builders (including disabling failed ones) and collecting files for finished builds. There is a table called BuildQueue that contains a record of all outstanding or in-progress builds. When the build completes, the relevant record is removed from the table. More information about the buildd-manager can be found at XXX.
More about build creation
When a source is accepted and a build created for it, several pre-requisite conditions need to be met:
The source's requested architectures must be buildable in the target DistroSeries, by virtue of a DistroArchSeries record being present with a matching architecture tag.
- There must be a builder available that builds the requested architecture(s).
There must be a chroot file available for the DistroArchSeries. (see below for more about chroots)
Managing builders and chroots
Buildd admins manage the build farm. One of the important tasks is creating chroot images. These are a "tarball" of the initial filesystem layout required to start a build. When a build starts on a builder, it unpacks the tarball sent to it from the manager and "chroots" inside it. Each DistroArchSeries has its own chroot file, which is periodically updated so that the builders have a quicker start-up time so that apt-get dist-upgrade has less work to do at the start of each build.
The manage-chroot script in lp:ubuntu-archive-tools deals with adding and removing chroot tarballs in the librarian. It uses the setChroot and removeChroot methods on DistroArchSeries via the webservice.
<Add DB schema diagram here>
After the buildd-manager collects the built files from the builder, it puts them in an "incoming" queue area, very similar to where txpkgupload puts its source uploads. There is a separate cron job that runs process-upload.py on the buildmaster machine which works almost identically to processing sources, except it runs with an upload policy that allows unsigned uploads.
After the build upload is processed, the build record serves as a link between the source and its binary. This makes for some fun SQL queries all over Soyuz.