[[Soyuz/TechnicalDetails/Publishing|Next Page: Publishing]]

= Soyuz Building =

 * lib/lp/buildmaster/
 * lib/lp/soyuz/model/binarypackagebuild*

Launchpad has a [[https://launchpad.net/builders|build farm]] capable of building different types of builds, such as Bazaar source recipes and translations files, but here we will concentrate on the Soyuz aspects: turning a source package into a binary package (or packages).

There are two types of builder, virtual and non-virtual. The non-virtual builders are only used for a small number of architectures where we do not yet have adequate virtualisation infrastructure.  All other builds are done on virtual builders.

=== Buildd Manager ===

The `buildd-manager` is a Twisted daemon that is responsible for dispatching builds to the builders, monitoring the builders (including disabling failed ones) and collecting files for finished builds.  There is a table called BuildQueue that contains a record of all outstanding or in-progress builds.  When the build completes, the relevant record is removed from the table.  More information about the `buildd-manager` can be found at XXX.

=== More about build creation ===

When a source is accepted and a build created for it, several pre-requisite conditions need to be met:

 * The source's requested architectures must be buildable in the target `DistroSeries`, by virtue of a `DistroArchSeries` record being present with a matching architecture tag.
 * There must be a builder available that builds the requested architecture(s).
 * There must be a ''chroot'' file available for the `DistroArchSeries`. (see below for more about chroots)

=== Managing builders and chroots ===

[[https://launchpad.net/~launchpad-buildd-admins|Buildd admins]] manage the build farm. One of the important tasks is creating chroot images. These are a "tarball" of the initial filesystem layout required to start a build. When a build starts on a builder, it unpacks the tarball sent to it from the manager and "chroots" inside it.  Each `DistroArchSeries` has its own chroot file, which is periodically updated so that the builders have a quicker start-up time so that `apt-get dist-upgrade` has less work to do at the start of each build.

The `manage-chroot` script in lp:ubuntu-archive-tools deals with adding and removing chroot tarballs in the librarian.  It uses the `setChroot` and `removeChroot` methods on `DistroArchSeries` via the webservice.

=== Processing builds ===

<Add DB schema diagram here>

After the `buildd-manager` collects the built files from the builder, it puts them in an "incoming" queue area, very similar to where txpkgupload puts its source uploads. There is a separate cron job that runs `process-upload.py` on the buildmaster machine which works almost identically to processing sources, except it runs with an upload policy that allows unsigned uploads.

After the build upload is processed, the build record serves as a link between the source and its binary. This makes for some fun SQL queries all over Soyuz.

[[Soyuz/TechnicalDetails/Publishing|Next Page: Publishing]]