Building live filesystems in Launchpad

Launchpad now supports building live filesystems using the livecd-rootfs package in Ubuntu. This replaces an older system where we had a dedicated builder per architecture just for this purpose, and allows us to parallelise this task across the Launchpad build farm instead.

There are new LiveFS and LiveFSBuild objects in Launchpad to facilitate this, visible on the "devel" version of the webservice as livefs and livefs_build respectively. LiveFS objects specify the general metadata for an image type (for instance, there is a LiveFS for Kubuntu in Ubuntu utopic, and a LiveFS for Ubuntu Touch in Ubuntu trusty); they are created by the (hidden) Person:+new-livefs page, or by the livefses.new webservice method. LiveFSBuild objects identify a single build of a LiveFS, which must nominate an archive, DistroArchSeries, and pocket, and may also add additional items of metadata which take effect for just that build; they are created by the livefs.requestBuild webservice method.

The archive attached to a LiveFSBuild is a source archive, not a target archive. That is, it specifies the archive that packages in the live filesystem image should come from. In the common case, this is just the Ubuntu primary archive, but it may also be a PPA, in which case both the primary archive and that PPA will be used as a source of packages. Since this is a source archive, there is no requirement for the user requesting the build to have upload access to the archive. (At present, launchpad-buildd and livecd-rootfs are not quite smart enough to work out all the details of this just from the archive parameter to livefs.requestBuild, and typically need help in the form of an extra_ppas entry in the metadata_overrides dictionary. ubuntu-cdimage knows how to deal with this.)

There are some security issues surrounding live filesystem builds. Firstly, they are a very easy way to consume lots of time on the Launchpad build farm, so only members of the launchpad-livefs-builders team can create new LiveFS or LiveFSBuild objects. Secondly, building a live filesystem involves executing code from packages in the source archive; therefore, in order to build live filesystems on devirtualised builders (which support more architectures than virtualised builders at present, but have no effective sandboxing, so may only be used by trusted people), both the LiveFS parameters and the source archive must be devirtualised. At present (June 2014) it is not possible to build live filesystems on virtualised builders at all due to compatibility problems with the hardy kernel used there, although we expect this restriction to be lifted with the advent of scalingstack; furthermore non-x86 builds even on scalingstack currently use qemu to execute non-x86 code, which may not work well for building live filesystems.