launchpad development
LXD VM Support
This is now on stable and allows for management of VMs with the same LXD CLI.
For now, we need to use the images: source for images, rather than the ubuntu: images. The default ubuntu images do not have the LXD agent preinstalled. Once they do, this gets a bit simpler.
It is also slightly simpler to use the ubuntu user, as it is already available in the image and doesn't require as many hoops jumped to get uid/gid mapping to work.
Create a LXD profile for VMs
This is a convenience helper profile for VMs that will add users and run cloud-init for installing the LXD VM agent. It is not required and you can pass the options on the lxc command. The password for the user can be generated using mkpasswd -m sha-512 <password>. mkpasswd lives in the whois package.
For now, we are using the LXD provided cloud images as it has the LXD agent and cloud-init preinstalled. This requires a smaller LXD profile, but needs some extra commands afterwards.
To create this run lxc profile create vm and then lxc profile edit vm.
name: vm
config:
limits.cpu: "2"
limits.memory: 4GB
user.vendor-data: |
#cloud-config
package_update: true
ssh_pwauth: yes
packages:
- openssh-server
- byobu
- language-pack-en
users:
- name: "ubuntu"
passwd: "<shell password hash>"
lock_passwd: false
groups: lxd
shell: /bin/bash
sudo: ALL=(ALL) NOPASSWD:ALL
ssh-import-id: <lp username>
description: ""
devices:
config:
source: cloud-init:config
type: disk
eth0:
name: eth0
nictype: bridged
parent: lxdbr0
type: nic
work:
path: <path to folder you want to share to in the VM>
source: <path to folder you want to share from the host>
type: disk
Start the LXD VM
Start a VM via downloading the images: cloud image
lxc launch images:ubuntu/<release name>/cloud -p vm -p default <vm name> --vm
This will take a while to settle. You can monitor it's progress with lxd console <vm name>. Once it has complete cloud-init, you should then see an IP assigned in lxc list and be able to execute a bash shell with lxc exec <vm name> bash.
Configure password and ssh
This should be done by the cloud-init config in the profile, but the package is not installed at the time that is run, so do it afterwards manually:
lxc exec <vm name> sudo passwd ubuntu lxc exec <vm name> --user 1000 "/usr/bin/ssh-import-id" <launchpad user id>
This will not be required once we can use the ubuntu: image source in LXD.
Launchpad Buildd
We'll need a clone of this and then build and install it for running.
Branch
sudo apt install bzr bzr branch lp:launchpad-buildd
Install dependencies
cd launchpad-buildd sudo apt-add-repository ppa:launchpad/ubuntu/buildd-staging sudo apt-add-repository ppa:launchpad/ubuntu/ppa vi /etc/apt/sources.list.d/launchpad-ubuntu-ppa-bionic.list <uncomment deb-src line> sudo apt update sudo apt build-dep launchpad-buildd
Make and install the package
make cd .. # This may fail with missing dependencies. sudo dpkg -i python-lpbuildd_<version>_all.deb launchpad-buildd_<version>_all.deb # If the previous line failed, do the following. There is no need to rerun the previous line: sudo apt install -f
Run the buildd
Edit /etc/launchpad-buildd/default and change 'ntphosts' to something valid (91.189.89.199 might be useful here.) sudo mkdir -p /var/run/launchpad-buildd sudo chown ubuntu: /var/run/launchpad-buildd /usr/bin/python /usr/bin/twistd --no_save --pidfile /var/run/launchpad-buildd/default.pid --python /usr/lib/launchpad-buildd/buildd-slave.tac -n
Making changes
The package is installed as a system deb, so to make changes you will need to rebuild and reinstall the package following the 'Make and install' section.
Testing
You probably want the next section ('Configuring Launchpad') at this point, but if you are doing any buildd development and need to test your changes without having to have the whole system running, you can use the XML-RPC interface to cause builds to happen.
Getting a base image
First, we need a base image to use for the builds. Usually this is pulled as part of a build, but if we don't have Launchpad involved, we need to set this up manually.
bzr branch lp:ubuntu-archive-tools sudo apt install python-ubuntutools ./manage-chroot -s bionic -a amd64 get sha256sum livecd.ubuntu-base.rootfs.tar.gz mv livecd.ubuntu-base.rootfs.tar.gz <sha256sum from previous line> cp <sha256sum named file> /home/buildd/filecache-default
Running a build
You can try running a build via the XML-RPC interface. Start a python/ipython repl and run.
import xmlrpclib
proxy = xmlrpclib.ServerProxy("http://localhost:8221/rpc")
proxy.status()Assuming that works, a sample build can be created using (relying on the OCI capabilities being merged into launchpad-buildd):
proxy.build('1-3', 'oci', '540947c8a8a5222e9f34150afed945f9f01ccf43', {}, {'name': 'test-build', 'series': 'bionic', 'arch_tag': 'amd64', 'git_repository': 'https://github.com/tomwardill/test-docker-repo.git', 'archives': ['deb http://archive.ubuntu.com/ubuntu bionic main restricted', 'deb http://archive.ubuntu.com/ubuntu bionic-updates main restricted', 'deb http://archive.ubuntu.com/ubuntu bionic universe']})
Configuring Launchpad
Running soyuz and adding data
First, you'll need to run some extra bits in Launchpad:
utilities/start-dev-soyuz.sh utilities/soyuz-sampledata-setup.py make run
User setup
It's convenient to add your user to the correct groups, so you can interact with it, without being logged in as admin.
1. Log in as admin 2. Go to https://launchpad.test/~launchpad-buildd-admins and add your user 3. Go to https://launchpad.test/~ubuntu-team and add your user
Registering the buildd
The buildd that you have just installed needs registering with Launchpad so that builds can be dispatched to it.
1. Go to https://launchpad.net/builders 2. Press 'Register a new build machine' 3. Fill in the details.
The 'URL' is probably 'http://<buildd ip>:8221'.
- 'Virtualized' should be true
- VM reset protocol can be either 1.1 or 2.0
4. After 30 seconds or so, the status of the builder on the builders page should be 'Idle'. This page does not auto-update, so refresh!
