Size: 6527
Comment: manage-chroot uses python3 now
|
Size: 7033
Comment:
|
Deletions are marked like this. | Additions are marked like this. |
Line 12: | Line 12: |
The password for the user can be generated using `mkpasswd -m sha-512 <password>`. `mkpasswd` lives in the `whois` package. | The password for the user can be generated using: {{{ mkpasswd -m sha-512 <password> }}} `mkpasswd` lives in the `whois` package. |
Line 16: | Line 21: |
To create this run `lxc profile create vm` and then `lxc profile edit vm`. | To create this run: {{{ lxc profile create vm }}} and then: {{{ lxc profile edit vm }}} |
Line 61: | Line 74: |
This will take a while to settle. You can monitor it's progress with `lxd console <vm name>`. | This will take a while to settle. You can monitor its progress with `lxd console <vm name>`. |
Line 82: | Line 95: |
sudo apt install bzr bzr branch lp:launchpad-buildd |
sudo apt install git git clone https://git.launchpad.net/launchpad-buildd |
Line 94: | Line 107: |
sudo apt build-dep launchpad-buildd | sudo apt build-dep launchpad-buildd fakeroot sudo apt install -f |
Line 102: | Line 116: |
sudo apt install ./python-lpbuildd_<version>_all.deb ./launchpad-buildd_<version>_all.deb | sudo dpkg -i ./python3-lpbuildd_<version>_all.deb ./launchpad-buildd_<version>_all.deb |
Line 108: | Line 122: |
Edit /etc/launchpad-buildd/default and change 'ntphosts' to something valid (91.189.89.199 might be useful here.) | Edit /etc/launchpad-buildd/default and change 'ntphost' to something valid (ntp.ubuntu.com should work) |
Line 111: | Line 125: |
/usr/bin/python /usr/bin/twistd --no_save --pidfile /var/run/launchpad-buildd/default.pid --python /usr/lib/launchpad-buildd/buildd-slave.tac -n | cd launchpad-buildd /usr/bin/python3 /usr/bin/twistd --no_save --pidfile /var/run/launchpad-buildd/default.pid --python /usr/lib/launchpad-buildd/buildd-slave.tac -n |
Line 128: | Line 143: |
sudo apt install python3-ubuntutools | sudo apt install python3-launchpadlib python3-ubuntutools |
Line 130: | Line 145: |
sha256sum livecd.ubuntu-base.rootfs.tar.gz | sha256sum livecd.ubuntu-base.rootfs.tar.gz/home/buildd/filecache-default |
Line 132: | Line 147: |
cp <sha256sum named file> /home/buildd/filecache-default | sudo cp <sha256sum named file> /home/buildd/filecache-default sudo chown buildd: /home/buildd/filecache-default/<sha256sum named file> |
Line 148: | Line 164: |
proxy.build('1-3', 'oci', '540947c8a8a5222e9f34150afed945f9f01ccf43', {}, {'name': 'test-build', 'series': 'bionic', 'arch_tag': 'amd64', 'git_repository': 'https://github.com/tomwardill/test-docker-repo.git', 'archives': ['deb http://archive.ubuntu.com/ubuntu bionic main restricted', 'deb http://archive.ubuntu.com/ubuntu bionic-updates main restricted', 'deb http://archive.ubuntu.com/ubuntu bionic universe']}) | proxy.build('1-3', 'oci', '<sha256 of base, possibly from previous section>', {}, {'name': 'test-build', 'series': 'bionic', 'arch_tag': 'amd64', 'git_repository': 'https://github.com/tomwardill/test-docker-repo.git', 'archives': ['deb http://archive.ubuntu.com/ubuntu bionic main restricted', 'deb http://archive.ubuntu.com/ubuntu bionic-updates main restricted', 'deb http://archive.ubuntu.com/ubuntu bionic universe']}) |
Line 164: | Line 180: |
=== Image Setup === Consult the 'Launchpad Configuration' section of [[Soyuz/HowToUseSoyuzLocally|here]] to do the correct `manage-chroot` dance to register an image with launchpad. Without this, you will have no valid buildable architectures. |
|
Line 168: | Line 188: |
1. Log in as admin 2. Go to https://launchpad.test/~launchpad-buildd-admins and add your user 3. Go to https://launchpad.test/~ubuntu-team and add your user |
1. Log in as admin 2. Go to https://launchpad.test/~launchpad-buildd-admins and add your user 3. Go to https://launchpad.test/~ubuntu-team and add your user |
Line 176: | Line 196: |
1. Go to https://launchpad.net/builders 2. Press 'Register a new build machine' 3. Fill in the details. * The 'URL' is probably 'http://<buildd ip>:8221'. * 'Virtualized' should be true * VM reset protocol can be either 1.1 or 2.0 4. After 30 seconds or so, the status of the builder on the builders page should be 'Idle'. This page does not auto-update, so refresh! |
1. Go to https://launchpad.test/builders 2. Press 'Register a new build machine' 3. Fill in the details. * The 'URL' is probably 'http://<buildd ip>:8221'. * 'Virtualized' should be true * VM reset protocol can be either 1.1 or 2.0 4. After 30 seconds or so, the status of the builder on the builders page should be 'Idle'. This page does not auto-update, so refresh! |
LXD VM Support
This is now on stable and allows for management of VMs with the same LXD CLI.
For now, we need to use the images: source for images, rather than the ubuntu: images. The default ubuntu images do not have the LXD agent preinstalled. Once they do, this gets a bit simpler.
It is also slightly simpler to use the ubuntu user, as it is already available in the image and doesn't require as many hoops jumped to get uid/gid mapping to work.
Create a LXD profile for VMs
This is a convenience helper profile for VMs that will add users and run cloud-init for installing the LXD VM agent. It is not required and you can pass the options on the lxc command. The password for the user can be generated using:
mkpasswd -m sha-512 <password>
mkpasswd lives in the whois package.
For now, we are using the LXD provided cloud images as it has the LXD agent and cloud-init preinstalled. This requires a smaller LXD profile, but needs some extra commands afterwards.
To create this run:
lxc profile create vm
and then:
lxc profile edit vm
name: vm config: limits.cpu: "2" limits.memory: 4GB user.vendor-data: | #cloud-config package_update: true ssh_pwauth: yes packages: - openssh-server - byobu - language-pack-en users: - name: "ubuntu" passwd: "<shell password hash>" lock_passwd: false groups: lxd shell: /bin/bash sudo: ALL=(ALL) NOPASSWD:ALL ssh-import-id: <lp username> description: "" devices: config: source: cloud-init:config type: disk eth0: name: eth0 nictype: bridged parent: lxdbr0 type: nic work: path: <path to folder you want to share to in the VM> source: <path to folder you want to share from the host> type: disk
Start the LXD VM
Start a VM via downloading the images: cloud image
lxc launch images:ubuntu/<release name>/cloud -p vm -p default <vm name> --vm
This will take a while to settle. You can monitor its progress with lxd console <vm name>. Once it has complete cloud-init, you should then see an IP assigned in lxc list and be able to execute a bash shell with lxc exec <vm name> bash.
Configure password and ssh
This should be done by the cloud-init config in the profile, but the package is not installed at the time that is run, so do it afterwards manually:
lxc exec <vm name> sudo passwd ubuntu lxc exec <vm name> --user 1000 "/usr/bin/ssh-import-id" <launchpad user id>
This will not be required once we can use the ubuntu: image source in LXD.
Launchpad Buildd
We'll need a clone of this and then build and install it for running.
Branch
sudo apt install git git clone https://git.launchpad.net/launchpad-buildd
Install dependencies
cd launchpad-buildd sudo apt-add-repository ppa:launchpad/ubuntu/buildd-staging sudo apt-add-repository ppa:launchpad/ubuntu/ppa vi /etc/apt/sources.list.d/launchpad-ubuntu-ppa-bionic.list <uncomment deb-src line> sudo apt update sudo apt build-dep launchpad-buildd fakeroot sudo apt install -f
Make and install the package
make cd .. sudo dpkg -i ./python3-lpbuildd_<version>_all.deb ./launchpad-buildd_<version>_all.deb
Run the buildd
Edit /etc/launchpad-buildd/default and change 'ntphost' to something valid (ntp.ubuntu.com should work) sudo mkdir -p /var/run/launchpad-buildd sudo chown ubuntu: /var/run/launchpad-buildd cd launchpad-buildd /usr/bin/python3 /usr/bin/twistd --no_save --pidfile /var/run/launchpad-buildd/default.pid --python /usr/lib/launchpad-buildd/buildd-slave.tac -n
Making changes
The package is installed as a system deb, so to make changes you will need to rebuild and reinstall the package following the 'Make and install' section.
Testing
You probably want the next section ('Configuring Launchpad') at this point, but if you are doing any buildd development and need to test your changes without having to have the whole system running, you can use the XML-RPC interface to cause builds to happen.
Getting a base image
First, we need a base image to use for the builds. Usually this is pulled as part of a build, but if we don't have Launchpad involved, we need to set this up manually.
bzr branch lp:ubuntu-archive-tools sudo apt install python3-launchpadlib python3-ubuntutools ./manage-chroot -s bionic -a amd64 get sha256sum livecd.ubuntu-base.rootfs.tar.gz/home/buildd/filecache-default mv livecd.ubuntu-base.rootfs.tar.gz <sha256sum from previous line> sudo cp <sha256sum named file> /home/buildd/filecache-default sudo chown buildd: /home/buildd/filecache-default/<sha256sum named file>
Running a build
You can try running a build via the XML-RPC interface. Start a python/ipython repl and run.
import xmlrpclib proxy = xmlrpclib.ServerProxy("http://localhost:8221/rpc") proxy.status()
Assuming that works, a sample build can be created using (relying on the OCI capabilities being merged into launchpad-buildd):
proxy.build('1-3', 'oci', '<sha256 of base, possibly from previous section>', {}, {'name': 'test-build', 'series': 'bionic', 'arch_tag': 'amd64', 'git_repository': 'https://github.com/tomwardill/test-docker-repo.git', 'archives': ['deb http://archive.ubuntu.com/ubuntu bionic main restricted', 'deb http://archive.ubuntu.com/ubuntu bionic-updates main restricted', 'deb http://archive.ubuntu.com/ubuntu bionic universe']})
Configuring Launchpad
Running soyuz and adding data
First, you'll need to run some extra bits in Launchpad:
utilities/start-dev-soyuz.sh utilities/soyuz-sampledata-setup.py make run
Image Setup
Consult the 'Launchpad Configuration' section of here to do the correct manage-chroot dance to register an image with launchpad. Without this, you will have no valid buildable architectures.
User setup
It's convenient to add your user to the correct groups, so you can interact with it, without being logged in as admin.
- Log in as admin
Go to https://launchpad.test/~launchpad-buildd-admins and add your user
Go to https://launchpad.test/~ubuntu-team and add your user
Registering the buildd
The buildd that you have just installed needs registering with Launchpad so that builds can be dispatched to it.
- Press 'Register a new build machine'
- Fill in the details.
The 'URL' is probably 'http://<buildd ip>:8221'.
- 'Virtualized' should be true
- VM reset protocol can be either 1.1 or 2.0
- After 30 seconds or so, the status of the builder on the builders page should be 'Idle'. This page does not auto-update, so refresh!