Setting up the Launchpad machine for parallel testing

The Yellow Squad has developed a tool called lpsetup to replace the original rocketfuel-* scripts that were used by Launchpad developers to get the required pieces needed to begin contributing to Launchpad. While rocketfuel was written for that single purpose, lpsetup has another mission, which is to allow the complete set up and running of buildbot for parallel testing on EC2 or locally on LXC.

Create buildbot slave?

Assumptions

No use of the juju charms at all.
lpsetup can only be run inside the LXC.
Root changes on the host will only be done via puppet scripts written by IS.

Steps

Setup host using puppet (as above)

Install required packages and PPAs

     apt-add-repository ppa:yellow/ppa
     apt-add-repository deb http://us.archive.ubuntu.com/ubuntu/ lucid main universe
     apt-get install ssh bzr apache2.2-common buildbot/lucid testrepository haveged subunit lxc libvirt-bin

Install buildbot from lp:~launchpad/lpbuildbot/public/

Make buildbot user/group as needed # Some versions of LXC require the user to have a group. Bug #942850. addgroup buildbot usermod --gid buildbot buildbot usermod -s /bin/sh buildbot

Generate ssh keys for buildbot user named launchpad_lxc_id_rsa ssh-keyscan -t rsa bazaar.launchpad.net >> ~buildbot/.ssh/known_hosts Add new public key to ~buildbot/.ssh/authorized_keys Set up bzr whoami: bzr whoami “Launchpad PQM <launchpad-pqm@canonical.com>“ Disable apparmor profiles for lxc:

ln -s /etc/apparmor.d/usr.bin.lxc-start /etc/apparmor.d/usr.bin.lxc-start /etc/apparmor.d/disable/ and apparmor_parser -R /etc/apparmor.d/usr.bin.lxc-start

Create /etc/lxc/lp-setup.conf:

lxc.network.type = veth
lxc.network.link = lxcbr0
lxc.network.flags = up

Create an LXC named “lptests”: lxc-create -t ubuntu -n lptests -f /etc/lxc/lp-setup.conf -- -r lucid -a i386 -b buildbot Authorize root connections to the LXC:

mkdir /var/lib/lxc/lptests/rootfs/root/.ssh
cp ~buildbot/.ssh/authorized_keys /var/lib/lxc/lptests/rootfs/root/.ssh/

Append to /var/lib/lxc/lptests/fstab

none dev/shm tmpfs defaults 0 0

This step must be removed when lucid is no longer used as the LXC OS.

Start the lxc: lxc-start -n lptests -d Wait for the LXC to start and accept connections. Install the three scripts needed by buildbot: lp-setup-lxc-build, lp-setup-lxc-cleanup, and lp-setup-lxc-test. They will go into /usr/local/bin. Create /etc/sudoers.d/launchpad-buildbot with

{{{buildbot ALL = (ALL) NOPASSWD: /usr/local/bin/lp-setup-lxc-cleanup,

/usr/local/bin/lp-setup-lxc-build, /usr/local/bin/lp-setup-lxc-test}}}

chmod 0440 /etc/sudoers.d/launchpad-buildbot Disable hardlink restriction: echo 0 > /proc/sys/kernel/yama/protected_nonaccess_hardlinks

Remove when bug=lp:944386 is resolved.

Install lpsetup:

        apt-get install python-software-properties
        add-apt-repository ppa:yellow/ppa
        apt-get clean && apt-get update
        apt-get upgrade
        apt-get install lpsetup

Now use lpsetup to finish the provisioning of the container: lp-setup init-host --yes -u buildbot -S launchpad_lxc_id_rsa --skip-steps setup_home

Inside container as buildbot lp-setup init-repo --yes --use-http --no-checkout lp-setup update ~buildbot/launchpad/lp-branches/devel --use-http Inside container as root: lp-setup finish-init-host ~buildbot/launchpad/lp-branches/devel --user buildbot --yes poweroff If that doesn’t work lxc-stop -n lptests in the host.

launchpad development

yellow/ParallelTestingSetupForDataCentre

Setting up the Launchpad machine for parallel testing

Assumptions

Steps