5086
Comment: document the u1 hack for rabbit and *why* it works.
|
5958
|
Deletions are marked like this. | Additions are marked like this. |
Line 1: | Line 1: |
## page was copied from Running/VirtualMachine This page explains how to set up and run Launchpad (for development) inside a LXC. |
This page explains how to set up and run Launchpad (for development) inside an LXC container. |
Line 6: | Line 5: |
Launchpad development setup makes significant changes to your machine; its nice to be unaffected by those except when you are actually doing such development. | Launchpad development setup makes significant changes to your machine; it's nice to be unaffected by those when you're not doing such development. Also, multiple containers can be used to work around Launchpad's limitations regarding concurrent test runs on a single machine. |
Line 8: | Line 7: |
Also, launchpad has some limitations on concurrent testing per-machine and so forth - multiple container's can be used to work around this. | These instructions should work on Ubuntu 12.04 LTS and later. Older versions of LXC are significantly less reliable and polished, so if you've used a version of LXC older than 12.04 LTS's final release on your development machine, you'll want to remove `/var/cache/lxc` first to ensure that you don't have a broken cache. |
Line 10: | Line 9: |
= Make a LXC = | = Create an LXC container = |
Line 12: | Line 11: |
1. Install lxc | 1. Install LXC's userspace tools. |
Line 17: | Line 16: |
1. Work around Bug:800456 and Bug:801002 | 1. Create a container. You might want to use an HTTP proxy or alternate Ubuntu mirror; you can do this by specifying an http_proxy or MIRROR environment variable after `sudo`. |
Line 19: | Line 18: |
sudo apt-get install cgroup-bin libvirt-bin | sudo lxc-create -t ubuntu -n lpdev -- -r lucid -a i386 -b $USER |
Line 22: | Line 21: |
1. Work around Bug:784093 | 1. Start the container. You'll probably see a few early warnings about boot processes dying -- they're normal and can be ignored as long as you end up at a login prompt. |
Line 24: | Line 23: |
sudo dd of=/etc/cgconfig.conf << EOF mount { cpu = /sys/fs//cgroup/cpu; cpuacct = /sys/fs/cgroup/cpu; devices = /sys/fs/cgroup/cpu; memory = /sys/fs/cgroup/cpu; } EOF sudo service cgconfig restart |
sudo lxc-start -n lpdev |
Line 34: | Line 25: |
1. '''[Inside the container]''' Log in with your normal username and password. You'll have full sudo powers. | |
Line 35: | Line 28: |
1. Work around Bug:798476 (optional if you run i386 or have a -tonne- of memory and don't care about 64-bit footprint. Grab the patch from the bug and apply it to /usr/lib/lxc/templates/lxc-lucid. If you're running i386 already or want a 64-bit lxc then do not pass arch= on the lxc-create command line. 1. Create a config for your containers {{{ sudo dd of=/etc/lxc/local.conf << EOF lxc.network.type=veth lxc.network.link=virbr0 lxc.network.flags=up #fuse (workaround for Bug:800886) lxc.cgroup.devices.allow = c 10:229 rwm # part of the Bug:798476 workaround - # remove if you are running a 64 bit lxc or # 32 bit on 32-bit base os lxc.arch = i686 EOF }}} 1. Create a container {{{ sudo arch=i386 lxc-create -n lucid-test-lp -t lucid -f /etc/lxc/local.conf }}} If you want to use a proxy {{{ sudo arch=i386 http_proxy=http://host:port/ lxc-create -n lucid-test-lp -t lucid -f /etc/lxc/local.conf }}} And if you want to set a custom mirror, similar to http_proxy, but set MIRROR= instead. 1. (Outside the container) grab your user id and username so you can setup a bind mount outside the container: {{{ id -u id -nu }}} 1. Start the container {{{ sudo lxc-start -n lucid-test-lp }}} Ignore the warning about openssh crashing - it restarts on a later event. The initial credentials are root:root. 1. Grab the ip address (handed out via libvirt's dhcp server) - you may wish to ssh in rather than using the console (seems to have better termcap experience). |
1. '''[Inside the container]''' Get the container's IP address from the console. You'll generally want to `ssh` in for convenience and better termcap functionality, and the default Ubuntu LXC dnsmasq setup doesn't provide name resolution from the host |
Line 81: | Line 33: |
1. The new container won't have your proxy / mirror settings preserved. Customise it at this point before going further if you care about this. 1. Enable multiverse (rocketfuel-setup wants it, don't ask me why). 1. Install some additional packages we'll need to run rocketfuel-setup etc. |
1. '''[Inside the container]''' IP addresses suck, so we should install Avahi to allow us to 'ssh lpdev.local' to get in. (Remove the previous point if Avahi works reliably everywhere). |
Line 87: | Line 35: |
apt-get install python-software-properties apt-add-repository ppa:ubuntu-virt apt-get update apt-get install bzr less sudo lxcguest # select I for 'install' when prompted about console.conf |
sudo apt-get install avahi-daemon |
Line 94: | Line 38: |
1. Inside the container add the user: | 1. '''[Inside the container]''' Install some additional packages needed before running `rocketfuel-setup` and the rest of the Launchpad setup process. The language pack is mandatory and must match your configured locale, or [[#postgresql-locale-breakage|PostgreSQL will break]]. |
Line 96: | Line 40: |
adduser --uid $id $username adduser $username sudo |
sudo apt-get install bzr language-pack-en |
Line 100: | Line 43: |
1. Change the /etc/hosts file - make sure that the local host name (e.g. lucid-test-lp) is the first item on its line. Something like: {{{ 127.0.0.1 localhost 127.0.1.1 lucid-test-lp }}} This is needed because rabbit looks up its hostname, then resolves it (which if its on the 127.0.0.1 line returns 127.0.0.1) and then resolves that *back* to a hostname (and the 127.0.0.1 lookup returns localhost, not the hostname) - if they don't all match, it bails. 1. To stop it now run 'poweroff -n' (normally you would use lxc-stop, but see Bug:784093). 1. Setup a bind mount so you can access your home dir (and thus your LP source code) from within the lxc container: * edit /var/lib/lxc/lucid-test-lp/fstab * Add a line: |
1. '''[Inside the container]''' Shut down by running `sudo poweroff` inside the container, and you should eventually be dumped back out to your host system. If it looks like it's hanging, force it to stop with `sudo lxc-stop -n lpdev` on the host. 1. Start it up again, headless this time (`-d`). The same IP address will be used, so you don't need console access. |
Line 113: | Line 47: |
/home/$username /var/lib/lxc/lucid-test-lp/rootfs/home/$username none bind 0 0 }}} 1. Start it up again - headless now, we have the ip address from before. {{{ sudo lxc-start -n lucid-test-lp -d |
sudo lxc-start -n lpdev -d |
Line 121: | Line 50: |
1. ssh <vm IP address> to connect to the VM. Your ssh key is already present because of the bind mount to your home dir. | 1. `ssh <IP address obtained above>` to connect to the VM. If your SSH key is in your local `authorized_keys` file you shouldn't be prompted for a password, as your home directory (including public and private keys) is bind mounted into the container. `ssh -A` might give you a better agent experience when dealing with Launchpad code hosting. |
Line 123: | Line 52: |
1. You can now follow the [[Getting|getting-started]] on LP instructions. Be warned that changes in ~ will affect you outside the container. You will want to run rocketfuel-setup with --no-workspace if your home already has a workarea. You may need to run utilities/launchpad-database-setup separately too. | 1. '''[Inside the container]''' You can now follow the normal [[Getting|LP installation instructions]]. Be warned that changes in your home directory will also be seen outside the container and vice versa. If your home directory already has a Launchpad work area set up you'll want to run `rocketfuel-setup --no-workspace` to avoid trying to recreate it, but all subsequent steps are still required. |
Line 125: | Line 54: |
1. You probably want to follow [[Running/RemoteAccess]] has a discussion for how you can configure things so your non-container browser can access web pages from within the container. | 1. Follow [[Running/RemoteAccess]] to set up access from the host's applications to the container's Launchpad instance. |
Line 127: | Line 56: |
1. rabbitmq may fail to start up. If that happens it appears to be a [[http://lists.rabbitmq.com/pipermail/rabbitmq-discuss/2010-April/007024.html|mnesia glitch]] best sorted by zapping mnesia. | = Problems = <<Anchor(postgresql-locale-breakage)>> == launchpad-database-setup fails == PostgreSQL will fail to create a cluster during installation if your locale is configured to something non-C but not supported by the container, so you need to install the relevant language pack. You will know you need to do this if bzr or apt commands have been spewing locale warnings. For instance, if your computer has a localised English locale, use this: |
Line 129: | Line 72: |
apt-get install language-pack-en }}} If you didn't install the language pack before running rocketfuel-setup, you'll need to run `sudo pg_createcluster 8.4 main` afterwards to fix the damage. == rabbitmq does not start up == rabbitmq may fail to start up. If that happens it appears to be a [[http://lists.rabbitmq.com/pipermail/rabbitmq-discuss/2010-April/007024.html|mnesia glitch]] best sorted by zapping mnesia. {{{ |
|
Line 132: | Line 86: |
== lxc-start hangs == [[http://paste.ubuntu.com/772517/|The symptom looks like this]]. It hangs after that. No fix or workaround identified yet, other than making a new lxc container. To debug, try '''{{{lxc-start -n $containername -l debug -o outout}}}''' and look at output. == DNS fails inside the container == After restarting in daemon mode and logging in as a regular user, DNS was not working. Ensure there is a nameserver in the container's /etc/resolv.conf, which is created at startup by resolverconf. Stopping and starting the container solved the problem. == Random flakiness == Using lxc via juju I ran into all sorts of problems with DNS, version mismatches, etc. Since it was via juju I wasn't able to muck around with /etc/resolv.conf (the damage was done before I got the chance to ssh to the guest.) I found {{{sudo rm -rf /var/cache/lxc}}} solved the problem. It is rather brutal but worked. Of course the next run took a long time as all of that previously cached stuff had to be refetched. == Other problems == If other lxc users don't have an idea (known lxc users as of this writing include lifeless, wgrant, frankban and gary_poster) try asking hallyn or Spamaps on #ubuntu-server on freenode. |
This page explains how to set up and run Launchpad (for development) inside an LXC container.
Why?
Launchpad development setup makes significant changes to your machine; it's nice to be unaffected by those when you're not doing such development. Also, multiple containers can be used to work around Launchpad's limitations regarding concurrent test runs on a single machine.
These instructions should work on Ubuntu 12.04 LTS and later. Older versions of LXC are significantly less reliable and polished, so if you've used a version of LXC older than 12.04 LTS's final release on your development machine, you'll want to remove /var/cache/lxc first to ensure that you don't have a broken cache.
Create an LXC container
- Install LXC's userspace tools.
sudo apt-get install lxc
Create a container. You might want to use an HTTP proxy or alternate Ubuntu mirror; you can do this by specifying an http_proxy or MIRROR environment variable after sudo.
sudo lxc-create -t ubuntu -n lpdev -- -r lucid -a i386 -b $USER
- Start the container. You'll probably see a few early warnings about boot processes dying -- they're normal and can be ignored as long as you end up at a login prompt.
sudo lxc-start -n lpdev
[Inside the container] Log in with your normal username and password. You'll have full sudo powers.
[Inside the container] Get the container's IP address from the console. You'll generally want to ssh in for convenience and better termcap functionality, and the default Ubuntu LXC dnsmasq setup doesn't provide name resolution from the host
ip addr show dev eth0 | grep 'inet'
[Inside the container] IP addresses suck, so we should install Avahi to allow us to 'ssh lpdev.local' to get in. (Remove the previous point if Avahi works reliably everywhere).
sudo apt-get install avahi-daemon
[Inside the container] Install some additional packages needed before running rocketfuel-setup and the rest of the Launchpad setup process. The language pack is mandatory and must match your configured locale, or PostgreSQL will break.
sudo apt-get install bzr language-pack-en
[Inside the container] Shut down by running sudo poweroff inside the container, and you should eventually be dumped back out to your host system. If it looks like it's hanging, force it to stop with sudo lxc-stop -n lpdev on the host.
Start it up again, headless this time (-d). The same IP address will be used, so you don't need console access.
sudo lxc-start -n lpdev -d
ssh <IP address obtained above> to connect to the VM. If your SSH key is in your local authorized_keys file you shouldn't be prompted for a password, as your home directory (including public and private keys) is bind mounted into the container. ssh -A might give you a better agent experience when dealing with Launchpad code hosting.
[Inside the container] You can now follow the normal LP installation instructions. Be warned that changes in your home directory will also be seen outside the container and vice versa. If your home directory already has a Launchpad work area set up you'll want to run rocketfuel-setup --no-workspace to avoid trying to recreate it, but all subsequent steps are still required.
Follow Running/RemoteAccess to set up access from the host's applications to the container's Launchpad instance.
Problems
launchpad-database-setup fails
PostgreSQL will fail to create a cluster during installation if your locale is configured to something non-C but not supported by the container, so you need to install the relevant language pack.
You will know you need to do this if bzr or apt commands have been spewing locale warnings.
For instance, if your computer has a localised English locale, use this:
apt-get install language-pack-en
If you didn't install the language pack before running rocketfuel-setup, you'll need to run sudo pg_createcluster 8.4 main afterwards to fix the damage.
rabbitmq does not start up
rabbitmq may fail to start up. If that happens it appears to be a mnesia glitch best sorted by zapping mnesia.
sudo rm -rf /var/lib/rabbitmq/mnesia/rabbit/* sudo service rabbit-mq start
lxc-start hangs
The symptom looks like this. It hangs after that.
No fix or workaround identified yet, other than making a new lxc container.
To debug, try lxc-start -n $containername -l debug -o outout and look at output.
DNS fails inside the container
After restarting in daemon mode and logging in as a regular user, DNS was not working. Ensure there is a nameserver in the container's /etc/resolv.conf, which is created at startup by resolverconf. Stopping and starting the container solved the problem.
Random flakiness
Using lxc via juju I ran into all sorts of problems with DNS, version mismatches, etc. Since it was via juju I wasn't able to muck around with /etc/resolv.conf (the damage was done before I got the chance to ssh to the guest.) I found sudo rm -rf /var/cache/lxc solved the problem. It is rather brutal but worked. Of course the next run took a long time as all of that previously cached stuff had to be refetched.
Other problems
If other lxc users don't have an idea (known lxc users as of this writing include lifeless, wgrant, frankban and gary_poster) try asking hallyn or Spamaps on #ubuntu-server on freenode.
References