Diff for "Running/LXC"

Not logged in - Log In / Register
Differences between revisions 23 and 92 (spanning 69 versions)
⇤ ← Revision 23 as of 2011-06-22 06:40:59 →
Size: 4028
Editor: lifeless
Comment: create the user
← Revision 92 as of 2012-05-03 04:26:11 → ⇥
Size: 5402
Editor: wgrant
Comment: Note that pre-12.04 LTS LXCs are likely to have created a flaky cache
Deletions are marked like this. Additions are marked like this.
Line 1: Line 1:
## page was copied from Running/VirtualMachine
Line 6: Line 5:
Launchpad development setup makes significant changes to your machine; its nice to be unaffected by those except when you are actually doing such development. Launchpad development setup makes significant changes to your machine; its nice to be unaffected by those except when you are actually doing such development. Also, Launchpad has some limitations on concurrent testing per-machine and so forth - multiple containers can be used to work around this.
Line 8: Line 7:
Also, launchpad has some limitations on concurrent testing per-machine and so forth - multiple container's can be used to work around this.
 These instructions should work on Ubuntu 12.04 LTS and later. If you've used a version of LXC older than 12.04 LTS's final release on your development machine, you'll want to remove `/var/cache/lxc` to ensure that you don't have a broken cache.
Line 14: Line 12:

{{{		  {{{
Line 19: Line 16:
 1. Work around Bug:800456
{{{
sudo apt-get install cgroup-bin		  1. Create a container. In the following commands, replace ''$username'' with your username on the host.
 {{{
sudo lxc-create -t ubuntu -n lpdev -- -r lucid -a i386 -b $username
Line 24: Line 21:
 1. Work around Bug:784093     * If you want a proxy:
 {{{
sudo http_proxy=http://host:port/ lxc-create -t ubuntu -n lpdev -- -r lucid -a i386 -b $username
}}}
 
    * And if you want to set a custom mirror, similar to http_proxy, but set MIRROR= instead.

 1. Start the container
 {{{
sudo lxc-start -n lpdev
}}}
    Ignore the warning about openssh crashing - it restarts on a later event.
    
 1. '''[Inside the container]''' Log in with your normal username and password. You'll have full sudo powers.

 1. '''[Inside the container]''' Grab the IP address (handed out via LXC's dnsmasq DHCP server) - you may wish to SSH in rather than using the console (seems to have better termcap experience).
 {{{
ip addr show dev eth0 | grep 'inet'
}}}

 1. '''[Inside the container]''' Install some additional packages we'll need to run rocketfuel-setup etc. Most people with an English locale will simply want to do this: `apt-get install bzr language-pack-en`

    If your locale is not English, or if you want more details, try/read this.
 {{{
apt-get install bzr
# if you have a localised (non-C) locale:
# not doing this will cause postgresql to fail to install, with -hilarious- results as database-developer-setup will think you have 8.2 installed.
# You can tell if you need this if the prior apt commands spewed locale warnings.
# Pick your specific language pack.
apt-get install language-pack-en
}}}

 1. To stop it now run 'poweroff' in the lxc container. If it works smoothly, you will eventually be dumped back out to your host system. If it looks like it is hanging, then use "{{{sudo lxc-stop -n lpdev}}}" in the host.
 
 1. Start it up again, headless this time (-d). The previous IP address will be used.
 {{{
sudo lxc-start -n lpdev -d
}}}

 1. ssh <vm IP address> to connect to the VM. Your ssh key is already present because of the bind mount to your home dir, though using ssh -A might give you a better ssh agent experience.

 1. You can now follow the [[Getting|getting-started]] on LP instructions. Be warned that changes in ~ will affect you outside the container. You will want to run rocketfuel-setup with --no-workspace if your home already has a workarea. You may need to run utilities/launchpad-database-setup separately too.

 1. You probably want to follow [[Running/RemoteAccess]] has a discussion for how you can configure things so your non-container browser can access web pages from within the container.

= Problems =

== rabbitmq does not start up ==

rabbitmq may fail to start up. If that happens it appears to be a [[http://lists.rabbitmq.com/pipermail/rabbitmq-discuss/2010-April/007024.html|mnesia glitch]] best sorted by zapping mnesia.
 {{{
sudo rm -rf /var/lib/rabbitmq/mnesia/rabbit/*
sudo service rabbit-mq start
}}}

== database-developer-setup fails, and thinks you are on Postgres 8.2 ==

As noted above, if you have a localised (non-C) locale, you need to install your specific language pack. For instance, if your computer has a localised English locale, use this:
Line 27: Line 81:
sudo dd of=/etc/cgconfig.conf << EOF
mount {
 cpu = /sys/fs//cgroup/cpu;
 cpuacct = /sys/fs/cgroup/cpu;
 devices = /sys/fs/cgroup/cpu;
 memory = /sys/fs/cgroup/cpu;
}
EOF
sudo service cgconfig restart		 apt-get install language-pack-en
Line 38: Line 84:
 1. Work around Bug:798476 (optional if you run i386 or have a -tonne- of memory and don't care about 64-bit footprint.
    Grab the patch from the bug and apply it to /usr/lib/lxc/templates/lxc-lucid. If you're running i386 already or want a 64-bit lxc then do not pass arch= on the lxc-create command line.		 == lxc-start hangs ==
Line 41: Line 86:
 1. Create a config for your containers
{{{
sudo dd of=/etc/lxc/local.conf << EOF
lxc.network.type=veth
lxc.network.link=virbr0
lxc.network.flags=up
EOF
}}}		 [[http://paste.ubuntu.com/772517/|The symptom looks like this]]. It hangs after that.
Line 50: Line 88:
 1. Create a container
{{{
sudo arch=i386 lxc-create -n lucid-test-lp -t lucid -f /etc/lxc/local.conf
}}}
    If you want to use a proxy
{{{
sudo arch=i386 http_proxy=http://host:port/ lxc-create -n lucid-test-lp -t lucid -f /etc/lxc/local.conf
}}}
    And if you want to set a custom mirror, similar to http_proxy, but set MIRROR= instead.		 No fix or workaround identified yet, other than making a new lxc container.
Line 60: Line 90:
 1. Start it
{{{
sudo lxc-start -n lucid-test-lp
}}}
    Ignore the warning about openssh crashing - it restarts on a later event.
    The initial credentials are root:root.		 To debug, try '''{{{lxc-start -n $containername -l debug -o outout}}}''' and look at outout.
Line 67: Line 92:
 1. To stop it log in and run 'poweroff -n'.
 
 1. The new container won't have your proxy / mirror settings preserved. Customise it at this point before going further if you care about this.		 == DNS fails inside the container ==
Line 71: Line 94:
 1. Grab your user id and username so you can setup a bind mount outside the container:
{{{
id -u
id -nu
}}}		 After restarting in daemon mode and logging in as a regular user, DNS was not working.
Ensure there is a nameserver in the container's /etc/resolv.conf, which is created at startup by resolverconf. Stopping and starting the container solved the problem.
Line 77: Line 97:
 1. Inside the container add the user:
{{{
adduser --uid $id $username
}}}		 == Random flakiness ==
Line 82: Line 99:
 1. below this is not yet updated from the vm instructions Using lxc via juju I ran into all sorts of problems with DNS, version mismatches, etc. Since it was via juju I wasn't able to muck around with /etc/resolv.conf (the damage was done before I got the chance to ssh to the guest.) I found {{{sudo rm -rf /var/cache/lxc}}} solved the problem. It is rather brutal but worked. Of course the next run took a long time as all of that previously cached stuff had to be refetched.
Line 84: Line 101:
 1. After its installed, connect to the image and install {{{acpid}}} and {{{openssh-server}}} == Other problems ==
Line 86: Line 103:
 1. Use ssh-copy-id to copy your public key into the VM.

 1. ssh -A <vm IP address> to connect to the VM.

 1. {{{bzr whoami "Your Name <your.email@example.com>"}}} to set your bzr identity in the VM.

 1. You can now follow the [[Getting|getting-started]] on LP instructions.		 If other lxc users don't have an idea (known lxc users as of this writing include lifeless, wgrant, frankban and gary_poster) try asking hallyn or Spamaps on #ubuntu-server on freenode.
Line 96: Line 107:
See also this email thread about [[https://lists.launchpad.net/launchpad-dev/msg03456.html|running Launchpad in a virtual machine]], and this [[https://lists.launchpad.net/launchpad-dev/msg03454.html|discussion of the differences]] between running in a [[Running/Schroot|chroot]] environment and running a VM. [[Running/RemoteAccess]] has a discussion for how you can configure the VM to allow the host machine to access the web pages, etc.
Line 100: Line 110:
You can skip some manual steps of installing from an ISO using a command like this:

{{{
sudo ubuntu-vm-builder kvm lucid --domain vm --dest ~/vm/lp-dev \
 --hostname lp-dev \
 --mem 2048 --cpus 2 \
 --components main,universe,multiverse,restricted \
 --mirror http://10.113.3.35:3142/mirror.internode.on.net/pub/ubuntu/ubuntu \
 --libvirt qemu:///system \
 --debug -v \
 --ssh-user-key ~/.ssh/id_rsa.pub --ssh-key ~/.ssh/id_rsa.pub \
 --rootsize 24000 \
 --user $USER
}}}

After installation completes, it should show up in your virt-manager menu.

= In LXC =

It seems like it would be nice to run Launchpad in [[http://lxc.teegra.net/|LXC containers]]: they should be more efficient than a VM (especially with regard to memory and disk) but more isolated than a chroot. More testing or documentation is needed.		 You can also run in a [[Running/Schroot|chroot]] environment or a [[Running/VirtualMachine|VM]].

This page explains how to set up and run Launchpad (for development) inside a LXC.

Why?

Launchpad development setup makes significant changes to your machine; its nice to be unaffected by those except when you are actually doing such development. Also, Launchpad has some limitations on concurrent testing per-machine and so forth - multiple containers can be used to work around this.

These instructions should work on Ubuntu 12.04 LTS and later. If you've used a version of LXC older than 12.04 LTS's final release on your development machine, you'll want to remove /var/cache/lxc to ensure that you don't have a broken cache.

Make a LXC

  1. Install lxc 
    sudo apt-get install lxc

  2. Create a container. In the following commands, replace $username with your username on the host. 

    sudo lxc-create -t ubuntu -n lpdev -- -r lucid -a i386 -b $username
    • If you want a proxy: 
    sudo http_proxy=http://host:port/ lxc-create -t ubuntu -n lpdev -- -r lucid -a i386 -b $username
    • And if you want to set a custom mirror, similar to http_proxy, but set MIRROR= instead.
  3. Start the container 
    sudo lxc-start -n lpdev
    • Ignore the warning about openssh crashing - it restarts on a later event.

  4. [Inside the container] Log in with your normal username and password. You'll have full sudo powers.

  5. [Inside the container] Grab the IP address (handed out via LXC's dnsmasq DHCP server) - you may wish to SSH in rather than using the console (seems to have better termcap experience). 

    ip addr show dev eth0 | grep 'inet'

  6. [Inside the container] Install some additional packages we'll need to run rocketfuel-setup etc. Most people with an English locale will simply want to do this: apt-get install bzr language-pack-en

    • If your locale is not English, or if you want more details, try/read this. 
    apt-get install bzr
# if you have a localised (non-C) locale:
# not doing this will cause postgresql to fail to install, with -hilarious- results as database-developer-setup will think you have 8.2 installed.
# You can tell if you need this if the prior apt commands spewed locale warnings.
# Pick your specific language pack.
apt-get install language-pack-en

  7. To stop it now run 'poweroff' in the lxc container. If it works smoothly, you will eventually be dumped back out to your host system. If it looks like it is hanging, then use "sudo lxc-stop -n lpdev" in the host.

  8. Start it up again, headless this time (-d). The previous IP address will be used. 
    sudo lxc-start -n lpdev -d

  9. ssh <vm IP address> to connect to the VM. Your ssh key is already present because of the bind mount to your home dir, though using ssh -A might give you a better ssh agent experience.

  10. You can now follow the getting-started on LP instructions. Be warned that changes in ~ will affect you outside the container. You will want to run rocketfuel-setup with --no-workspace if your home already has a workarea. You may need to run utilities/launchpad-database-setup separately too.

  11. You probably want to follow Running/RemoteAccess has a discussion for how you can configure things so your non-container browser can access web pages from within the container.

Problems

rabbitmq does not start up

rabbitmq may fail to start up. If that happens it appears to be a mnesia glitch best sorted by zapping mnesia. 

  • sudo rm -rf /var/lib/rabbitmq/mnesia/rabbit/*
sudo service rabbit-mq start

database-developer-setup fails, and thinks you are on Postgres 8.2

As noted above, if you have a localised (non-C) locale, you need to install your specific language pack. For instance, if your computer has a localised English locale, use this: 

apt-get install language-pack-en

lxc-start hangs

The symptom looks like this. It hangs after that.

No fix or workaround identified yet, other than making a new lxc container.

To debug, try lxc-start -n $containername -l debug -o outout and look at outout.

DNS fails inside the container

After restarting in daemon mode and logging in as a regular user, DNS was not working. Ensure there is a nameserver in the container's /etc/resolv.conf, which is created at startup by resolverconf. Stopping and starting the container solved the problem.

Random flakiness

Using lxc via juju I ran into all sorts of problems with DNS, version mismatches, etc. Since it was via juju I wasn't able to muck around with /etc/resolv.conf (the damage was done before I got the chance to ssh to the guest.) I found sudo rm -rf /var/cache/lxc solved the problem. It is rather brutal but worked. Of course the next run took a long time as all of that previously cached stuff had to be refetched.

Other problems

If other lxc users don't have an idea (known lxc users as of this writing include lifeless, wgrant, frankban and gary_poster) try asking hallyn or Spamaps on #ubuntu-server on freenode.

References

Alternatively

You can also run in a chroot environment or a VM.