4404
Comment: Attach the scripts.
|
5543
|
Deletions are marked like this. | Additions are marked like this. |
Line 1: | Line 1: |
You're going to run Soyuz in a branch you create for the purpose. To get the whole experience, you'll also be installing the slave-side {{{launchpad-buildd}}} package on your system. |
|
Line 3: | Line 5: |
* Alter the following sections of the configuration (`configs/development/launchpad-lazr.conf`) as shown: | * Run {{{utilities/start-dev-soyuz.sh}}} to ensure that some Soyuz-related services are running. Some of these may already be running, in which case you'll get some failures that are probably harmless. Note: these services eat lots of memory. * Once you've set up your test database, run {{{utilities/soyuz-sampledata-setup.py -e you@example.com}}} (where ''you@example.com'' should be an email address you own and have a GPG key for). This prepares more suitable sample data in the {{{launchpad_dev}}} database, including recent Ubuntu series. If you get a "duplicate key" error, {{{make schema}}} and run again. * `make run` (or if you also want to use codehosting, `make run_codehosting`—some services may fail to start up because you already started them, but it shouldn't be a problem). * Open https://launchpad.dev/~ppa-user/+archive/test-ppa in a browser to get to your pre-made testing PPA. Log in with your own email adddress and password ''test''. This user has your GPG key associated, has signed the Ubuntu Code of Conduct, and is a member {{{ubuntu-team}}} (conferring upload rights to the primary archive). == Extra PPA dependencies == The testing PPA has an external dependency on Lucid. If that's not enough, or not what you want: * Log in as `admin@canonical.com:test` (I suggest using a different browser so you don't break up your ongoing session). * Open https://launchpad.dev/~ppa-user/+archive/test-ppa/+admin * Edit external dependencies. They normally look like: {{{ deb http://archive.ubuntu.com/ubuntu %(series)s main restricted universe multiverse }}} = Setup a build slave = == Installation == * Create a new trusty virtual-machine with kvm (recommended), or alternatively a trusty lxc container. If using lxc, set `lxc.aa_profile = unconfined` in `/var/lib/lxc/container-name/config` which is required to disable App``Armor support. If you are running Launchpad in a container, you will more than likely want your VM's network bridged on `lxcbr0`. In your slave vm/lxc: * `sudo apt-add-repository ppa:launchpad` * `sudo apt-get update` * `sudo apt-get install launchpad-buildd` Alternatively, launchpad-buildd can be built from `lp:launchpad-buildd` with `dpkg-buildpackage -b`. * Edit `/etc/launchpad-buildd/default` and make sure ntphost points to an existing NTP server. You can check the [[http://www.pool.ntp.org/|NTP server pool]] to find one near you. == Launchpad Configuration == From your host system: * Get an Ubuntu buildd chroot from Launchpad, using `manage-chroot` from [[https://code.launchpad.net/+branch/ubuntu-archive-tools|lp:ubuntu-archive-tools]]: * `manage-chroot -s precise -a i386 get` * `LP_DISABLE_SSL_CERTIFICATE_VALIDATION=1 manage-chroot -l dev -s precise -a i386 -f chroot-ubuntu-precise-i386.tar.bz2 set` * Point `url` to your slave's IP and mark Bob the Builder as OK (https://launchpad.dev/builders/bob/+edit) Bob, shortly thereafter should report a successful status of 'idle'. = Drive slave through rpc = With librarian running, fire up a python shell and: |
Line 6: | Line 59: |
[archivepublisher] root: /var/tmp/archive base_url: http://archive.launchpad.dev/ # <- this [builddmaster] root: /var/tmp/builddmaster/ uploader: scripts/process-upload.py -Mvv --context buildd # <- and this |
import xmlrpclib proxy = xmlrpclib.ServerProxy('http://localhost:8221/rpc') proxy.ensurepresent('d267a7b39544795f0e98d00c3cf7862045311464', 'http://launchpad.dev:58080/93/chroot-ubuntu-lucid-i386.tar.bz2', '', '') proxy.build('1-1', 'translation-templates', 'd267a7b39544795f0e98d00c3cf7862045311464', {}, {'archives': ['deb http://archive.ubuntu.com/ubuntu/ lucid main'], 'branch_url': '/home/buildd/gimp-2.6.8'}) proxy.status() proxy.clean() # Clean up if it failed |
Line 15: | Line 68: |
* In /etc/hosts, point `archive.launchpad.dev` at 127.0.0.88 like the rest. * Add to `/etc/apache2/site-available/local-launchpad`: {{{ <VirtualHost 127.0.0.88:80> ServerName archive.launchpad.dev LogLevel debug DocumentRoot /var/tmp/archive <Directory /var/tmp/archive/> Order Deny,Allow Deny from all Allow from 127.0.0.0/255.0.0.0 Options Indexes </Directory> </VirtualHost> }}} * Grab and run [[attachment:make-ubuntu-sane.py|make-ubuntu-sane.py]], to set up some Ubuntu series that still exist, along with some state such as permissions. * Grab and run [[attachment:start-soyuz.sh|start-soyuz.sh]], which will fire up all Soyuz services. This will eat lots of RAM. = Configure an account and PPA = * `make run` * Create an account for yourself (`utilities/make-lp-user USERNAME`) * Log in with `USERNAME@example.com:test`. * Add your OpenPGP key * `gpg --keyserver keyserver.launchpad.dev --send-keys DEADBEEF` * Add it to your Launchpad account, as normal. * Check email in `root`'s mailbox, and use it to activate the key. * Sign the code of conduct. * Create a PPA. = Set up the PPA = * Log out. * Log in as `admin@example.com:test`. * Browse to the new PPA. * Click "Administer". * Ensure that the PPA is allowed to build on non-virtual builders. * Add an external dependency so you don't have to import the entire Ubuntu archive just for a PPA to work: {{{ deb http://archive.ubuntu.com/ubuntu %(series) main restricted universe multiverse }}} = Configure a buildd = * `cd lib/canonical/buildd` * `dpkg-buildpackage -b` * `sudo dpkg -i ../launchpad-buildd_50_all.deb` * Make it work. * Add `$SUDO cp /etc/resolv.conf /etc/hosts "$HOME/build-$BUILDID/chroot-autobuild/etc` to the end of `/usr/share/launchpad-buildd/slavebin/mount-chroot`, so name resolution works. * Might need to add `--umask=022` to `twistd` args in `/etc/init.d/launchpad-buildd`, or `pkg-create-dbgsym` fails obscurely. * (>= Karmic) Might need to alter "`dpkg-source: extracting`" output grepping in sbuild to "`dpkg-source: info: extracting`" * Get an Ubuntu buildd chroot. * Look for the latest build of your favourite architecture and series at https://launchpad.net/ubuntu/+builds. * Find the chroot SHA1 in the third line of the log. * Search for it at http://launchpadlibrarian.net/search?digest=SOMESHA1 * Grab one of the listed URLs. * http://launchpadlibrarian.net/34617356/chroot-ubuntu-lucid-i386.tar.bz2 is the current lucid i386 chroot at the time of writing. * `scripts/ftpmaster-tools/manage-chroot.py -s lucid -a i386 add -f chroot-ubuntu-lucid-i386.tar.bz2` * Mark Bob the Builder as OK (https://launchpad.dev/builders/bob/+edit) |
You may have to calculate a new sha1sum of the chroot file. |
Line 80: | Line 72: |
* Run `scripts/process-upload.py /var/tmp/poppy` (creates hierarchy) | * Run `scripts/process-upload.py /var/tmp/txpkgupload` (creates hierarchy) |
Line 91: | Line 83: |
* `dput lpdev:~USERNAME/PPANAME/ubuntu some_source.changes` * `scripts/process-upload.py /var/tmp/poppy # Accept the source upload.` * Within five seconds of upload acceptance, the buildd should start building. Wait until it is complete. * `scripts/process-accepted.py --ppa ubuntu # Create publishings for the binaries.` * `scripts/publish-distro.py --ppa # Publish the source and binaries.` |
* Find a source package `some_source` with a changes file `some_source.changes` * `dput -u lpdev:~ppa-user/test-ppa/ubuntu some_source.changes` * `scripts/process-upload.py /var/tmp/txpkgupload -C absolutely-anything -vvv # Accept the source upload.` * If this is your first time running soyuz locally, you'll also need to publish ubuntu: `scripts/publish-distro.py -C` * Within five seconds of upload acceptance, the buildd should start building. Wait until it is complete (the build page will say "Uploading build"). * `scripts/process-upload.py -vvv --builds -C buildd /var/tmp/builddmaster # Process the build upload.` * `scripts/process-accepted.py -vv --ppa ubuntu # Create publishings for the binaries.` * `scripts/publish-distro.py -vv --ppa # Publish the source and binaries.` |
Line 100: | Line 95: |
* Add yourself to `~ubuntu-team`. | |
Line 102: | Line 96: |
* `scripts/process-upload.py /var/tmp/poppy` | * `scripts/process-upload.py -vvv /var/tmp/txpkgupload` |
Line 106: | Line 100: |
* `scripts/process-accepted.py ubuntu` * `scripts/publish-distro.py` |
* `scripts/process-accepted.py -vv ubuntu` * `scripts/publish-distro.py -vv` |
You're going to run Soyuz in a branch you create for the purpose. To get the whole experience, you'll also be installing the slave-side launchpad-buildd package on your system.
Initial setup
Run utilities/start-dev-soyuz.sh to ensure that some Soyuz-related services are running. Some of these may already be running, in which case you'll get some failures that are probably harmless. Note: these services eat lots of memory.
Once you've set up your test database, run utilities/soyuz-sampledata-setup.py -e you@example.com (where you@example.com should be an email address you own and have a GPG key for). This prepares more suitable sample data in the launchpad_dev database, including recent Ubuntu series. If you get a "duplicate key" error, make schema and run again.
make run (or if you also want to use codehosting, make run_codehosting—some services may fail to start up because you already started them, but it shouldn't be a problem).
Open https://launchpad.dev/~ppa-user/+archive/test-ppa in a browser to get to your pre-made testing PPA. Log in with your own email adddress and password test. This user has your GPG key associated, has signed the Ubuntu Code of Conduct, and is a member ubuntu-team (conferring upload rights to the primary archive).
Extra PPA dependencies
The testing PPA has an external dependency on Lucid. If that's not enough, or not what you want:
Log in as admin@canonical.com:test (I suggest using a different browser so you don't break up your ongoing session).
Open https://launchpad.dev/~ppa-user/+archive/test-ppa/+admin
- Edit external dependencies. They normally look like:
deb http://archive.ubuntu.com/ubuntu %(series)s main restricted universe multiverse
Setup a build slave
Installation
Create a new trusty virtual-machine with kvm (recommended), or alternatively a trusty lxc container. If using lxc, set lxc.aa_profile = unconfined in /var/lib/lxc/container-name/config which is required to disable AppArmor support.
If you are running Launchpad in a container, you will more than likely want your VM's network bridged on lxcbr0.
In your slave vm/lxc:
sudo apt-add-repository ppa:launchpad
sudo apt-get update
sudo apt-get install launchpad-buildd
Alternatively, launchpad-buildd can be built from lp:launchpad-buildd with dpkg-buildpackage -b.
Edit /etc/launchpad-buildd/default and make sure ntphost points to an existing NTP server. You can check the NTP server pool to find one near you.
Launchpad Configuration
From your host system:
Get an Ubuntu buildd chroot from Launchpad, using manage-chroot from lp:ubuntu-archive-tools:
manage-chroot -s precise -a i386 get
LP_DISABLE_SSL_CERTIFICATE_VALIDATION=1 manage-chroot -l dev -s precise -a i386 -f chroot-ubuntu-precise-i386.tar.bz2 set
Point url to your slave's IP and mark Bob the Builder as OK (https://launchpad.dev/builders/bob/+edit)
Bob, shortly thereafter should report a successful status of 'idle'.
Drive slave through rpc
With librarian running, fire up a python shell and:
import xmlrpclib proxy = xmlrpclib.ServerProxy('http://localhost:8221/rpc') proxy.ensurepresent('d267a7b39544795f0e98d00c3cf7862045311464', 'http://launchpad.dev:58080/93/chroot-ubuntu-lucid-i386.tar.bz2', '', '') proxy.build('1-1', 'translation-templates', 'd267a7b39544795f0e98d00c3cf7862045311464', {}, {'archives': ['deb http://archive.ubuntu.com/ubuntu/ lucid main'], 'branch_url': '/home/buildd/gimp-2.6.8'}) proxy.status() proxy.clean() # Clean up if it failed
You may have to calculate a new sha1sum of the chroot file.
Upload a source to the PPA
Run scripts/process-upload.py /var/tmp/txpkgupload (creates hierarchy)
Add to ~/.dput.cf:
[lpdev] fqdn = ppa.launchpad.dev:2121 method = ftp incoming = %(lpdev)s login = anonymous
Find a source package some_source with a changes file some_source.changes
dput -u lpdev:~ppa-user/test-ppa/ubuntu some_source.changes
scripts/process-upload.py /var/tmp/txpkgupload -C absolutely-anything -vvv # Accept the source upload.
If this is your first time running soyuz locally, you'll also need to publish ubuntu: scripts/publish-distro.py -C
- Within five seconds of upload acceptance, the buildd should start building. Wait until it is complete (the build page will say "Uploading build").
scripts/process-upload.py -vvv --builds -C buildd /var/tmp/builddmaster # Process the build upload.
scripts/process-accepted.py -vv --ppa ubuntu # Create publishings for the binaries.
scripts/publish-distro.py -vv --ppa # Publish the source and binaries.
- Note that private archive builds will not be dispatched until their source is published.
Dealing with the primary archive
dput lpdev:ubuntu some_source.changes
scripts/process-upload.py -vvv /var/tmp/txpkgupload
- Watch the output -- the upload might end up in NEW.
- If it does, go to the queue and accept it.
- Your builder should now be busy. Once it finishes, the binaries might go into NEW. Accept them if required.
scripts/process-accepted.py -vv ubuntu
scripts/publish-distro.py -vv
The first time, add -C to ensure a full publication of the archive.