Diff for "Soyuz/HowToUseSoyuzLocally"

Not logged in - Log In / Register
Differences between revisions 2 and 56 (spanning 54 versions)
⇤ ← Revision 2 as of 2009-11-05 12:33:24 →
Size: 4409
Editor: wgrant
Comment: Fix bad start-soyuz.sh URL.
← Revision 56 as of 2011-05-07 08:17:26 → ⇥
Size: 4528
Editor: wgrant
Comment: edge is dead.
Deletions are marked like this. Additions are marked like this.
Line 1: Line 1:
You're going to run Soyuz in a branch you create for the purpose. To get the whole experience, you'll also be installing the slave-side {{{launchpad-buildd}}} package on your system.
Line 3: Line 5:
 * Alter the following sections of the configuration (`configs/development/launchpad-lazr.conf`) as shown:  * Run {{{utilities/start-dev-soyuz.sh}}} to ensure that some Soyuz-related services are running. Some of these may already be running, in which case you'll get some failures that are probably harmless. Note: these services eat lots of memory.
 * Once you've set up your test database, run {{{utilities/soyuz-sampledata-setup.py -e you@example.com}}} (where ''you@example.com'' should be an email address you own and have a GPG key for). This prepares more suitable sample data in the {{{launchpad_dev}}} database, including recent Ubuntu series. If you get a "duplicate key" error, {{{make schema}}} and run again.
 * `make run` (or if you also want to use codehosting, `make run_codehosting`—some services may fail to start up because you already started them, but it shouldn't be a problem).
 * Open https://launchpad.dev/~ppa-user/+archive/test-ppa in a browser to get to your pre-made testing PPA. Log in with your own email adddress and password ''test''. This user has your GPG key associated, has signed the Ubuntu Code of Conduct, and is a member {{{ubuntu-team}}} (conferring upload rights to the primary archive).
Line 5: Line 10:

== Extra PPA dependencies ==

The testing PPA has an external dependency on Lucid. If that's not enough, or not what you want:

 * Log in as `admin@canonical.com:test` (I suggest using a different browser so you don't break up your ongoing session).
 * Open https://launchpad.dev/~ppa-user/+archive/test-ppa/+admin
 * Edit external dependencies. They normally look like:
Line 6: Line 19:
[archivepublisher]
root: /var/tmp/archive
base_url: http://archive.launchpad.dev/ # <- this

[builddmaster]
root: /var/tmp/builddmaster/
uploader: scripts/process-upload.py -Mvv --context buildd # <- and this		 deb http://archive.ubuntu.com/ubuntu %(series)s main restricted universe multiverse
Line 15: Line 22:
 * In /etc/hosts, point `archive.launchpad.dev` at 127.0.0.88 like the rest.
 * Add to `/etc/apache2/site-available/local-launchpad`:

{{{
 <VirtualHost 127.0.0.88:80>
  ServerName archive.launchpad.dev
  LogLevel debug
  DocumentRoot /var/tmp/archive
  <Directory /var/tmp/archive/>
   Order Deny,Allow
   Deny from all
   Allow from 127.0.0.0/255.0.0.0
   Options Indexes
  </Directory>
 </VirtualHost>
}}}

 * Grab and run http://williamgrant.id.au/f/1/2009/make-ubuntu-sane.py, to set up some Ubuntu series that still exist, along with some state such as permissions.
 * Grab and run http://williamgrant.id.au/f/1/2009/start-soyuz.sh, which will fire up all Soyuz services. This will eat lots of RAM.

= Configure an account and PPA =

 * `make run`
 * Create an account for yourself (`utilities/make-lp-user USERNAME`)
 * Log in with `USERNAME@example.com:test`.
 * Add your OpenPGP key
  * `gpg --keyserver keyserver.launchpad.dev --send-keys DEADBEEF`
  * Add it to your Launchpad account, as normal.
  * Check email in `root`'s mailbox, and use it to activate the key.
 * Sign the code of conduct.
 * Create a PPA.

= Set up the PPA =

 * Log out.
 * Log in as `admin@example.com:test`.
 * Browse to the new PPA.
 * Click "Administer".
 * Ensure that the PPA is allowed to build on non-virtual builders.
 * Add an external dependency so you don't have to import the entire Ubuntu archive just for a PPA to work:

{{{
deb http://archive.ubuntu.com/ubuntu %(series) main restricted universe multiverse
}}}
Line 62: Line 25:
This assumes a local install of the {{{launchpad-buildd}}} package. A less intrusive setup using chroot is under [[BuildFarm/TryOutBuildSlave|TryOutBuildSlave]].
Line 63: Line 28:
 * `debian/rules package`
Line 64: Line 30:
 * `sudo dpkg -i ../launchpad-buildd_50_all.deb`  * `sudo dpkg -i ../launchpad-buildd_*_all.deb`
 * `sudo apt-get -f install`
Line 66: Line 33:
  * Add `$SUDO cp /etc/resolv.conf /etc/hosts "$HOME/build-$BUILDID/chroot-autobuild/etc` to the end of `/usr/share/launchpad-buildd/slavebin/mount-chroot`, so name resolution works.
  * Might need to add `--umask=022` to `twistd` args in `/etc/init.d/launchpad-buildd`, or `pkg-create-dbgsym` fails obscurely.
  * (>= Karmic) Might need to alter "`dpkg-source: extracting`" output grepping in sbuild to "`dpkg-source: info: extracting`"
 * Get an Ubuntu buildd chroot.
  * Look for the latest build of your favourite architecture and series at https://launchpad.net/ubuntu/+builds.
  * Find the chroot SHA1 in the third line of the log.
  * Search for it at http://launchpadlibrarian.net/search?digest=SOMESHA1
  * Grab one of the listed URLs.
   * http://launchpadlibrarian.net/34617356/chroot-ubuntu-lucid-i386.tar.bz2 is the current lucid i386 chroot at the time of writing. 		  * Edit `/etc/launchpad-buildd/default` and make sure `ntphost` points to an existing NTP server. You can check the [[http://www.pool.ntp.org/|NTP server pool]] to find one near you.
 * Get an Ubuntu buildd chroot from Launchpad
  * Find the URL at https://launchpad.net/api/devel/ubuntu/lucid/i386/chroot_url (or similar).
  * If you want lucid and your architecture is i386, download the tarball with:
{{{
wget -O - -q https://launchpad.net/api/devel/ubuntu/lucid/i386/chroot_url | xargs wget
}}}
Line 80: Line 45:
 * Run `scripts/process-upload.py /var/tmp/poppy` (creates hierarchy)  * Run `scripts/process-upload.py /var/tmp/poppy` (creates hierarchy) 
Line 91: Line 56:
 * `dput lpdev:~USERNAME/PPANAME/ubuntu some_source.changes`
 * `scripts/process-upload.py /var/tmp/poppy # Accept the source upload.`
 * Within five seconds of upload acceptance, the buildd should start building. Wait until it is complete.
 * `scripts/process-accepted.py --ppa ubuntu # Create publishings for the binaries.`
 * `scripts/publish-distro.py --ppa # Publish the source and binaries.`		  * Find a source package `some_source` with a changes file `some_source.changes`
 * `dput -u lpdev:~ppa-user/test-ppa/ubuntu some_source.changes`
 * `scripts/process-upload.py /var/tmp/poppy -C absolutely-anything -vvv # Accept the source upload.`
 * If this is your first time running soyuz locally, you'll also need to publish ubuntu: `scripts/publish-distro.py -C`
 * Within five seconds of upload acceptance, the buildd should start building. Wait until it is complete (the build page will say "Uploading build").
 * `scripts/process-upload.py -vvv --builds -C buildd /var/tmp/builddmaster # Process the build upload.`
 * `scripts/process-accepted.py -vv --ppa ubuntu # Create publishings for the binaries.`
 * `scripts/publish-distro.py -vv --ppa # Publish the source and binaries.`
Line 100: Line 68:
 * Add yourself to `~ubuntu-team`.
Line 102: Line 69:
 * `scripts/process-upload.py /var/tmp/poppy`  * `scripts/process-upload.py -vvv /var/tmp/poppy`
Line 106: Line 73:
 * `scripts/process-accepted.py ubuntu`
 * `scripts/publish-distro.py`		  * `scripts/process-accepted.py -vv ubuntu`
 * `scripts/publish-distro.py -vv`

You're going to run Soyuz in a branch you create for the purpose. To get the whole experience, you'll also be installing the slave-side launchpad-buildd package on your system.

Initial setup

  • Run utilities/start-dev-soyuz.sh to ensure that some Soyuz-related services are running. Some of these may already be running, in which case you'll get some failures that are probably harmless. Note: these services eat lots of memory.

  • Once you've set up your test database, run utilities/soyuz-sampledata-setup.py -e you@example.com (where you@example.com should be an email address you own and have a GPG key for). This prepares more suitable sample data in the launchpad_dev database, including recent Ubuntu series. If you get a "duplicate key" error, make schema and run again.

  • make run (or if you also want to use codehosting, make run_codehosting—some services may fail to start up because you already started them, but it shouldn't be a problem).

  • Open https://launchpad.dev/~ppa-user/+archive/test-ppa in a browser to get to your pre-made testing PPA. Log in with your own email adddress and password test. This user has your GPG key associated, has signed the Ubuntu Code of Conduct, and is a member ubuntu-team (conferring upload rights to the primary archive).

Extra PPA dependencies

The testing PPA has an external dependency on Lucid. If that's not enough, or not what you want: 

deb http://archive.ubuntu.com/ubuntu %(series)s main restricted universe multiverse

Configure a buildd

This assumes a local install of the launchpad-buildd package. A less intrusive setup using chroot is under TryOutBuildSlave.

  • cd lib/canonical/buildd

  • debian/rules package

  • dpkg-buildpackage -b

  • sudo dpkg -i ../launchpad-buildd_*_all.deb

  • sudo apt-get -f install

  • Make it work.

    • Edit /etc/launchpad-buildd/default and make sure ntphost points to an existing NTP server. You can check the NTP server pool to find one near you.

  • Get an Ubuntu buildd chroot from Launchpad 

wget -O - -q https://launchpad.net/api/devel/ubuntu/lucid/i386/chroot_url | xargs wget

Upload a source to the PPA

  • Run scripts/process-upload.py /var/tmp/poppy (creates hierarchy)

  • Add to ~/.dput.cf: 

[lpdev]
fqdn = ppa.launchpad.dev:2121
method = ftp
incoming = %(lpdev)s
login = anonymous

  • Find a source package some_source with a changes file some_source.changes

  • dput -u lpdev:~ppa-user/test-ppa/ubuntu some_source.changes

  • scripts/process-upload.py /var/tmp/poppy -C absolutely-anything -vvv # Accept the source upload.

  • If this is your first time running soyuz locally, you'll also need to publish ubuntu: scripts/publish-distro.py -C

  • Within five seconds of upload acceptance, the buildd should start building. Wait until it is complete (the build page will say "Uploading build").

  • scripts/process-upload.py -vvv --builds -C buildd /var/tmp/builddmaster # Process the build upload.

  • scripts/process-accepted.py -vv --ppa ubuntu # Create publishings for the binaries.

  • scripts/publish-distro.py -vv --ppa # Publish the source and binaries.

    • Note that private archive builds will not be dispatched until their source is published.

Dealing with the primary archive

  • dput lpdev:ubuntu some_source.changes

  • scripts/process-upload.py -vvv /var/tmp/poppy

  • Watch the output -- the upload might end up in NEW.
    • If it does, go to the queue and accept it.
  • Your builder should now be busy. Once it finishes, the binaries might go into NEW. Accept them if required.

  • scripts/process-accepted.py -vv ubuntu

  • scripts/publish-distro.py -vv

    • The first time, add -C to ensure a full publication of the archive.

Soyuz/HowToUseSoyuzLocally (last edited 2022-12-10 08:09:22 by jugmac00)