Diff for "Soyuz/HowToUseSoyuzLocally"

Not logged in - Log In / Register

Differences between revisions 41 and 42
Revision 41 as of 2010-03-05 11:38:44
Size: 8202
Editor: jtv
Comment:
Revision 42 as of 2010-03-08 08:00:39
Size: 7422
Editor: jtv
Comment:
Deletions are marked like this. Additions are marked like this.
Line 1: Line 1:
= Initial setup = = Setting up a local Soyuz =
Line 5: Line 5:
/!\ These scripts are being replaced with ones that live in the Launchpad tree. The new scripts will take away a lot of the manual work. —jtv  * Run {{{utilities/start-dev-soyuz.sh}}} to ensure that some Soyuz-related services are running. Some of these may already be running, in which case you'll get some failures that are probably harmless. Note: these services eat lots of memory.
 * Once you've set up your test database, run {{{utilities/soyuz-sampledata-setup.py -e you@example.com}}} (where ''you@example.com'' should be an email address you own and have a GPG key for). This prepares more suitable sample data in the {{{launchpad_dev}}} database. If you get a "duplicate key" error, {{{make schema}}} and run again.
Line 7: Line 8:
 * Once you've set up your test database, run {{{utilities/soyuz-sampledata-cleanup.py}}} to set up some Ubuntu series that still exist, along with some state such as permissions ([[#amd64chroots|Please see note on amd64 chroots]]). ''(Adding --amd64 option to script per bug Bug:527170 —jtv)''
 * Grab and run [[attachment:start-soyuz.sh|start-soyuz.sh]], which will fire up all Soyuz services. This will eat lots of RAM. ''(Moving into {{{utilities}}} script per bug Bug:527170; you'll run it ''before'' the other script —jtv)''
 * `mkdir /var/tmp/zeca` ''(Moving into script per bug Bug:527170 —jtv)''
After this, your sample data will include a realistic sampling of Ubuntu releases, and you'll have a user called "ppa-user" with password "test" and using your own email address. You'll be able to log in with your own email address, and sign things for this user with your own GPG key. The user is an Ubuntero and a member of {{{ubuntu-team}}}.
Line 13: Line 13:
 * `make run`
 * Create an account for yourself (`utilities/make-lp-user USERNAME`) ''(Moving into script per bug Bug:527170 —jtv)''
 * Log in with `USERNAME@example.com:test`. ''(To become your own email address per bug Bug:527170 —jtv)''
 * Add your OpenPGP key ''(Moving into script per bug Bug:527170 —jtv)''
  * `gpg --keyserver keyserver.launchpad.dev --send-keys DEADBEEF` (where "DEADBEEF" is your key id). (Note: if you're running Lucid and [[https://bugs.edge.launchpad.net/pygpgme/+bug/452194|bug 452194]] is not fixed, see [[#pygpgme|pygpgme errors]]).
  * Import it to your Launchpad account, as normal. ''(Fixed URL per bug Bug:527170: https://edge.launchpad.net/~ppa-user/+editpgpkeys —jtv)''
  * Get the token from the `logintoken` table, or check the email in `root`'s mailbox, and use it to activate the key (potential gotchas: [[#runmta|no MTA]], [[#gpgcrc|gpg CRC errors]])
   * To use the logintoken, go to launchpad.dev/token/$TOKEN
 * Sign the code of conduct (or [[https://launchpad.dev/codeofconduct/console/+new|simply add one as an admin]]). ''(Moving into script per bug Bug:527170 —jtv)''
 * Create a PPA. ''(Moving into script per bug Bug:527170 —jtv)''
 * `make run` (or if you also want to use codehosting, `make run_codehosting`—some services may fail to start up because you already started them, but it shouldn't be a problem).
 * Open https://launchpad.dev/ in a browser. Log in with your own email adddress and password ''test''.
Line 26: Line 19:
 * Log out. You'll find a pre-made PPA for ppa-user. It has an external dependency on Lucid. If that's not enough, or not what you want:
Line 29: Line 23:
 * Click "Administer". ''(Moving into script per bug Bug:527170 —jtv)''
  * Ensure that the PPA is allowed to build on non-virtual builders.
  * Add an external dependency so you don't have to import the entire Ubuntu archive just for a PPA to work:
 * Click "Administer". ''
 * Edit external dependencies. They normally look like:
Line 36: Line 28:

Setting up a local Soyuz

You're going to run Soyuz in a branch you create for the purpose. To get the whole experience, you'll also be installing the slave-side launchpad-buildd package on your system.

  • Run utilities/start-dev-soyuz.sh to ensure that some Soyuz-related services are running. Some of these may already be running, in which case you'll get some failures that are probably harmless. Note: these services eat lots of memory.

  • Once you've set up your test database, run utilities/soyuz-sampledata-setup.py -e you@example.com (where you@example.com should be an email address you own and have a GPG key for). This prepares more suitable sample data in the launchpad_dev database. If you get a "duplicate key" error, make schema and run again.

After this, your sample data will include a realistic sampling of Ubuntu releases, and you'll have a user called "ppa-user" with password "test" and using your own email address. You'll be able to log in with your own email address, and sign things for this user with your own GPG key. The user is an Ubuntero and a member of ubuntu-team.

Configure an account and PPA

  • make run (or if you also want to use codehosting, make run_codehosting—some services may fail to start up because you already started them, but it shouldn't be a problem).

  • Open https://launchpad.dev/ in a browser. Log in with your own email adddress and password test.

Set up the PPA

You'll find a pre-made PPA for ppa-user. It has an external dependency on Lucid. If that's not enough, or not what you want:

  • Log in as admin@canonical.com:test.

  • Browse to the new PPA.
  • Click "Administer".

  • Edit external dependencies. They normally look like:

deb http://archive.ubuntu.com/ubuntu %(series)s main restricted universe multiverse

Configure a buildd

  • cd lib/canonical/buildd

  • debian/rules package

  • dpkg-buildpackage -b

  • sudo dpkg -i ../launchpad-buildd_50_all.deb

  • sudo apt-get -f install

  • Make it work.
    • Might need to add --umask=022 to twistd args in /etc/init.d/launchpad-buildd, or pkg-create-dbgsym fails obscurely.

    • Edit /etc/launchpad-buildd/default and make sure ntphost points to an existing NTP server. You can check the NTP server pool to find one near you.

  • Get an Ubuntu buildd chroot from Launchpad

wget -O - -q https://edge.launchpad.net/api/devel/ubuntu/lucid/i386/chroot_url | xargs wget

Please note a "builder" and a "chroot" are not the same thing, nor are they "associated" in any way. The logic is as follows:

  • PPAs require builders (a machine where to build packages)
  • The builder may be virtualized (for instance, launchpad.net uses Xen-virtualized builders) or the same machine where Launchpad (meaning "your launchpad.dev") is running, which is what this page details. Using the very same machine where Launchpad.dev is running limits the architectures you can build for: if you are running on i386, you can't build for PowerPC, for instace.
  • A builder needs to setup a bootstrap (what you get with debootstrap) Linux. That bootstrap is what you get in the chroot tarballs mentioned above.
  • When you upload a package and tell Launchpad to process it (see below for this), the builder will pick the chroot it needs (for instance, karmic i386), uncompress it and start building the package.
  • In summary, there are two lists: a list of builders and a list of chroots.

Upload a source to the PPA

  • mkdir /var/tmp/poppy (Moving into script per bug 527170 —jtv)

  • Run scripts/process-upload.py /var/tmp/poppy (creates hierarchy)

  • Add to ~/.dput.cf:

[lpdev]
fqdn = ppa.launchpad.dev:2121
method = ftp
incoming = %(lpdev)s
login = anonymous
  • dput lpdev:~USERNAME/PPANAME/ubuntu some_source.changes (fixing USERNAME per bug 527170 —jtv)

  • scripts/process-upload.py /var/tmp/poppy # Accept the source upload.

  • If this is your first time running soyuz locally, you'll also need to publish ubuntu: scripts/publish-distro.py -C

  • Within five seconds of upload acceptance, the buildd should start building. Wait until it is complete.
  • scripts/process-accepted.py --ppa ubuntu # Create publishings for the binaries.

  • scripts/publish-distro.py --ppa # Publish the source and binaries.

    • Note that private archive builds will not be dispatched until their source is published.

Dealing with the primary archive

  • Add yourself to ~ubuntu-team. (Moving into script per bug 527170 —jtv)

  • dput lpdev:ubuntu some_source.changes

  • scripts/process-upload.py /var/tmp/poppy

  • Watch the output -- the upload might end up in NEW.
    • If it does, go to the queue and accept it.
  • Your builder should now be busy. Once it finishes, the binaries might go into NEW. Accept them if required.
  • scripts/process-accepted.py ubuntu

  • scripts/publish-distro.py

    • The first time, add -C to ensure a full publication of the archive.

Notes

Amd64 chroots

  • (No longer needed per bug 527170 —jtv)

Run the query below (e.g. using "psql -d launchpad_dev") before running make-ubuntu-sane.py in case you want to be able to set up amd64 chroots for build slaves. (Moving into script per bug 527170 —jtv)

INSERT INTO distroarchseries (distroseries, processorfamily, architecturetag, owner, official) VALUES (3, 3, 'amd64', 1, true);

It adds an amd64 DistroArchSeries for hoary from which all the subsequent distro series are cloned by the make-ubuntu-sane.py script.

MTA needed

  • (No longer needed per bug 527170 —jtv)

You need to have some sort of MTA (e.g. postfix) running in order for the email to be placed in the root's mailbox. In cases where you have Launchpad running in a chroot you'll need to stop the "normal" MTA and start it in the chroot or configure the one in the chroot to use a different port.

Gpg CRC errors

  • (No longer needed per bug 527170 —jtv)

Run gpg like this "gpg --ignore-crc-error -d <file>" in case you see the following error message while decrypting the email sent by launchpad:

gpg: malformed CRC
gpg: quoted printable character in armor - probably a buggy MTA has been used

Errors importing key to local zeca

If you are getting an error when importing your key to zeca and you are running Lucid, and bug 452194 is not fixed, you will need to:

  1. Grab the one line patch.,

  2. Apply it (patch -p0 sourcecode/pygpgme/src/pygpgme-context.c < pygpgme.patch)

  3. make compile and then restart your local development zeca as well as the web app.

Soyuz/HowToUseSoyuzLocally (last edited 2022-12-10 08:09:22 by jugmac00)