Diff for "Soyuz/HowToUseSoyuzLocally"

Not logged in - Log In / Register

Differences between revisions 22 and 26 (spanning 4 versions)
Revision 22 as of 2010-01-10 21:57:47
Size: 6391
Editor: abentley
Comment:
Revision 26 as of 2010-02-25 16:51:25
Size: 6944
Editor: jtv
Comment:
Deletions are marked like this. Additions are marked like this.
Line 3: Line 3:
 * Grab and run [[attachment:make-ubuntu-sane.py|make-ubuntu-sane.py]], to set up some Ubuntu series that still exist, along with some state such as permissions ([[#amd64chroots|Please see note on amd64 chroots]]).
 * Grab and run [[attachment:start-soyuz.sh|start-soyuz.sh]], which will fire up all Soyuz services. This will eat lots of RAM.
 * `mkdir /var/tmp/zeca`
You're going to run Soyuz in a branch you create for the purpose. To get the whole experience, you'll also be installing the slave-side {{{launchpad-buildd}}} package on your system.

/!\ These scripts are currently broken, and are being replaced with ones that live in the Launchpad tree. The new scripts will take away a lot of the manual work. —jtv

* Once you've set up your test database, grab and run [[attachment:make-ubuntu-sane.py|make-ubuntu-sane.py]] to set up some Ubuntu series that still exist, along with some state such as permissions ([[#amd64chroots|Please see note on amd64 chroots]]). ''(Moving into {{{utilities}}} script per bug Bug:527170 —jtv)''
 * Grab and run [[attachment:start-soyuz.sh|start-soyuz.sh]], which will fire up all Soyuz services. This will eat lots of RAM. ''(Moving into {{{utilities}}} script per bug Bug:5264620 —jtv)''
 * `mkdir /var/tmp/zeca` ''(Moving into script per bug Bug:527170 —jtv)''
Line 10: Line 14:
 * Create an account for yourself (`utilities/make-lp-user USERNAME`)
 * Log in with `USERNAME@example.com:test`.
 * Create an account for yourself (`utilities/make-lp-user USERNAME`) ''(Moving into script per bug Bug:527170 —jtv)''
 * Log in with `USERNAME@example.com:test`.  ''(To become a fixed username {{{ppa-user}}} per bug Bug:527170 —jtv)''
Line 17: Line 21:
 * Sign the code of conduct (or [[https://launchpad.dev/codeofconduct/console/+new|simply add one as an admin]]).  * Sign the code of conduct (or [[https://launchpad.dev/codeofconduct/console/+new|simply add one as an admin]]). ''(Moving into script per bug Bug:526462 —jtv)''
Line 36: Line 40:
 * `debian/rules package`
Line 40: Line 45:
  * Add `$SUDO cp /etc/resolv.conf /etc/hosts "$HOME/build-$BUILDID/chroot-autobuild/etc"` to the end of `/usr/share/launchpad-buildd/slavebin/mount-chroot`, so name resolution works.
Line 42: Line 46:
  * (>= Karmic) Might need to alter "`dpkg-source: extracting`" output grepping in sbuild to "`dpkg-source: info: extracting`"
Line 62: Line 65:
 * mkdir `/var/tmp/poppy`
Line 97: Line 101:
Run the query below (e.g. using "`psql -d launchpad_dev`") '''before''' running `make-ubuntu-sane.py` in case you want to be able to set up `amd64` chroots for build slaves. Run the query below (e.g. using "`psql -d launchpad_dev`") '''before''' running `make-ubuntu-sane.py` in case you want to be able to set up `amd64` chroots for build slaves. ''(Moving into script per bug Bug:527170 —jtv)''

Initial setup

You're going to run Soyuz in a branch you create for the purpose. To get the whole experience, you'll also be installing the slave-side launchpad-buildd package on your system.

/!\ These scripts are currently broken, and are being replaced with ones that live in the Launchpad tree. The new scripts will take away a lot of the manual work. —jtv

  • Once you've set up your test database, grab and run make-ubuntu-sane.py to set up some Ubuntu series that still exist, along with some state such as permissions (Please see note on amd64 chroots). (Moving into utilities script per bug 527170 —jtv)

  • Grab and run start-soyuz.sh, which will fire up all Soyuz services. This will eat lots of RAM. (Moving into utilities script per bug 5264620 —jtv)

  • mkdir /var/tmp/zeca (Moving into script per bug 527170 —jtv)

Configure an account and PPA

  • make run

  • Create an account for yourself (utilities/make-lp-user USERNAME) (Moving into script per bug 527170 —jtv)

  • Log in with USERNAME@example.com:test. (To become a fixed username ppa-user per bug 527170 —jtv)

  • Add your OpenPGP key
    • gpg --keyserver keyserver.launchpad.dev --send-keys DEADBEEF

    • Add it to your Launchpad account, as normal.
    • Get the token from the logintoken table, or check the email in root's mailbox, and use it to activate the key (potential gotchas: no MTA, gpg CRC errors)

      • To use the logintoken, go to launchpad.dev/token/$TOKEN
  • Sign the code of conduct (or simply add one as an admin). (Moving into script per bug 526462 —jtv)

  • Create a PPA.

Set up the PPA

  • Log out.
  • Log in as admin@canonical.com:test.

  • Browse to the new PPA.
  • Click "Administer".
    • Ensure that the PPA is allowed to build on non-virtual builders.
    • Add an external dependency so you don't have to import the entire Ubuntu archive just for a PPA to work:

deb http://archive.ubuntu.com/ubuntu %(series)s main restricted universe multiverse

Configure a buildd

Please note a "builder" and a "chroot" are not the same thing, nor are they "associated" in any way. The logic is as follows:

  • PPAs require builders (a machine where to build packages)
  • The builder may be virtualized (for instance, launchpad.net uses Xen-virtualized builders) or the same machine where Launchpad (meaning "your launchpad.dev") is running, which is what this page details. Using the very same machine where Launchpad.dev is running limits the architectures you can build for: if you are running on i386, you can't build for PowerPC, for instace.
  • A builder needs to setup a bootstrap (what you get with debootstrap) Linux. That bootstrap is what you get in the chroot tarballs mentioned above.
  • When you upload a package and tell Launchpad to process it (see below for this), the builder will pick the chroot it needs (for instance, karmic i386), uncompress it and start building the package.
  • In summary, there are two lists: a list of builders and a list of chroots.

Upload a source to the PPA

  • mkdir /var/tmp/poppy

  • Run scripts/process-upload.py /var/tmp/poppy (creates hierarchy)

  • Add to ~/.dput.cf:

[lpdev]
fqdn = ppa.launchpad.dev:2121
method = ftp
incoming = %(lpdev)s
login = anonymous
  • dput lpdev:~USERNAME/PPANAME/ubuntu some_source.changes

  • scripts/process-upload.py /var/tmp/poppy # Accept the source upload.

  • Within five seconds of upload acceptance, the buildd should start building. Wait until it is complete.
  • scripts/process-accepted.py --ppa ubuntu # Create publishings for the binaries.

  • scripts/publish-distro.py --ppa # Publish the source and binaries.

    • Note that private archive builds will not be dispatched until their source is published.

Dealing with the primary archive

  • Add yourself to ~ubuntu-team.

  • dput lpdev:ubuntu some_source.changes

  • scripts/process-upload.py /var/tmp/poppy

  • Watch the output -- the upload might end up in NEW.
    • If it does, go to the queue and accept it.
  • Your builder should now be busy. Once it finishes, the binaries might go into NEW. Accept them if required.
  • scripts/process-accepted.py ubuntu

  • scripts/publish-distro.py

    • The first time, add -C to ensure a full publication of the archive.

Notes

Amd64 chroots

Run the query below (e.g. using "psql -d launchpad_dev") before running make-ubuntu-sane.py in case you want to be able to set up amd64 chroots for build slaves. (Moving into script per bug 527170 —jtv)

INSERT INTO distroarchseries (distroseries, processorfamily, architecturetag, owner, official) VALUES (3, 3, 'amd64', 1, true);

It adds an amd64 DistroArchSeries for hoary from which all the subsequent distro series are cloned by the make-ubuntu-sane.py script.

MTA needed

You need to have some sort of MTA (e.g. postfix) running in order for the email to be placed in the root's mailbox. In cases where you have Launchpad running in a chroot you'll need to stop the "normal" MTA and start it in the chroot or configure the one in the chroot to use a different port.

Gpg CRC errors

Run gpg like this "gpg --ignore-crc-error -d <file>" in case you see the following error message while decrypting the email sent by launchpad:

gpg: malformed CRC
gpg: quoted printable character in armor - probably a buggy MTA has been used

Soyuz/HowToUseSoyuzLocally (last edited 2022-12-10 08:09:22 by jugmac00)