Build from branch into main

Short description of feature

Contact: MartinPool
LEP: https://dev.launchpad.net/LEP/BuildFromBranchIntoMain
On Launchpad: build-from-branch bugs

As an Ubuntu developer
I want Launchpad to build source and binary packages from branches
so that I don't have to both push the branch and also dput a source package

Rationale

At the moment, after making a change, Ubuntu developers must use bzr-builddeb to build a source package, and then dput to upload it. This is unnecessary work. Some developers only upload and don't push the branch.

What value does this give our users?

• It removes one unnecessary step.
• It moves us closer to keeping all code in branches and being able to work with it there.

What costs does this impose on our users?

* If the server-side building of packages does not work reliably, they will be blocked from uploading. If we retain source package uploads at least as a transitional option there will be an escape from this.

• Errors in packaging may be more difficult to see, understand, or debug if they happen on a server rather than locally. However, developers will still have the option to build the package locally.

Why are we doing this now?

What value does this give our users? Which users? Background:

• https://wiki.ubuntu.com/NoMoreSourcePackages

• https://wiki.ubuntu.com/DistributedDevelopment/

Stakeholders

• Ubuntu developers
• Linaro developers
• Ubuntu Technical Board
• James Westby
• Kiko Reis
• Ubuntu technical architect (Allison Randall)
• Lamont Jones

Constraints and Requirements

Must

• Be as secure as the current dput system. There is great potential for damage here.

What MUST the new behaviour provide?

Nice to have

Must not

What MUST it not do?

Out of scope

Subfeatures

Other LaunchpadEnhancementProposals that form a part of this one.

Workflows

Publish a change into Ubuntu

1. User branches from the source package branch.
2. Make changes
3. Commit
4. Push
5. Request publication

Success

How will we know when we are done?

• You can build from a branch directly into the main Ubuntu archive

How will we measure how well we have done?

• Number of packages published in this way.
• Reported satisfaction
• Number of packages willing to proceed to turning off source package uploads

Thoughts?

• For integration testing, it seems like we need a place where Launchpad developers can publish changes into something that looks a lot like the main Ubuntu archive, without actually being that archive. This could be a separate reserved pocket, or it could be a different Launchpad instance
• Whereas recipe builds use bzr-builder, this will probably use bzr-builddeb because no recipe is needed
  • We can have bzr-builder degrade to do the same thing as bzr-builddeb.
• Should packages build and publish on every commit, or should there be a separate request?
• Eventually it may be useful to have a per-package, per-distrorelease option to turn off source package uploads for areas we have decided branches are the best option. That isn't needed in the first iteration.
• The things missing from bzr-builder would be:
  • Control over the changelog message used
  • Better control over the resulting version number
  • GPG signatures required for something to end up in Ubuntu (API request isn't GPG signed)
  • Ability to produce non-native tarballs

Changes needed

• Installer bzr-builder into buildd vms. (May already be done.)
• If we're not going to build on every commit, then there must be a mechansm to request a build. (Probably a web service api, and a bzr command that calls it.)
• New type of schedule-able job to request a source package be built from a branch. There is already a db enum for it, but we need to add a new build farm job type.

• The uploader (or something else?) should sign packages built by the buildds (with which key?). (bug 579870)

• After that, the binary can be built and published by the existing mechanism, so probably no other changes are needed.
• As a follow-on, add a mechanism to turn off/refuse dput uploads.