• Immutable Page
• Info
• Attachments
• More Actions: Raw Text Print View Render as Docbook Delete Cache ------------------------ Check Spelling Like Pages Local Site Map ------------------------ Rename Page Delete Page ------------------------ Subscribe User ------------------------ Remove Spam Revert to this revision Package Pages Sync Pages ------------------------ Load Save SlideShow

Branch Merge Queues

Branch merge queues are a way for us to continue a branch merge proposal through code review and follow it all the way into being merged into the branch, whether by a human, or a robot like Tarmac. Basically, it's PQM with Launchpad integration and without legacy arch support.

As a developer using Launchpad for development,
I want to queue merges to be handled by a bot
so that I can do more interesting work like actually writing code.

Rationale

We are doing this now to reduce the pain the Launchpad team experiences in the current PQM/buildbot story. It will also provide benefits to existing users of tarmac (U1, OpenStack) by clarifying the automated merge story on Launchpad itself.

Stakeholders

• U1 - Ubuntu One uses Tarmac because it's much easier to set up than PQM, and much easier to maintain. They are 90% there, and would like some merge queues to make sure the story is complete. I talk with dobey, who is on the ops+ team on U1, regularly about this.
• Launchpad - Paul spoke with Gary at the Launchpad Epic about what the current issues are with the PQM/buildbot story. There are commonly so many moving parts that it makes it difficult to work with.

Constraints and Requirements

Must

• Support landing to many different branches on a single project (e.g. devel & db-devel with the Launchpad project)

• Allow one-button merging of a branch from a merge proposal page
• Support workflows where a merge must pass some automated process (e.g. a test suite). Note that this process cannot run within the Launchpad data center.
• Allow someone (XXX - don't know who yet - jml) to move queue merges up and down in the queue
• Allow someone (XXX - same person! - jml) to delete items from a queue
• Time estimates for queued merges
• APIs for gathering statistics about landings, e.g.
  • Time
  • How often they fail
• Notify branch owner, merge proposal submitter and merge queue submitter when item is merged / fails to merge
• Queue must be discoverable by non-owners
• Contributors to a project must be able to quickly gain an understanding of how their patches will be merged
• Clearly indicate when merge is done and whether or not it is successful

Nice to have

• Where merges take a long time (e.g. Launchpad), some kind of visual progress indicator
• Simple way to set up own machines for doing compilation / tests and reporting back
• Mail interface
• Command-line client (perhaps Bazaar plugin)
• One-button merging of a branch that does not have a merge proposal

Must not

• Require any explicit configuration before use in projects which simply merge branches without testing/building
• Require refreshing of the queue page to see updates to the queue

Out-of-scope

• Actually running builds / tests for users within the Launchpad data centre

• Any system for storing structured test result data (see LEP/ResultsTracker)

• Providing web interface for inspecting historical queue data

Workflows

• Add to queue
• Configure queue
• Merge succeeds in queue
• Merge fails in queue
• Re-order queue
• Freeze queue
• Unfreeze queue

See Code/MergeQueues for user stories and planning.

Success

How will we know when we are done?

We know we are done when U1 can completely abandon PQM for Tarmac and Merge Queues. As a side, we hope this also means that we've also satisfied all of the issues Launchpad-foundations needs, although their needs may be so specific that we can't put it squarely in Launchpad, and will have to build some external scripts (probably Tarmac plugins)

• Alternatively, perhaps we should simplify our needs until they can fit within Launchpad. -- jml

How will we measure how well we have done?

Paul would like to set up a few graphs to see this:

• Total number of merge queues created
• Number of merge queue items created in a single day
• Number of merge queue items completed in a single day
• Number of branches merged by one queue

jml would like to see:

• Number of projects using merge queues compared to number of projects that actually merge branches
• For projects that use merge queues, how many merges done with queue vs how many done manually
• How many projects outside Canonical actually use merge queues

Thoughts?

• The vast majority of cases are going to be focused on merging to trunk. Let's make sure the UI reflects this.
• Although we have to support many branches to handle Launchpad's case, perhaps we can simplify the UI by only allowing one queue per project.
• For a test running bot, one strategy is to optimistically merge everything, run tests, and then bisect in case of failure. It would be nice if this strategy were supported by our web UIs. jml doesn't know how this would look, though.