Derivative Distributions

As an OEM provider
I want to make a customized version of Ubuntu
so that I can ship the OS best suited to my hardware

As a Linaro partner
I want to create a new distribution/archive in the hosting software in 3 or 4 clicks
so that this does not require deploying a new instance for each distribution.

As a Linaro partner
I want this new distribution visible only to a certain set of people, and only a subset of that group can upload
so that it's easy to see who is in each of those groups on a single page, at most 1 click from the overview page of the distribution. It is also simple to modify each of the groups, with at most 3 clicks required to add or remove someone from one of the groups.

As Meg who works for a partner enabling pre-release hardware
I want to upload modified packages to an archive for a hardware project
so that others who are working on the project can then build on top of this work.

As Lou, the lead developer on the project
I want to go to a web page and see the current state of the archive
so that I am able to remove some obsolete packages. This can be done from a single page, and is at most three clicks to remove a package. Bulk operation should be possible.

Rationale

We are doing this so it's easier for Linaro partners and Canonical's OEM team to build custom distributions.

This brings Launchpad's famous usability and feature set to more people who want to use it but currently have to use 3rd party products to do this.

Stakeholders

• Linaro
• OEM Services

Constraints

• Ability to sync packages from the parent series, some auto, some manual, optionally rebuilding them
  • merge-o-matic type functionality, see merges.ubuntu.com
• Allow one distribution to have many derived distributions
  • These derived distributions must be discoverable from the "parent" distribution.
• Track state of packages compared those in the parent's archive:
  • Report of differences against parent(s)
    • what might I want? what do I need to push upstream?
  • Action on reports: syncing packages in both directions
  • Sync all packages changed in parent but not changed in child
    • cron job which we can enable/disable in UI, or via an API call

      • XXX - "Cron job" is definitely not a requirement -- jml

• Package branches for derived archives/distros
  • XXX - do we maintain official package branches in Launchpad or are they the responsibility of the derived distro "owner"? - jml
• Create new derived distro series based on packagesets
  • Only admins can make new distributions, however the "driver" of the new distribution will be allowed to create/derive new series
    • XXX - Why only admins? - jml
  • XXX - Why do people want to do this?
• Arbitrarily-named suites

  • XXX - how are these created? who has permission to do so? how do they interact with bug affects & package branches? - jml

• Support different release cycles and release models from the parent distro
  • OEM case is frequently a small, finite number of rolling releases
• Configurable constraints like:
  • preventing someone from uploading a package with the same version as a package in a parent distribution (binaries will have conflicting checksums)
  • uploading a package with an incorrect version (e.g. a version that will most likely be higher then the next upload in a parent distribution; ie. version scheme constraints).
  • XXX - "like"? Please list the configurable constraints we're providing.
  • XXX - What are the motivations for these features? - jml

Nice to have

• new packagesets for derived distros based on parent's packagesets
• diff of packagesets

Workflows

• Linaro partner creates a new distribution based on Ubuntu.

• After a couple of weeks of hacking on a derived distribution, Meg syncs all the changes from Ubuntu into her derived distribution.

• Lou proposes a branch from his derived distribution to be merged into the upstream distribution

• Kiko looks at the Linaro distribution (derived from Ubuntu), gets a visual sense of the size of the delta between Ubuntu & Linaro. He investigates further, and sees that there are only one or two packages that are significantly different. He requests a merge of the Linaro packages downstream.

• Meg figures out that a bug in her derived distribution is actually a bug in the parent distribution. She forwards the bug upstream.

• Bryce is brought in to consult on an X problem in Linaro. He looks at the Linaro distribution and is able to quickly see the differences between Ubuntu & Linaro for X packages only.

Success

How will we know when we are done?

From https://wiki.ubuntu.com/Specs/M/ARMArchiveBranching: There will be a number of milestone testcases that we have:

• Having basic archive management working and able to accept uploads.
  • XXX - source package uploads? - jml. If so, "accept" means build and publish, right?
• Ability to branch a subset of Ubuntu in to the new archive management software.
• Ability to view differences between Ubuntu and a derivative using the web interface.
• Ability to do the same via the API.
• Ability to trigger syncs using the web interface.
• Ability to do the same via the API.

How will we measure how well we have done?

Mockups

Creating a new derived distroseries:

New portlet for the distroseries page:

Showing packages differences:

Thoughts?

• Check over the Ubuntu owner / bug supervisor etc model and see if it's relevant to non-standard projects
• Look up past literature on this subject
• What Ubuntu-specific scripts are we running against Launchpad?
  • XXX - how would you find out?
• Derivative distributions should be first class distributions.
• Configurable, entirely flexible, powerful policies about behaviors are key.
• XXX - What about conflict resolution when syncing?

I see three main "features" here:

1. Creating distributions that can be uploaded to that will then build & publish packages.

2. Synchronizing distributions and the associated UI & new controls.

3. Creating derived distribution series.

There's also a bunch of associated stuff that must be considered:

• merging between branches that are associated via a derived distribution link
• searching for bugs etc. upstream, downstream and in "sibling" distributions