Derivative Distributions

As an OEM provider
I want to make a customized version of Ubuntu
so that I can ship the OS best suited to my hardware

As a Linaro partner
I want to create a new distribution/archive in the hosting software in 3 or 4 clicks
so that this does not require deploying a new instance for each distribution.

As a Linaro partner
I want this new distribution visible only to a certain set of people, and only a subset of that group can upload
so that it's easy to see who is in each of those groups on a single page, at most 1 click from the overview page of the distribution. It is also simple to modify each of the groups, with at most 3 clicks required to add or remove someone from one of the groups.

As Meg who works for a partner enabling pre-release hardware
I want to upload modified packages to an archive for a hardware project
so that others who are working on the project can then build on top of this work.

As Lou, the lead developer on the project
I want to go to a web page and see the current state of the archive
so that I am able to remove some obsolete packages. This can be done from a single page, and is at most three clicks to remove a package. Bulk operation should be possible.

Rationale

We are doing this so it's easier for Linaro partners and Canonical's OEM team to build custom distributions.

This brings Launchpad's famous usability and feature set to more people who want to use it but currently have to use 3rd party products to do this.

Stakeholders

• Linaro
• OEM Services

Constraints

• Ability to sync packages from the parent series, some auto, some manual, optionally rebuilding them
  • merge-o-matic type functionality, see merges.ubuntu.com
• Allow one distribution to have many derived distributions
  • These derived distributions must be discoverable from the "parent" distribution.
• Track state of packages compared those in the parent's archive:
  • Report of differences against parent(s)
    • what might I want? what do I need to push upstream?
  • Action on reports: syncing packages in both directions
  • Sync all packages changed in parent but not changed in child
    • User-defined periodic auto-sync of explicitly listed packages from parent
• Package branches for derived archives/distros
  • XXX - do we maintain official package branches in Launchpad or are they the responsibility of the derived distro "owner"? - jml
• Create a new distribution
  • Only XXX admins (commercial admins right now but this should change) can make new distributions, however the "driver" of the new distribution will be allowed to create/derive new series
• Create new derived distro series based on packagesets
  • Restricted to distro drivers
  • Required because people don't want to import all of parent (think about build time)
• Support different release cycles and release models from the parent distro
  • OEM case is frequently a small, finite number of rolling releases

Nice to have

• new packagesets for derived distros based on parent's packagesets
• diff of packagesets
• Arbitrarily-named suites

  • XXX - how are these created? who has permission to do so? how do they interact with bug affects & package branches? - jml

• Configurable constraints like:
  • preventing someone from uploading a package with the same version as a package in a parent distribution (binaries will have conflicting checksums)
  • uploading a package with an incorrect version (e.g. a version that will most likely be higher then the next upload in a parent distribution; ie. version scheme constraints).
  • XXX - "like"? Please list the configurable constraints we're providing.
  • XXX - What are the motivations for these features? - jml

Workflows

• Linaro partner creates a new distribution and derives a new series from an Ubuntu series.

• After a couple of weeks of hacking on a derived distribution, Meg syncs all the changes from Ubuntu into her derived series.

• Lou proposes a branch from his derived distribution to be merged into the upstream distribution

• Kiko looks at the Linaro distribution (derived from Ubuntu), gets a visual sense of the size of the delta between Ubuntu & Linaro. He investigates further, and sees that there are only one or two packages that are significantly different. He requests a merge of the Linaro packages to the upstream.

• Meg figures out that a bug in her derived distribution is actually a bug in the parent distribution. She files a bug and adds a bugtask for the upstream distribution package and a bugtask for her own package.

• Bryce is brought in to consult on an X problem in Linaro. He looks at the Linaro distribution and is able to quickly see the differences between Ubuntu & Linaro for X packageset packages only.

Success

How will we know when we are done?

From https://wiki.ubuntu.com/Specs/M/ARMArchiveBranching: There will be a number of milestone test cases that we have:

• Having basic archive management working and able to accept uploads, build packages and publish them to an archive.
• Ability to branch a subset of Ubuntu in to the new archive management software.
• Ability to view differences between Ubuntu and a derivative using the web interface.
• Ability to do the same via the API.
• Ability to trigger syncs using the web interface.
• Ability to do the same via the API.

How will we measure how well we have done?

Mockups

Creating a new derived distroseries

Questions

• Should we use checkboxes? I personally think that we should consider checkboxes for the architectures... even if there are say 15 different architectures, they can be adequately grouped using fieldsets (eg. one for all arm arches) where appropriate. This would enable people to know what they are selecting from at a glance. My main concerns with a js picker (like the tags one) is that (I imagine) users need to know some letters of the arch to search for. -- michael.nelson 2010-08-17 07:08:27

New portlet for the distroseries page

Showing packages differences

Derived series differences

Questions for Derived series differences

• Should we enable per-item comments? If so, I think we'll need to either (1) restrict this enhancement to JS only (fine - it is a progressive enhancement), or (2) update the UI so that each row in the list can be viewed as a page on its own (ie. you click on something in the row to display that row on its own page where it can be edited etc.). If (1) is OK, then I'd also be keen to know if commenting on a specific row is something you'd only do when you wanted to ignore it.

• How can we present/enable ignoring of certain rows? If the results on this page are batched (as displayed in the mockup), then we need to show everything in the batch (ie. we can't display 1 -> 20 of 56, and then hide 5 of those 20 so there's only 15 displayed). This means that the filter itself ("Show packages containing") will need to be updated to include a checkbox 'include ignored items' or similar. So when you add a comment about why you're not syncing it, this automatically ensures that it is marked as 'ignored' and can't be selected for syncing etc., but is still visible when the 'include ignored' filter is selected) in a disabled/unselectable state (and has an option to re-enable - better terminology required). We could also have a link for "There are currently [link]5 items being ignored[/link]" -- michael.nelson 2010-08-17 07:27:40

Derived series missing packages

Derived series unique packages

User testing

Round 1

Took place as remote audio calls on July 28th and August 2nd 2010, with:

• James Westby
• Mathias Gug
• Didier Roche.

Transcripts and audio
Recommendations

Thoughts?

• Check over the Ubuntu owner / bug supervisor etc model and see if it's relevant to non-standard projects
• Look up past literature on this subject
• What Ubuntu-specific scripts are we running against Launchpad?
  • XXX - how would you find out?
• Derivative distributions should be first class distributions.
• Configurable, entirely flexible, powerful policies about behaviors are key.

I see three main "features" here:

1. Creating distributions that can be uploaded to that will then build & publish packages.

2. Synchronizing distributions and the associated UI & new controls.

3. Creating derived distribution series.

There's also a bunch of associated stuff that must be considered:

• merging between branches that are associated via a derived distribution link
• searching for bugs etc. upstream, downstream and in "sibling" distributions

Old mockups

Creating a new derived distroseries:

New portlet for the distroseries page:

Showing packages differences: