• Immutable Page
• Info
• Attachments
• More Actions: Raw Text Print View Render as Docbook Delete Cache ------------------------ Check Spelling Like Pages Local Site Map ------------------------ Rename Page Delete Page ------------------------ Subscribe User ------------------------ Remove Spam Revert to this revision Package Pages Sync Pages ------------------------ Load Save SlideShow

Overview

• Launchpad entry: https://blueprints.edge.launchpad.net/soyuz/+spec/archive-derivatives

• Created: 2008-07-15 by JulianEdwards

• Contributors: CelsoProvidelo, MuharemHrnjadovic

• Depends on: n/a (although SoyuzDisklessArchives would be beneficial)

Overall Summary

Summary: Introduce the concept of a generalised "copy archive". The copy archive will never have direct uploads but will instead have packages copied to it from other archives. The copied packages can originate at any archive in Soyuz; e.g. another PPA, a main distribution archive or even another copy archive.

Copy archives

• may have packages copied to them multiple times, each time from a different source archive if desired.
• will be used for point releases, snapshots, rebuilds and any other reason the user can think of.
• are always associated with a particular distro series.
• may depend on another archive for the purpose of satisfying build dependencies.

Package copy operations to destination copy archives:

• should not copy packages repeatedly if these exist in the destination archive already (merge copy)

• that do not include binaries are tantamount to rebuild requests and result in the

  • creation of appropriate build records for the source packages copied
  • update of the copy archive's rebuild status

Goal/Deliverables: Soyuz will have the ability for people to create a new archive, give it a name, and populate it with packages desired. The archive will be tied to a particular distro series.

We will know we have finished when we can create generalised ad-hoc archives, initialise them with a list of packages copied from other archives, and publish them.

Release Note

This section should include a paragraph describing the end-user impact of this change. It is meant to be included in the first part of our release change log.

It is mandatory.

Rationale

The ability to have generalised ad-hoc archives allows us to support new workflows by the Distro team, such as point releases and snapshots.

Use cases

• Harry wishes to create a point release for the Ubuntu Hardy distro series. He goes to a page in Launchpad, assigns a name for the new point release, gives a reason, says to copy all packages from Hardy, decides whether to have it published or not, and Launchpad creates a new archive.
• Hermione wishes to rebuild a set of packages in the current development distroseries. She goes to a page in Launchpad, selects the components she wants to rebuild and requests that only the sources are copied to the new archive. Builds are created for the copied sources in the new archive and it will start rebuilding. Build progress is shown in the Launchpad UI.
• Dumbledore is working on an embedded distribution and wants a stable snapshot of Ubuntu Hardy to use for this. He creates a new archive via a page in Launchpad, copies packages from Hardy and asks for the new archive to be published so that he can see and build against the packages.
• Hagrid has already created a point release for Ubuntu Hardy LTS and wants to make a new point release that includes the newly-released KDE4. He creates a new archive and specifies that it is initialised from the existing point release archive. He then copies in the KDE4 packages from the KDE4 PPA. Please note that packages that exist in the destination archive already will not be copied again (merge-copy behaviour).
• A few weeks later Hagrid checks to see which packages have been updated in the KDE PPA. He is also interested in packages that are in the PPA but not in his copy archive. He wants the latter copied to his archive as well.
• Phineas populates a copy archive with source packages from hardy main and hardy restricted and a rebuild for these packages starts. Phineas decides to cancel the entire rebuild.
• Sirius later realizes that the rebuild of the hardy/restricted packages is still needed after all and resumes the rebuild for these.
• Ravenclaw populated a copy archive with source packages for a rebuild but then realized that he does not need any of the packages in the UPDATES pocket and cancels the rebuild for these.
• Narcissa comes to realize that the security pocket package rebuilds in her copy archive failed due to missing archive dependencies. She repairs these and requests a mass retry of the failed builds in the security pocket.
• Regulus is not sure he will be using his copy archive for a certain while and "pushes the big red button" and disables the archive with the effect that
  • no uploads to the archive are accepted
  • no builds are dispatched to the build farm

Assumptions

• We don't need a copy history

User Interface

Initial UI implementation

The page at /ubuntu/+archives will by default list all the non-main (i.e. not PRIMARY, PARTNER or PPA) archives, with the following information:

Each archive title will be click-able, with the address /ubuntu/+archive/<archive-name>. This archive page will contain:

• a pie chart + figures for fail/succeed/pending build counters

• a summary of all PackageCopyRequests that have been used to create the archive

It will not contain:

• copy/delete links
• source list information like on a PPA page

Further UI implementation

• UI to create a copy archive with form fields:
  • Publish (yes/no)
  • Virtualised (yes/no)
  • Archive name
  • Reason text
  • distro series
  • optional: Architectures list
  • optional: build dependency to archive X
  The UI will create the new Archive
• GUI/script to request a package copy operation to destination copy archive:
  • source archive
  • destination archive
  • Binaries (yes/no)
  • Packages to copy:
    • All packages
    • List of mappings: source component(/pockets?) to destination
      • component(/pocket?).
    • Explicit list of package names (file upload)

  The UI will create a new PackageCopyRequest

• GUI to display individual package copy operations and their status/progress
• GUI to cancel individual package copy operations
• GUI to cancel/resume/retry rebuild operations on archive level
• Script to perform package copy operations

Implementation

• Get rid of the ArchiveRebuild table and add/adjust a few columns in the archive table:

  • add optional DistroSeries foreign key (NOT NULL if the archive purpose is "copy archive" (constraint))

  • add date created
  • reuse description column to capture copy archive creation reason.

• Add new table ArchiveRequest that captures the status and the specific parameters of the following archive related request types:

  • package copy with the option to
    • include binary packages xor
    • copy source packages only, followed by a rebuild
  • cancel builds for a specified archive/distroseries/component/pocket
  • resume builds for a specified archive/distroseries/component/pocket
  • retry failed builds for a specified archive/distroseries/component/pocket
• UI work as above

Estimations:

• 1 day for db schema
• 5 days UI (a lot of it is already done)
• 5 days for the script

Code Changes

• Rename the ArchivePurpose "REBUILD" to "COPY"

• Create an enum for the status of an ArchiveRequest so it has the values:

  • NEW
  • INPROGRESS
  • FAILED
  • SUCCEEDED
  • CANCELLED

• Create an enum for the type of an ArchiveRequest so it has the values:

  • COPY
  • BUILD_CANCEL
  • BUILD_RESUME
  • BUILD_RETRY

Schema Changes

Migration

Include:

• data migration, if any
• redirects from old URLs to new ones, if any

• how users will be pointed to the new way of doing things, if necessary. (If your change is big enough, consider using the rollout template.)

Unresolved issues

In this section list out any issues which are unresolved and will impact or block the implementation of this spec.

CategoryProposal