• Immutable Page
• Info
• Attachments
• More Actions: Raw Text Print View Render as Docbook Delete Cache ------------------------ Check Spelling Like Pages Local Site Map ------------------------ Rename Page Delete Page ------------------------ Subscribe User ------------------------ Remove Spam Revert to this revision Package Pages Sync Pages ------------------------ Load Save SlideShow

Soyuz Package Sets in Karmic

This page exists to track the work required to complete the packagesets implementation. The current implementation will work for karmic, but action is required for karmic+1.

Current blockers

• The existing packagesets need to be linked to a distroseries, since they need to be distroseries-specific
• There needs to be a Packageset: header in the archive index for sources to show which packageset it's from
  • Question: What should happen if the package is not in a packageset?

Points to note

• No need for a web UI (yet), Colin is happy with the webservice tools for now.
• The component ACL will act as a fallback for karmic. It will remain the primary ACL for older series.

Potential solutions

The background summary is that packagesets should be tied to a distroseries somewhow, because: upload permissions for packages can change between series, package support can change between series and packages can get dropped entirely.

Muharem's longer explanation is below, but my executive summary is that we had three solutions:

1. Add distroseries FK to packageset.
   • very easily to implement both schema-wise and code-wise
   • one-off performance hit when a new series is opened to clone the

packagesets

• could result in unnecessary copies of packagesets

1. copy on write; use a link table to cross-reference distroseries to

packagesets.

• requires complicated schema changes involving trigger changes
• packagesets copied to a distroseries when required
• results in minimal packageset growth
• possible hidden performance hit when doing write/copy operation to clone

the packageset at request time

1. "Poor-man's" copy on write; as above but the cloning is manually done by

the person making the request.

• Potential confusion for end-users
• Still somewhat complicated for us
• Removes the hidden performance hit and makes it explicit

We favour the second solution, because it reduces the unnecessary duplication of packagesets, which in turn makes them more useful as a first class object in Launchpad.

----------  Forwarded Message  ----------

Subject: associating package sets and distro series -- potential solutions
Date: Tuesday 14 July 2009
From: Muharem Hrnjadovic <muharem@canonical.com>
To: Julian Edwards <julian.edwards@canonical.com>, Celso Providelo 
<celso.providelo@gmail.com>, Michael Nelson <michael.nelson@canonical.com>

= Associating package sets and distro series =

One of the requirements for associating package sets and distro series
is that there is a historical record of what package sets existed in
the context of a particular distro series and of their properties
(e.g. what source packages they included).

In order to satisfy that requirement we could e.g.

  * give every distro series its own copy of the package set data
  * share package sets across multiple distro series but have some
    sort of "copy-on-write" mechanism kick in upon an attempt to
    modify shared package sets or their associated archive
    permissions.

Some of the consequences of associating package sets and distro series
are as follows:

  * package set names will only be unique within the context of a
    distro series as opposed to being globally unique
  * package sets and hence their associated archive permissions will
    only be applicable within the context of a distro series.
  * all operations on package sets and archive permissions will need
    to take a distro series parameter. That parameter could be
    optional and default to the "current" distro series for the sake
    of convenience.

The following is the list of possible solutions we discussed so far.


== Shared nothing ==

The 'packageset' table will be extended to have a 'distroseries'
foreign key (NOT NULL). Package set names will only be unique within
the context of a single distro series as opposed to being globally
unique.

Package sets and the archive permissions defined for them will only be
applicable within the context of a single distro series (i.e. one
distro series to many package sets).

When a new distro series (ds-N+1) is created, copies of all package
sets and of all archive permissions that exist in the current distro
series (ds-N) will be made (initialise-from-parent?) and associated
with ds-N+1.

=== Advantages ===

  * simplicity
  * no changes to the complex trigger procedures that maintain the
    transitive closure of the package set graph/hierarchy
  * one-off performance hit (at the time of initialisation of a new
    distro series)

=== Disadvantages ===

  * potentially many "unnecessary" copies of package set and archive
    permission table rows
  * the package set and archive permission tables will grow in size
    (though fairly moderately?)

== Copy on write ==

Package sets are shared across multiple distro series (i.e. many
distro series to many package sets) and the association is effected
via a 'distroseries2packageset' link table where every row holds a
distro series and a package set key respectively.

When a new distro series (ds-N+1) is created, all package sets already
associated with the current distro series ds-N will have rows inserted
into the 'distroseries2packageset' link table also associating them
with ds-N+1.

The attempt to modify a shared (associated with 2 or more distro
series) package set ps-X or some of its archive permissions in distro
series ds-Z would

  * trigger a clone operation that
    - creates a copy ps-Y of ps-X and of all its package set and
      source package inclusion data
    - disassociates ps-X from ds-Z (removes the (ds-Z, ps-X) row from
      the 'distroseries2packageset' table)
    - associates ps-Y with ds-Z (inserts a (ds-Z, ps-Y) row into the
      link table)
    - insert copies of all of ps-X's archive permission rows but
      referring to ps-Y instead
    - takes place inside a single database transaction for consistency
      reasons
  * operate on ps-Y -- the newly created clone of ps-X

All modifying IPackageset, IPackagesetSet, IArchivePermission,
IArchivePermissionSet interface methods would need to be revised to
perform the copy-on-write check.

=== Advantages ===

  * no unnecessary copies of package set and archive permission table
    rows
  * minimum growth of the package set and archive permission tables

=== Disadvantages ===

  * pretty complex solution
  * some change to the complex trigger procedures that maintain the
    transitive closure of the package set graph/hierarchy will be
    required
  * clone operations take place at request time potentially resulting
    in delays that are not explicable to the end user.


== Poor man's copy on write ==

Like the above "Copy on write" solution but no automated package set
cloning.

Instead we

  * make a public IPackagesetSet.clone(packageset) method available to
    the end users
  * educate the users what operations on package sets and/or archive
    permissions require the involved package set to be cloned
    beforehand
  * leave it to the users to do the right thing

=== Open questions ===

  * How does the end user figure out whether a package set is shared
    and hence requires cloning prior to an operation that will modify
    it?