= Soyuz Package Sets in Karmic = This page exists to track the work required to complete the packagesets implementation. The current implementation will work for karmic, but action is required for karmic+1. == Current blockers == * The existing packagesets need to be linked to a distroseries, since they need to be distroseries-specific * There needs to be a Packageset: header in the archive index for sources to show which packageset it's from * Question: What should happen if the package is not in a packageset? == Points to note == * No need for a web UI (yet), Colin is happy with the webservice tools for now. * The component ACL will act as a fallback for karmic. It will remain the primary ACL for older series. == Potential solutions == The background summary is that packagesets should be tied to a distroseries somewhow, because: upload permissions for packages can change between series, package support can change between series and packages can get dropped entirely. Muharem's longer explanation is below, but my executive summary is that we had three solutions: 1. Add distroseries FK to packageset. * very easily to implement both schema-wise and code-wise * one-off performance hit when a new series is opened to clone the packagesets * could result in unnecessary copies of packagesets 2. copy on write; use a link table to cross-reference distroseries to packagesets. * requires complicated schema changes involving trigger changes * packagesets copied to a distroseries when required * results in minimal packageset growth * possible hidden performance hit when doing write/copy operation to clone the packageset at request time 3. "Poor-man's" copy on write; as above but the cloning is manually done by the person making the request. * Potential confusion for end-users * Still somewhat complicated for us * Removes the hidden performance hit and makes it explicit We favour the second solution, because it reduces the unnecessary duplication of packagesets, which in turn makes them more useful as a first class object in Launchpad. {{{ ---------- Forwarded Message ---------- Subject: associating package sets and distro series -- potential solutions Date: Tuesday 14 July 2009 From: Muharem Hrnjadovic To: Julian Edwards , Celso Providelo , Michael Nelson = Associating package sets and distro series = One of the requirements for associating package sets and distro series is that there is a historical record of what package sets existed in the context of a particular distro series and of their properties (e.g. what source packages they included). In order to satisfy that requirement we could e.g. * give every distro series its own copy of the package set data * share package sets across multiple distro series but have some sort of "copy-on-write" mechanism kick in upon an attempt to modify shared package sets or their associated archive permissions. Some of the consequences of associating package sets and distro series are as follows: * package set names will only be unique within the context of a distro series as opposed to being globally unique * package sets and hence their associated archive permissions will only be applicable within the context of a distro series. * all operations on package sets and archive permissions will need to take a distro series parameter. That parameter could be optional and default to the "current" distro series for the sake of convenience. The following is the list of possible solutions we discussed so far. == Shared nothing == The 'packageset' table will be extended to have a 'distroseries' foreign key (NOT NULL). Package set names will only be unique within the context of a single distro series as opposed to being globally unique. Package sets and the archive permissions defined for them will only be applicable within the context of a single distro series (i.e. one distro series to many package sets). When a new distro series (ds-N+1) is created, copies of all package sets and of all archive permissions that exist in the current distro series (ds-N) will be made (initialise-from-parent?) and associated with ds-N+1. === Advantages === * simplicity * no changes to the complex trigger procedures that maintain the transitive closure of the package set graph/hierarchy * one-off performance hit (at the time of initialisation of a new distro series) === Disadvantages === * potentially many "unnecessary" copies of package set and archive permission table rows * the package set and archive permission tables will grow in size (though fairly moderately?) == Copy on write == Package sets are shared across multiple distro series (i.e. many distro series to many package sets) and the association is effected via a 'distroseries2packageset' link table where every row holds a distro series and a package set key respectively. When a new distro series (ds-N+1) is created, all package sets already associated with the current distro series ds-N will have rows inserted into the 'distroseries2packageset' link table also associating them with ds-N+1. The attempt to modify a shared (associated with 2 or more distro series) package set ps-X or some of its archive permissions in distro series ds-Z would * trigger a clone operation that - creates a copy ps-Y of ps-X and of all its package set and source package inclusion data - disassociates ps-X from ds-Z (removes the (ds-Z, ps-X) row from the 'distroseries2packageset' table) - associates ps-Y with ds-Z (inserts a (ds-Z, ps-Y) row into the link table) - insert copies of all of ps-X's archive permission rows but referring to ps-Y instead - takes place inside a single database transaction for consistency reasons * operate on ps-Y -- the newly created clone of ps-X All modifying IPackageset, IPackagesetSet, IArchivePermission, IArchivePermissionSet interface methods would need to be revised to perform the copy-on-write check. === Advantages === * no unnecessary copies of package set and archive permission table rows * minimum growth of the package set and archive permission tables === Disadvantages === * pretty complex solution * some change to the complex trigger procedures that maintain the transitive closure of the package set graph/hierarchy will be required * clone operations take place at request time potentially resulting in delays that are not explicable to the end user. == Poor man's copy on write == Like the above "Copy on write" solution but no automated package set cloning. Instead we * make a public IPackagesetSet.clone(packageset) method available to the end users * educate the users what operations on package sets and/or archive permissions require the involved package set to be cloned beforehand * leave it to the users to do the right thing === Open questions === * How does the end user figure out whether a package set is shared and hence requires cloning prior to an operation that will modify it? }}}