• Immutable Page
• Info
• Attachments
• More Actions: Raw Text Print View Render as Docbook Delete Cache ------------------------ Check Spelling Like Pages Local Site Map ------------------------ Rename Page Delete Page ------------------------ Subscribe User ------------------------ Remove Spam Revert to this revision Package Pages Sync Pages ------------------------ Load Save SlideShow

Overview

• Launchpad entry: https://blueprints.edge.launchpad.net/soyuz/+spec/soyuz-delayed-copies

• Created: 2009-06-04 by JulianEdwards

• Contributors: CelsoProvidelo

• Depends on: n/a

Overall Summary

Summary: Provide a way to have asynchronous package copy operations that are decoupled from the requesting operation. This is necessary because most copy operations necessarily take a long time and need to happen outside of a web-app request.

Goal/Deliverables: Decouple package copy requests from the act of performing them; ensure that existing webservice APIs and web UIs continue to work; automatically re-upload private files to the unrestricted librarian as appropriate; provide a way for PPA users to copy packages into Ubuntu provided they have existing upload privileges to it.

We will know we have finished when we have a base infrastructure that will allow us to remove the script that unembargoes packages from the security PPA.

Release Note

Rationale

The current package copying operations are all done either within a single webapp transaction, which frequently times out, or using scripts on the archive master machine. The latter prevents the security team from unembargoing their own packages into Ubuntu.

Use cases

• Kees has an urgent security update to release from his private PPA. He will use the existing API call to IArchive.syncSource() which will queue the request for later processing. The updated package will be copied to Ubuntu and any private files re-uploaded from the restricted to the unrestricted librarian.
• Jamie also has a security update to release. He uses the "copy packages" web interface on his private PPA to push the update into Ubuntu.
• Michael is an archive administrator and wants to copy the "dist-upgrader-all" files from "hardy-proposed" to "hardy-updates". He uses the IArchive.syncSource() API call and also asks for binaries to be copied. All custom files for a source package will also be copied.

Assumptions

User Interface

Implementation

• We will push copies by creating a record in the PackageUpload ACCEPTED queue, and fix process-accepted to cope.

• Some checks will be done more than once; once at the initiation stage and once at the operation stage, but this is fine and can be cleaned up later. (PackageUpload.verifyBeforeAccept() and check_copy())

• p-a will check for any privacy changes between archives and re-upload librarian files as necessary (as unembargo-package does now)
• Custom files can be copied if the callsite wants binaries copied too; we can check for custom files at the same time.

Code Changes

Add IPackageUpload.acceptFromCopy():

• Do not send emails
• close bugs
• grant karma

This is consistent with acceptFromQueue() and acceptFromUploader()

There will be changes in PackageUpload.realiseUpload() for calling override_from_ancestry() and update_files_privacy() for each generated publishing record.

And finally, the do_delayed_copy() function itself, which will have the same signature as do_copy() but instead of returning publishing records will return PackageUpload records.

Schema Changes

• We will remove the NOT NULL constraint from PackageUpload.changesfile, so on copies we won't have to create artificial files.

Migration

Include:

• data migration, if any
• redirects from old URLs to new ones, if any

• how users will be pointed to the new way of doing things, if necessary. (If your change is big enough, consider using the rollout template.)

Unresolved issues

In this section list out any issues which are unresolved and will impact or block the implementation of this spec.