launchpad development
Overview
Launchpad entry: https://blueprints.edge.launchpad.net/soyuz/+spec/distribution-scripts
See Also: bug 55798
Created: [2008-07-18] by CelsoProvidelo
Contributors: JulianEdwards, MuharemHrnjadovic
Depends on: n/a
Overall Summary
Summary: Convert the existing cron.publish-ftpmaster shell script into a Python LaunchpadCronscript. Errors will be reported to the scriptactivity table, and distro-specific hook scripts can be run at certain points in the publishing pipeline.
Goal/Deliverables: A new Python script to control distribution publishing, which will replace cron.publish-ftpmaster.
We will know we have finished when the new script can be used.
Release Note
This section should include a paragraph describing the end-user impact of this change. It is meant to be included in the first part of our release change log.
It is mandatory.
Rationale
- We need better error reporting and notification when publishing fails.
- The distribution maintainers need to be able to run their own scripts during the publishing process, rather than asking LP developers to amend code that is in the LP codebase.
Use cases
- Neo notices that a package that was built 3 hours ago has not been published in the Ubuntu archive. The publishing script should notify the Launchpad Operation System Administrators that there was a problem during publishing.
- The archive admin Agent Smith wishes to change way Ubuntu seeds are processed. The new publishing script allows archive admins to run their own scripts at different parts of the publishing pipeline so he simply amends a hook script that runs during post-publication time.
Assumptions
User Interface
n/a
Implementation
Write a new Python script that is a standard LaunchpadCronscript. It will report its status to the scriptactivity table, thus notify the LOSAs of problems.
The script will use run-parts to execute an arbitrary number of executable files in a pre-defined directory at certain points in the pipeline. The publication part of the script will run twice, with a "high" priority pass first and a "low" priority pass second. The high priority pass is intended for security publications that need to be pushed out quickly.
- Run process-accepted
- Get the list of dirty pockets.
- If there are dirty security pockets, run the "fast track" publication process.
- Run the normal publication process.
Publication part:
- rsync the temporary dists tree from the current one
- Run publish-distro with the required suites
- run-parts "pre-publication", with the priority as arg[0]
- Install the new dists tree
- run-parts "post-publication", with the priority as arg[0]
"priority" will be an integer where 0 is high and 1 is low priority.
The pre-publication files will initially be:
- Sign the Release files
- Remove uncompressed Packages and Sources files
- Copy the indices files in place (override.* files)
The post-publication files will initially be:
- Run commercial-compat.sh
- Do the touchlist
- Generate ls-lr
- Run dsync
- Remove empty directories
- Run cron.germinate
- trigger the mirrors
The hook scripts will be responsible for deciding whether to run or not, based on the priority supplied.
Code Changes
Code changes should include an overview of what needs to change, and in some cases even the specific details.
Schema Changes
What Database changes are you proposing? Do you need a new index created? Proposing a new table? If so, what does it look like?
Migration
Include:
- data migration, if any
- redirects from old URLs to new ones, if any
how users will be pointed to the new way of doing things, if necessary. (If your change is big enough, consider using the rollout template.)
