Soyuz Upload Processing
Once package files are available on the upload servers, we need to run the Upload Processor to deal with them. This usually runs periodically from a cron job.
The Upload Processor
scripts/process-upload.py
lib/lp/archiveuploader/
The process-upload.py script is initiated with a single argument which is the base directory of the file upload area - let's call it $BASE. The catch is that this is the parent of $INCOMING because once things are processed from $INCOMING they are moved elsewhere.
The file structure looks like this:
$BASE/ | +-incoming/ ← $INCOMING | +-accepted/ | +-failed/ | +-rejected/
process-upload.py does many, many checks and then moves the files from the upload into:
- accepted: if the upload passes the checks
- failed: if there is a problem with the package itself
- rejected: if the package is OK itself but it's rejected for some other reason, e.g. policy checks
The upload processor always sends email whether the package was accepted or rejected, except in the following circumstances:
- The package's changes file is not properly GPG signed, or we can't match the GPG signature to a Launchpad person.
- A bug that causes the upload processor to crash. The only known bug right now is that some exceptions are treated as a fatal error instead of an upload error, so they OOPS instead.
Ancestry and auto-acceptance
When a package is accepted, it can then go into one of three separate states in the PackageUpload table:
- if it is considered new, that is we've never seen that package before in the upload context, it is held in the distroseries NEW queue.
- if it's not new and has "ancestry" in the distroseries context, but the distroseries is either frozen or released, it is held in the UNAPPROVED queue
if it's not new and doesn't meet the previous condition, it is "auto-accepted" - the PackageUpload moves to the DONE state and a PENDING publishing record is created.
If a package being uploaded is for a PPA then it is always auto-accepted. PPAs do not have distroseries queues.