The APPocalypse

The apocalypse split the LP codebase in to app packages. We wish to continue this effort and clean up some remaining parts where the split isn't quite as clean as it should be.

We would then like to build on this in order to actually split the codebase in to separate branches, with the following gains:

• A subset of tests will cover each app, reducing the test cycle.
• Cleaner code with less mixing of concerns.
• Ability to deploy Soyuz without the other apps.

What we are not doing:

• Splitting every app in to a standalone project that can run independent of everything else.
• Splitting the database.

The plan

We will aim to have several clearly defined layers:

• Foundations (possibly more like a set of libraries)
• Registry
• Soyuz
• Everything else

and attempt to only have each item in that list import from the one above.

Where one part can communicate with lower layers we will use soft dependencies, possibly built around a messaging system to allow them to co-operate.

Design

Implementation

Making the registry not depend on other apps

Currently, every application depends on the registry, which in turn depends on every other application. That makes it impossible to break Launchpad into separate applications, so we'll start by making sure the registry does not have hard dependencies on any other application.

For example, the Product model class implements IBugTarget, but the code that does that uses stuff from lp.bugs. We will move that code into an adapter for IProduct to IBugTarget so that Product no longer depends on lp.bugs.

lp:~salgado/launchpad/project-to-bugs-adapter has an adapter for IProduct to IBugTarget (diff can be seen at http://paste.ubuntu.com/456511/).

Here are some issues/questions I have so far:

1. IProduct no longer extends IBugTarget, so the pages registered for IBugTarget won't be available for IProducts. (the diff above has a long comment with a few possible solutions)
   • To work around that, I've created ICanBeBugTarget, moved all IBugTarget pages to it and made both IBugTarget and IProduct inherit from it. That way the pages are available for classes implementing either IBugTarget or ICanBeBugTarget.
   • Another alternative, suggested by James, would be to change the implementation of our browser:page directive to register extra views for things that can be adapted into the class that the page is for. That would require a way to query all the adapters for anything to the page's context, but I'm not sure we have that. Also, although the zcml registration would be for one interface (e.g. IBugTask), the view class would actually get something else (e.g. IProduct), which might be confusing to developers.
2. I created an adapter specific to the IBugTarget interface, but instead we could have a single adapter that provides all the bugs-related interfaces currently implemented by Product (e.g. IProductBugs). The only advantage I see is if we want to do the automatic adaptation solution for item 1 above.
3. Some app-specific things are edited through the project's main +edit page (e.g. bug_reporting_guidelines, official_* and translation_focus, just to name a few). Although editing these doesn't seem to depend on any app-specific bits, it might do for others.

This is still a work in progress (lots of tests will probably fail), but it should help us identify possible pain points and see whether or not the whole idea is sane.

Webservice

The webservice is one thing which the adapter approach does not completely solve. See https://lists.launchpad.net/launchpad-dev/msg03676.html for a description of the issues and a discussion of a possible solution.

Making Soyuz not depend on other apps

1. Translation tarballs are currently uploaded via ITranslationImportQueue.

• - If we don't care about translations then this can be stopped completely. Presumably there would be no tarballs to upload, so we wouldn't even trigger the code. - We still need to break the link though. - This is a case for events/messages/something rather than directly calling ITranslationImportQueue. - Currently involves lib/lp/soyuz/model/sourcepackagerelease.py for doing the work and lib/lp/soyuz/doc/distroseriesqueue-translations.txt for testing the code.

1. Bugs are closed by uploads/copies.

• - When a package is accepted the bugs mentioned in the changelog are closed. - Again if we don't have a bugtracker then we don't care. - We will probably have bug references regardless of whether we have a bugtracker. - This is another case for events/messages/similar. - Currently involves lib/lp/soyuz/scripts/processaccepted.py for doing the work and lib/lp/soyuz/scripts/tests/test_copypackage.py and lib/lp/soyuz/scripts/tests/test_queue.py for testing.

1. Uploads of source packages in archiveuploader are linked to SourcePackageRecipeBuilds.

• - lib/lp/archiveuploader/dscfile.py looks up the SPRB when processing an upload resulting from a recipe build, and links this in to the SourcePackageRelease record. - How to fix this?

1. SourcePackageRecipeUploadPolicy has to look up the build when processing an upload.

• - Similar to the last one, in lib/lp/archiveuploader/uploadpolicy.py has to look up the requester of the build. - Possibly push this in to the .dsc?

1. IBuildFarmBranchJob

• - Links IBuildFarmJob and IBranchJob - Why is it needed? - How to fix it?

1. twistedsftp imports error from lp.codehosting.sftp

• - lib/lp/poppy/twistedsftp.py and lib/lp/poppy/tests/test_twistedsftp.py import FileIsADirectory from codehosting. - Move to a shared location, probably in lp.services.sshserver?

Unresolved issues

• Should we try to break the list of external dependencies (sourcedeps.conf and versions.cfg on a per-branch basis)?