Policy for Documenting Custom Distributions

This document refers to our dependencies managed through setup.py and zc.buildout.

Background

We prefer to use formally released packages of our dependencies. This encourages us to contribute our improvements and fixes upstream; and it encourages us to use software that has a higher chance of having been reviewed for correctness by the upstream team.

Corollary: If we want a fix, and we can use the trunk of a package, and we can release them ourselves, do it. For instance, various people on the Launchpad team have connections to releasing many Zope-related packages, Twisted, bzr, and Storm. Leaning on people for release is good.

The following circumstances are examples of why this may sometimes not work.

• We may not be able to get a package released fast enough to address a Launchpad bug.
• Some dependencies may not provide official releases.
• The change we want is not yet accepted upstream.
• Even though the fix has been accepted in the most recent release, Launchpad cannot yet use it because of backwards incompatibilities in the release that Launchpad does not yet accommodate.

Some concrete examples from recent Launchpad history include Storm, feedvalidator, and zc.buildout.

All of these circumstances should be regarded as temporary situations: we should return to using an officially-released distribution as soon as possible, and interact with upstream to help make this happen. This is an integral part of being part of the open-source community, and the Launchpad team should certainly be leading by example.

But meanwhile, what do we do for a temporary fix of these circumstances?

Problem

Sometimes we need to make custom, local distributions of packages that we use. We want all developers to know how to get an original copy of the code that we are using for our distributions, so that we can all make subsequent changes and releases, and we can all try to push patches upstream.

Considerations

Do we need to keep all branches in Launchpad, or is it acceptable to defer to upstream repositories for a given project?

Our current solution specifies that it is OK to defer to upstream repositories if we do not have an import of the branch. It is particularly suggested that you defer to upstream if you are using a (hopefully) short-lived branch from the upstream repository. Otherwise, if you are using a revision of the project's trunk, it is preferred that you use a Launchpad automatic import.

Proposed Solution

versions.cfg (in the top level of the Launchpad tree) is where we specify our versions. Whenever we add a custom local distribution there, we should precede it with at least three comment lines.

The first should indicate the location of the sourcecode (trunk or branch or tag), that was used to create the distribution. The location may be in Launchpad or in the canonical upstream repository for the code, if it uses bzr, git, hg, or svn (excluding cvs because I suspect many of us have forgotten how to work with it.

The second should indicate the revision number of the sourcecode.

The last comment lines should show the exact command(s) used to generate the local distribution from the given branch. This can be important because of setuptools flags like -r and -b that allow you to change the effective version identifier.

The idea is that another developer should be able to check out the sourcecode at the given revision id and run the listed command to get an exact copy of the custom distribution we are using.