Common Launchpad Component Hacking Tips

These hacking tips are common to all Launchpad open-sourced components (i.e., lazr.*). Additional component-specific information may be found in the HACKING.txt file for that component.

You may also want to see the Launchpad Hacking page.

Basics

To develop or run tests, first run $DESIRED_PYTHON bootstrap.py in the package's top level directory, where you replace $DESIRED_PYTHON with the python executable that want to used to build and run this project. At this time, a non-system Python, or a virtualenv executable that does not include site-packages, is highly recommended, and may be required. We are working to remove this limitation.

To use virtualenv, install it (sudo apt-get install python-virtualenv) and then run virtualenv --no-site-packages [name of directory]. Once you've done that, use the Python executable in bin to run bootstrap.py as described above.

Then run ./bin/buildout. (NOTE: Ubuntu Intrepid users, see the note at the bottom of this document if this generates errors.)

You can now run tests with ./bin/test. Use ./bin/test --help to read about the many options.

To gain access to a Python interpreter with the package and its dependent eggs available, use ./bin/py.

You can generate ctags and idutils files for a variety of editors using ./bin/tags (see ./bin/tags --help). The advantage of the files generated from this utility is that they include the sourcecode from the eggs used in this buildout.

To generate distributions, use ./bin/buildout setup . SETUP_CMD [...]. That is, you can use ./bin/buildout setup . as if it were python setup.py. The intended advantage is easy access to a pristine, local version of setuptools.

Global Cache

Every project using zc.buildout will keep its own collection of eggs by default. You may want to cache these eggs and their downloaded distributions globally so you can share them across all the projects you work with.

To do this we create a .buildout/ directory in our home folder. Buildout will look for the file .buildout/default.cfg and load settings from it.

mkdir -p ~/.buildout/eggs
mkdir ~/.buildout/download-cache

Create ~/.buildout/default.cfg with the following content, replacing "$HOME" with the path to your home::

[buildout]
eggs-directory=$HOME/.buildout/eggs
download-cache=$HOME/.buildout/download-cache

Working with Unreleased Packages

You may want to develop with an unreleased distribution of another package. To do so, you have many options. Here are three.

Source

If you have a source package, you can add it to this buildout (temporarily or, if really appropriate, permanently) as follows.

NOTE: We'll use "this software" to refer to the software/buildout to which you want to add a source package; and "the other software" to refer to the source package that you want to add.

• 1 Add the top-level directory (the one with the setup.py) of the other

  • software to somewhere accessible to this software. You might use bzr add, symbolic links, svn externals, or whatever the appropriate tool is for the situation. A reasonable option is to add it in the top-level directory of this software. The best thing to do is to locate it in a sibling tree so that its separation is clear.

  1 When running buildout you need to tell buildout where this other piece of software can be found. To do that set the develop key:

  • bin/buildout buildout:develop=". ../path/to/other/package" Alternatively you can set the key persistently in buildout.cfg but that will get committed and as such is rarely what you want.

  1 When you want to switch back to using an egg for that other component run bin/buildout build:develop=.

- Barring version conflicts in your setup.py or buildout configuration (which will be reported as errors when you run bin/buildout`), you should now be using the source version of the dependency. You can verify this by running bin/py and importing the library - check its version or some similar aspect.

Distribution

If you have a source distribution or an egg, and you set up the download-cache as described in the Global Cache_ section above, you can put the distribution (sdist or egg) in the download-cache's dist directory. Then rerun bin/buildout.

Alternatively, if you have an egg, you can put it in your eggs directory, or in the shared eggs directory as described in the Global Cache_ section, if you are using that. The rerun bin/buildout.

In both cases, to undo, you must remove the egg from the local or shared eggs directory. If you used the download-cache/dist directory, you must also remove it from there.

Private Source

If you want to distribute a source distribution privately, see http://pypi.python.org/pypi/zc.buildoutsftp/.

References

There is much, much more to learn about zc.buildout. It works well for small packages, but is even better for big projects. For more information about zc.buildout:

- http://grok.zope.org/documentation/tutorial/introduction-to-zc.buildout

• Presentation notes of an introduction to zc.buildout by its author, Jim Fulton.

- http://pypi.python.org/pypi/zc.buildout

• The user manual.

Ubuntu Intrepid Problem

Intrepid has a bug working with buildout. You may have to hack to fix it. Here's an example of the Python 2.4 version of the hack.

$ sudo rm /usr/lib/python2.4/site-packages/GMenuSimpleEditor/*.py $ sudo ln -s /var/lib/python-support/python2.4/GMenuSimpleEditor/*.py /usr/lib/python2.4/site-packages/GMenuSimpleEditor/

Python 2.5 would need the same kind of fix (replace all three instances of "python2.4" with "python2.5" in the above).

The author of this document tried an Intrepid upgrade and an aptitude reinstall of python-gmenu to no avail before doing the rough-and-tumble change described above.

See https://bugs.edge.launchpad.net/ubuntu/+source/gnome-menus/+bug/301571 and the bug linked to it by datakid.

Bazaar Configuration

When hacking on LAZR it's useful to have the put something like this in your bazaar .location.conf file:

# This is the directory containing all the LAZR projects you are hacking on.
# This could be a shared repository, or you can make each subdirectories a
# shared repository.
[/home/<yourname>/canonical]
email = Me <me@example.com>
pqm_email = Canonical PQM <launchpad@pqm.canonical.com>
push_location = bzr+ssh://bazaar.launchpad.net/~yourlpnick
push_location:policy = appendpath
public_branch = bzr+ssh://bazaar.launchpad.net/~yourlpnick
public_branch:policy = appendpath
submit_to = merge@code.launchpad.net
mail_client = editor

# You then add the following two stanzas for each LAZR project
[/home/<yourname>/canonical/lazr.config]
submit_branch = /home/<yourname>/canonical/lazr.config/trunk

[/home/<yourname>/canonical/lazr.config/trunk]
public_branch = bzr+ssh://bazaar.launchpad.net/~lazr-developers/lazr.config/trunk

# You should then be able to bzr push/send in any branches.

Landing your branches

If you're familiar with the Launchpad review process, the lazr process is very similar. If you're not familiar with the Launchpad review process, don't worry: the lazr process is simpler.

1. Branch from the public branch at lp:lazr.[project] and do your development.

2. Make sure all the existing tests pass and that you have good test coverage for your new code. (Feel free to ask for help in #launchpad-dev on FreeNode.)

3. Once you're done developing a branch, push it to Launchpad. bzr push should work if you've set up your bzr configuration as per above. Otherwise, you'll need to specify a destination URL: bzr push lp:~[yourlpnick]/lazr.[project]/[your-branch-name]

4. Visit the Launchpad page for your branch (it'll be linked from http://code.launchpad.net/~[yourlpnick]). Propose it for merging into the public branch at lp:lazr.[project]. Describe the changes you've made, with links to any bugs you're trying to fix.

5. Get your merge proposal reviewed. The #launchpad-reviews channel has one or more reviewers on call, most of the time Monday-Friday. If you can't find anyone to review your branch, contact Leonard Richardson (IRC:leonardr, email:leonard.richardson@canonical.com).

6. A review is an iterative process: the reviewer may ask you questions or ask you to make changes to your branch. Once the reviewer thinks your branch should be merged into the main branch, they'll vote 'approve' on your merge proposal.

7. If you are a Launchpad developer or a member of the lazr-developers team, it's your responsibility to land your own branch. If you're not, the person who reviewed your merge proposal is responsible for landing your branch. Don't let them forget, or your branch will just sit on Launchpad doing nothing!

Here's how to land a reviewed branch:

1. Check out a copy of the public branch. bzr co lp:lazr.[project] should do it. If you already have a copy, cd into the branch and use bzr up to pick up other peoples' changes.

2. bzr merge your development branch with the public branch. Resolve any conflicts (if these are serious, you might want to get the branch re-reviewed).

3. Build the branch and run the tests to be sure the buildout succeeds and all the tests pass.
4. Add an entry describing your changes to the NEWS.txt file.

5. bzr commit your changes to the development branch. Be sure to include the IRC nick of the person who reviewed your branch, and the bug number if appropriate Example:

bzr commit -m "[r=barry][bug=426323] Put 'title' attributes on sections in the API documentation."

If you're landing a branch for someone else, be sure to also mention the name of the person who did the original development.

bzr commit -m "[r=allenap] Add a credentials_file parameter to login_with(). Landed on behalf of Martin Pitt."

Use bzr log to look into the past and see what makes a good commit message.

Finally, push the updated development branch back to where it came from using bzr push lp:lazr.[project].

Releases

launchpadlib contains a script for uploading release tarballs to Launchpad. Here's how to do a release of a lazr package using this script. See also ReleaseChecklist.

This assumes you have made your desired changes in a branch, gotten them reviewed, and merged them to trunk. It also assumes that your changes include updating version.txt and NEWS.txt appropriately. The version number in version.txt should be bumped by a full release number (e.g., 1.X -> 2.0) rarely, and with the buy-in of the primary maintainer(s); by a major release number (e.g., 1.3 -> 1.4) for feature changes; and by a minor release number (e.g., 1.3 -> 1.3.1) for bugfix changes only. NEWS.txt should describe the changes for your release.

This also assumes (in step three) that you have a recent checkout of launchpadlib's trunk that you have prepared for development with buildout using a Python without site-packages. This can be a locally-built Python from python.org, or a virtualenv Python. (Work is under way to make it possible to use the system Python trivially.) Minimally, where "python" indicates the executable without site-packages, use bzr branch lp:launchpadlib && cd launchpadlib && python bootstrap.py && bin/buildout. You may want to see the discussion of the global cache, above.

1. Check the commit log and NEWS.txt file to be sure the important changes are reflected there.

2. Set the correct date and version for this release in the NEWS.txt file.

3. Set the correct version for this release in the version.txt file.

4. Run the buildout and tests to be sure everything is in working order.

5. Commit the above changes.

6. Create the tarball. (These steps use ./bin/buildout setup . rather than the more standard python setup.py because the first is more likely to work without effort. If you prefer python setup.py and it works for you, go ahead.)

$ ./bin/buildout setup . sdist

7. The tarball will be created in dist/lazr.[package]-[version].tar.gz. Sign it with your GPG key.

$ gpg --armor --sign --detach-sig dist/lazr.[package]-[version].tar.gz

8. That will create a signature file dist/lazr.[package]-[version].tar.gz.asc. Now, upload the tarball and its signature.

$ path/to/launchpadlib/bin/py path/to/launchpadlib/contrib/upload_release_tarball.py lazr.[package] [release series] [version] dist/lazr.[package]-[version].tar.gz

9. The tarball is now available from Launchpad. Now, register the package with PyPI.

$ ./bin/buildout setup . register

10. Tag the version.

$ bzr tag [version]

Now the world knows about the new release.