=========================================
 Common Launchpad Component Hacking Tips
=========================================

.. contents:: **Table of contents**

These hacking tips are common to all Launchpad open-sourced components (i.e., ``lazr.*``).  Additional component-specific information may be found in the ``HACKING.txt`` file for that component.

You may also want to see the `Launchpad Hacking <Hacking>`_ page.

Basics
======

To develop or run tests, first run ``$DESIRED_PYTHON bootstrap.py`` in the package's top level directory, where you replace ``$DESIRED_PYTHON`` with the python executable that want to used to build and run this project. 

Then run ``./bin/buildout``.

You can now run tests with ``./bin/test``.  Use ``./bin/test --help`` to read about the many options.

To gain access to a Python interpreter with the package and its dependent eggs available, use ``./bin/py``.

You can generate ``ctags`` and ``idutils`` files for a variety of editors using ``./bin/tags`` (see ``./bin/tags --help``). The advantage of the files generated from this utility is that they include the sourcecode from the eggs used in this buildout.

To generate distributions, use ``./bin/buildout setup . SETUP_CMD [...]``. That is, you can use ``./bin/buildout setup .`` as if it were ``python setup.py``.  The intended advantage is easy access to a pristine, local version of setuptools.

Global Cache
============

Every project using zc.buildout will keep its own collection of eggs by default.  You may want to cache these eggs and their downloaded distributions globally so you can share them across all the projects you work with.

We maintain a Bazaar branch with all the source distributions of all the dependencies used by Launchpad and our other components. It's the 
`lp:lp-source-dependencies <https://launchpad.net/lp-source-dependencies>`_ project. Here's the recommended way to set-up a
common cache (substitute `canonical` for the directory where you have set-up your lazr projects (as described in the `Bazaar Configuration`_ section)::

  mkdir -p ~/canonical/lp-source-dependencies/eggs
  cd ~/canonical/lp-source-dependencies
  bzr co --lightweight lp:lp-source-dependencies download-cache

You can then ``ln -s ~/canonical/lp-source-dependencies/{eggs,download-cache} .`` in your working directories to use the cache automatically.

Working with Unreleased Packages
================================

You may want to develop with an unreleased distribution of another package.
To do so, you have many options.  Here are three.

Source
------

If you have a source package, you can add it to this buildout (temporarily or, if really appropriate, permanently) as follows.

**NOTE:** We'll use "this software" to refer to the software/buildout to which you want to add a source package; and "the other software" to refer to the source package that you want to add.

1. Put the top-level directory (the one with the setup.py) of the other software somewhere accessible to this software. The best thing to do is to locate it in a sibling tree so that there is no confusion about it being separate.

2. Every time you run buildout and want the other software to be available you need to tell buildout where the other piece of software is located. To do that set the ``develop`` key

  ::

  $ bin/buildout buildout:develop=". ../path/to/other/package"

  You *can* set the key persistently in buildout.cfg but that will get committed and as such leads to mistaken commits which no longer work for other people.

3. When you want to switch back to using an egg for that other component just run ``bin/buildout``

Barring version conflicts in your setup.py or buildout configuration (which will be reported as errors when you run ``bin/buildout```), you should now be using the source version of the dependency. You can verify this by running ``bin/py`` and importing the library -- check its ``__version__`` or some similar aspect.

Distribution
------------

If you have a source distribution or an egg, and you set up the download-cache as described in the `Global Cache`_ section above, you can put the distribution (sdist or egg) in the download-cache's ``dist`` directory. Then rerun bin/buildout.

Alternatively, if you have an egg, you can put it in your eggs directory, or in the shared eggs directory as described in the `Global Cache`_ section, if you are using that. The rerun bin/buildout.

In both cases, to undo, you must remove the egg from the local or shared ``eggs`` directory. If you used the ``download-cache/dist`` directory, you must also remove it from there.

Private Source
--------------

If you want to distribute a source distribution privately, see http://pypi.python.org/pypi/zc.buildoutsftp/.

References
----------

There is much, much more to learn about zc.buildout.  It works well for small packages, but is even better for big projects.  For more information about zc.buildout:

- http://grok.zope.org/documentation/tutorial/introduction-to-zc.buildout

  Presentation notes of an introduction to zc.buildout by its author, Jim
  Fulton.

- http://pypi.python.org/pypi/zc.buildout

  The user manual.

Bazaar Configuration
====================

When hacking on LAZR it's useful to have the put something like this in your bazaar ``.location.conf`` file::

  # This is the directory containing all the LAZR projects you are hacking on.
  # This could be a shared repository, or you can make each subdirectories a
  # shared repository.
  [/home/<yourname>/canonical]
  email = Me <me@example.com>
  pqm_email = Canonical PQM <launchpad@pqm.canonical.com>
  push_location = bzr+ssh://bazaar.launchpad.net/~yourlpnick
  push_location:policy = appendpath
  public_branch = bzr+ssh://bazaar.launchpad.net/~yourlpnick
  public_branch:policy = appendpath
  submit_to = merge@code.launchpad.net
  mail_client = editor

  # You then add the following two stanzas for each LAZR project
  [/home/<yourname>/canonical/lazr.config]
  submit_branch = /home/<yourname>/canonical/lazr.config/trunk

  [/home/<yourname>/canonical/lazr.config/trunk]
  public_branch = bzr+ssh://bazaar.launchpad.net/~lazr-developers/lazr.config/trunk

You should then be able to bzr push/send in any branches.

Landing your branches
=====================

If you're familiar with the Launchpad review process, the lazr process is very similar. If you're not familiar with the Launchpad review process, don't worry: the lazr process is simpler.

1. Branch from the public branch at lp:lazr.[project] and do your development.
2. Make sure all the existing tests pass and that you have good test coverage for your new code. (Feel free to ask for help in #launchpad-dev on irc.libera.chat.)
3. Once you're done developing a branch, push it to Launchpad. ``bzr push`` should work if you've set up your bzr configuration as per above. Otherwise, you'll need to specify a destination URL: ``bzr push lp:~[yourlpnick]/lazr.[project]/[your-branch-name]``
4. Visit the Launchpad page for your branch (it'll be linked from http://code.launchpad.net/~[yourlpnick]). Propose it for merging into the public branch at lp:lazr.[project]. Describe the changes you've made, with links to any bugs you're trying to fix.
5. Get your merge proposal reviewed. The #launchpad-dev channel has one or more reviewers on call, most of the time Monday-Friday. If you can't find anyone to review your branch, hassle random people in that channel until someone is willing to review your change.
6. A review is an iterative process: the reviewer may ask you questions or ask you to make changes to your branch. Once the reviewer thinks your branch should be merged into the main branch, they'll vote 'approve' on your merge proposal.
7. If you are a Launchpad developer or a member of the lazr-developers team, it's your responsibility to land your own branch. If you're not, the person who reviewed your merge proposal is responsible for landing your branch.

*Don't let them forget, or your branch will just sit on Launchpad doing nothing!*

Here's how to land a reviewed branch

1. Check out a copy of the public branch. ``bzr co lp:lazr.[project]`` should do it. If you already have a copy, ``cd`` into the branch and use ``bzr up`` to pick up other peoples' changes.

2. ``bzr merge`` your development branch with the public branch. Resolve any conflicts (if these are serious, you might want to get the branch re-reviewed).

3. Build the branch and run the tests to be sure the buildout succeeds and all the tests pass.

4. Add an entry describing your changes to the NEWS.txt file.

5. ``bzr commit`` your changes to the development branch. Be sure to include the IRC nick of the person who reviewed your branch, and the bug number if appropriate

   ::

     bzr commit -m "[r=barry][bug=426323] Put 'title' attributes on sections in the API documentation."

   If you're landing a branch for someone else, be sure to also mention the name of the person who did the original development

   ::

     bzr commit -m "[r=allenap] Add a credentials_file parameter to login_with(). Landed on behalf of Martin Pitt."

Use ``bzr log`` to look into the past and see what makes a good commit message.

Finally, push the updated development branch back to where it came from using ``bzr push lp:lazr.[project]``.


Releases
========

launchpadlib contains a script for uploading release tarballs to Launchpad. Here's how to do a release of a lazr package using this script. See also ReleaseChecklist.

This assumes you have made your desired changes in a branch, gotten them reviewed, and merged them to trunk.  It also assumes that your changes include updating version.txt and NEWS.txt appropriately.  The version number in version.txt should be bumped by a full release number (e.g., 1.X -> 2.0) rarely, and with the buy-in of the primary maintainer(s); by a major release number (e.g., 1.3 -> 1.4) for feature changes; and by a minor release number (e.g., 1.3 -> 1.3.1) for bugfix changes only.  ``NEWS.txt`` should describe the changes for your release.

This also assumes (in step three) that you have a recent checkout of launchpadlib's trunk that you have prepared for development with buildout '''using a Python without site-packages'''.  This can be a locally-built Python from python.org, or a virtualenv Python.  (Work is under way to make it possible to use the system Python trivially.)  Minimally, where "python" indicates the executable without site-packages, use ``bzr branch lp:launchpadlib && cd launchpadlib && python bootstrap.py && bin/buildout``. You may want to see the discussion of the global cache, above.

1. Check the commit log and NEWS.txt file to be sure the important changes are reflected there.  Review all of the revisions since the last release and include them, not just the change you just committed.

2. Set the correct date and version for this release in the NEWS.txt file.

3. Set the correct version for this release in the version.txt file.

4. Run the buildout and tests to be sure everything is in working order.

5. Commit the above changes.

6. Create the tarball.  (These steps use ``./bin/buildout setup .`` rather than the more standard ``python setup.py`` because the first is more likely to work without effort.  If you prefer ``python setup.py`` and it works for you, go ahead.)

  ::

  $ ./bin/buildout setup . sdist

7. The tarball will be created in dist/lazr.[package]-[version].tar.gz. Sign it with your GPG key

  ::

  $ gpg --armor --sign --detach-sig dist/lazr.[package]-[version].tar.gz

8. That will create a signature file dist/lazr.[package]-[version].tar.gz.asc. Now, upload the tarball and its signature

   ::

   $ path/to/launchpadlib/bin/py path/to/launchpadlib/contrib/upload_release_tarball.py lazr.[package] [release series] [version] dist/lazr.[package]-[version].tar.gz

9. The tarball is now available from Launchpad. Now, register the package with PyPI

   ::

   $ ./bin/buildout setup . register

10. Upload the tarball and signature to PyPI so pip can find it. Make sure you attach it to the right release.

11. Tag the version

    ::

    $ bzr tag [version]

12. Push the tag to the trunk.

    ::

    $ bzr push [trunk]

13. Update the daily build recipe, if it exists, to use the new version number in the first line of the recipe.  Existing LAZR can be found at https://code.launchpad.net/~launchpad/+recipes .

Now the world knows about the new release.