launchpad development|
Size: 9036
Comment:
|
Size: 12076
Comment:
|
| Deletions are marked like this. | Additions are marked like this. |
| Line 1: | Line 1: |
| ## page was renamed from HackingOpenSource ## page was renamed from Hacking |
|
| Line 5: | Line 3: |
| <<TableOfContents(2)>> |
|
| Line 11: | Line 11: |
| To develop or run tests, first run ``$DESIRED_PYTHON bootstrap.py`` in the package's top level directory, where you replace ``$DESIRED_PYTHON`` with the python executable that want to used to build and run this project. At this time, a non-system Python, or a virtualenv executable that does not include site-packages, is highly recommended, and may be required. We are working to remove this limitation. Then run ``./bin/buildout``. (NOTE: Ubuntu Intrepid users, see the note at the bottom of this document if this generates errors.) You can now run tests with ``./bin/test``. Use ``./bin/test --help`` to read about the many options. To gain access to a Python interpreter with the package and its dependent eggs available, use ``./bin/py``. You can generate ctags and idutils files for a variety of editors using ``./bin/tags`` (see ``./bin/tags --help``). The advantage of the files generated from this utility is that they include the sourcecode from the eggs used in this buildout. To generate distributions, use ``./bin/buildout setup . SETUP_CMD [...]``. That is, you can use ``./bin/buildout setup .`` as if it were ``python setup.py``. The intended advantage is easy access to a pristine, local version of setuptools. |
To develop or run tests, first run `$DESIRED_PYTHON bootstrap.py` in the package's top level directory, where you replace `$DESIRED_PYTHON` with the python executable that want to used to build and run this project. At this time, a non-system Python, or a virtualenv executable that does not include site-packages, is highly recommended, and may be required. We are working to remove this limitation. To use virtualenv, install it (`sudo apt-get install python-virtualenv`) and then run `virtualenv --no-site-packages [name of directory]`. Once you've done that, use the Python executable in `bin` to run `bootstrap.py` as described above. Then run `./bin/buildout`. You can now run tests with `./bin/test`. Use `./bin/test --help` to read about the many options. To gain access to a Python interpreter with the package and its dependent eggs available, use `./bin/py`. You can generate `ctags` and `idutils` files for a variety of editors using `./bin/tags` (see `./bin/tags --help`). The advantage of the files generated from this utility is that they include the sourcecode from the eggs used in this buildout. To generate distributions, use `./bin/buildout setup . SETUP_CMD [...]`. That is, you can use `./bin/buildout setup .` as if it were `python setup.py`. The intended advantage is easy access to a pristine, local version of setuptools. |
| Line 39: | Line 27: |
| Every project using zc.buildout will keep its own collection of eggs by default. You may want to cache these eggs, and their downloaded distributions globally. This also allows you to share common eggs across multiple packages. To do this, perform the following instructions: |
Every project using zc.buildout will keep its own collection of eggs by default. You may want to cache these eggs and their downloaded distributions globally so you can share them across all the projects you work with. To do this we create a `.buildout/` directory in our home folder. Buildout will look for the file `.buildout/default.cfg` and load settings from it. |
| Line 66: | Line 53: |
| If you have a source package, you can add it to this buildout (temporarily or, if really appropriate, permanently) as follows. NOTE: We'll use "this software" to refer to the software/buildout to which you want to add a source package; and "the other software" to refer to the source package that you want to add. - Add the top-level directory (the one with the setup.py) of the other software to somewhere accessible to this software. You might use ``bzr add``, symbolic links, svn externals, or whatever the appropriate tool is for the situation. A reasonable option is to add it in the top-level directory of this software. - In this software's buildout.cfg, in the [buildout] section, look for a ``develop`` key. It will probably look like this: ``develop = .``. Edit it to include the directory of the other software. For instance, if you added it at the top level of this software's tree in a folder called "lazr_foo," you would change the line in buildout.cfg to read ``develop = . lazr_foo``. - Rerun ``bin/buildout``. Barring version conflicts in your setup.py or buildout configuration (which will be reported as errors when you run `bin/buildout``), you should now be using the source version of the dependency. Look at one of the scripts in ``bin``, such as ``bin/test`` to verify, if you like. To undo, remove the line in buildout.conf, remove the other software's directory if necessary for cleanliness, and rerun ``bin/buildout``. |
If you have a source package, you can add it to this buildout (temporarily or, if really appropriate, permanently) as follows. NOTE: We'll use "this software" to refer to the software/buildout to which you want to add a source package; and "the other software" to refer to the source package that you want to add. 1. Put the top-level directory (the one with the setup.py) of the other software somewhere accessible to this software. The best thing to do is to locate it in a sibling tree so that there is no confusion about it being separate. 1. Every time you run buildout and want the other software to be available you need to tell buildout where the other piece of software is located. To do that set the ``develop`` key: {{{bin/buildout buildout:develop=". ../path/to/other/package"}}} You ''can'' set the key persistently in buildout.cfg but that will get committed and as such leads to mistaken commits which no longer work for other people. 1. When you want to switch back to using an egg for that other component just run `bin/buildout` - Barring version conflicts in your setup.py or buildout configuration (which will be reported as errors when you run `bin/buildout``), you should now be using the source version of the dependency. You can verify this by running `bin/py` and importing the library - check its __version__ or some similar aspect. |
| Line 96: | Line 70: |
| If you have a source distribution or an egg, and you set up the download-cache as described in the `Global Cache`_ section above, you can put the distribution (sdist or egg) in the download-cache's ``dist`` directory. Then rerun bin/buildout. Alternatively, if you have an egg, you can put it in your eggs directory, or in the shared eggs directory as described in the `Global Cache`_ section, if you are using that. The rerun bin/buildout. In both cases, to undo, you must remove the egg from the local or shared ``eggs`` directory. If you used the ``download-cache/dist`` directory, you must also remove it from there. |
If you have a source distribution or an egg, and you set up the download-cache as described in the `Global Cache`_ section above, you can put the distribution (sdist or egg) in the download-cache's ``dist`` directory. Then rerun bin/buildout. Alternatively, if you have an egg, you can put it in your eggs directory, or in the shared eggs directory as described in the `Global Cache`_ section, if you are using that. The rerun bin/buildout. In both cases, to undo, you must remove the egg from the local or shared ``eggs`` directory. If you used the ``download-cache/dist`` directory, you must also remove it from there. |
| Line 111: | Line 78: |
| If you want to distribute a source distribution privately, see http://pypi.python.org/pypi/zc.buildoutsftp/. |
If you want to distribute a source distribution privately, see http://pypi.python.org/pypi/zc.buildoutsftp/. |
| Line 116: | Line 82: |
| There is much, much more to learn about zc.buildout. It works well for small packages, but is even better for big projects. For more information about zc.buildout: |
There is much, much more to learn about zc.buildout. It works well for small packages, but is even better for big projects. For more information about zc.buildout: |
| Line 129: | Line 93: |
| === Ubuntu Intrepid Problem === Intrepid has a bug working with buildout. You may have to hack to fix it. Here's an example of the Python 2.4 version of the hack. $ sudo rm /usr/lib/python2.4/site-packages/GMenuSimpleEditor/*.py $ sudo ln -s /var/lib/python-support/python2.4/GMenuSimpleEditor/*.py /usr/lib/python2.4/site-packages/GMenuSimpleEditor/ Python 2.5 would need the same kind of fix (replace all three instances of "python2.4" with "python2.5" in the above). The author of this document tried an Intrepid upgrade and an aptitude reinstall of python-gmenu to no avail before doing the rough-and-tumble change described above. See https://bugs.edge.launchpad.net/ubuntu/+source/gnome-menus/+bug/301571 and the bug linked to it by datakid. |
|
| Line 151: | Line 95: |
| When hacking on LAZR it's useful to have the put something like this in your bazaar `.location.conf` file: |
When hacking on LAZR it's useful to have the put something like this in your bazaar `.location.conf` file: |
| Line 173: | Line 116: |
| public_branch = bzr+ssh://bazaar.launchpad.net/~launchpad-pqm/lazr.config/trunk | public_branch = bzr+ssh://bazaar.launchpad.net/~lazr-developers/lazr.config/trunk |
| Line 178: | Line 121: |
| = Landing your branches = If you're familiar with the Launchpad review process, the lazr process is very similar. If you're not familiar with the Launchpad review process, don't worry: the lazr process is simpler. 1. Branch from the public branch at lp:lazr.[project] and do your development. 2. Make sure all the existing tests pass and that you have good test coverage for your new code. (Feel free to ask for help in #launchpad-dev on FreeNode.) 3. Once you're done developing a branch, push it to Launchpad. {{{bzr push}}} should work if you've set up your bzr configuration as per above. Otherwise, you'll need to specify a destination URL: {{{bzr push lp:~[yourlpnick]/lazr.[project]/[your-branch-name]}}} 4. Visit the Launchpad page for your branch (it'll be linked from http://code.launchpad.net/~[yourlpnick]). Propose it for merging into the public branch at lp:lazr.[project]. Describe the changes you've made, with links to any bugs you're trying to fix. 5. Get your merge proposal reviewed. The #launchpad-reviews channel has one or more reviewers on call, most of the time Monday-Friday. If you can't find anyone to review your branch, contact Leonard Richardson (IRC:leonardr, email:leonard.richardson@canonical.com). 6. A review is an iterative process: the reviewer may ask you questions or ask you to make changes to your branch. Once the reviewer thinks your branch should be merged into the main branch, they'll vote 'approve' on your merge proposal. 7. If you are a Launchpad developer or a member of the lazr-developers team, it's your responsibility to land your own branch. If you're not, the person who reviewed your merge proposal is responsible for landing your branch. ''Don't let them forget, or your branch will just sit on Launchpad doing nothing!'' Here's how to land a reviewed branch: 1. Check out a copy of the public branch. {{{bzr co lp:lazr.[project]}}} should do it. If you already have a copy, {{{cd}}} into the branch and use {{{bzr up}}} to pick up other peoples' changes. 2. {{{bzr merge}}} your development branch with the public branch. Resolve any conflicts (if these are serious, you might want to get the branch re-reviewed). 3. Build the branch and run the tests to be sure the buildout succeeds and all the tests pass. 4. Add an entry describing your changes to the NEWS.txt file. 5. {{{bzr commit}}} your changes to the development branch. Be sure to include the IRC nick of the person who reviewed your branch, and the bug number if appropriate Example: {{{ bzr commit -m "[r=barry][bug=426323] Put 'title' attributes on sections in the API documentation." }}} If you're landing a branch for someone else, be sure to also mention the name of the person who did the original development. {{{ bzr commit -m "[r=allenap] Add a credentials_file parameter to login_with(). Landed on behalf of Martin Pitt." }}} Use {{{bzr log}}} to look into the past and see what makes a good commit message. Finally, push the updated development branch back to where it came from using {{{bzr push lp:lazr.[project]}}}. |
|
| Line 180: | Line 157: |
| launchpadlib contains a script for uploading release tarballs to Launchpad. Here's how to do a release of a lazr package using this script. | launchpadlib contains a script for uploading release tarballs to Launchpad. Here's how to do a release of a lazr package using this script. See also ReleaseChecklist. |
| Line 186: | Line 163: |
| 1. Create the tarball. {{{ $ python setup.py sdist }}} 2. The tarball will be created in dist/lazr.[package]-[version].tar.gz. Sign it with your GPG key. |
1. Check the commit log and NEWS.txt file to be sure the important changes are reflected there. 2. Set the correct date and version for this release in the NEWS.txt file. 3. Set the correct version for this release in the version.txt file. 4. Run the buildout and tests to be sure everything is in working order. 5. Commit the above changes. 6. Create the tarball. (These steps use {{{./bin/buildout setup .}}} rather than the more standard {{{python setup.py}}} because the first is more likely to work without effort. If you prefer {{{python setup.py}}} and it works for you, go ahead.) {{{ $ ./bin/buildout setup . sdist }}} 7. The tarball will be created in dist/lazr.[package]-[version].tar.gz. Sign it with your GPG key. |
| Line 198: | Line 185: |
| 3. That will create a signature file dist/lazr.[package]-[version].tar.gz.asc. Now, upload the tarball and its signature. | 8. That will create a signature file dist/lazr.[package]-[version].tar.gz.asc. Now, upload the tarball and its signature. |
| Line 204: | Line 191: |
| 4. The tarball is now available from Launchpad. Now, register the package with PyPI. {{{ $ python setup.py register |
9. The tarball is now available from Launchpad. Now, register the package with PyPI. {{{ $ ./bin/buildout setup . register }}} 10. Tag the version. {{{ $ bzr tag [version] |
Common Launchpad Component Hacking Tips
Contents
These hacking tips are common to all Launchpad open-sourced components (i.e., lazr.*). Additional component-specific information may be found in the HACKING.txt file for that component.
You may also want to see the Launchpad Hacking page.
Basics
To develop or run tests, first run $DESIRED_PYTHON bootstrap.py in the package's top level directory, where you replace $DESIRED_PYTHON with the python executable that want to used to build and run this project. At this time, a non-system Python, or a virtualenv executable that does not include site-packages, is highly recommended, and may be required. We are working to remove this limitation.
To use virtualenv, install it (sudo apt-get install python-virtualenv) and then run virtualenv --no-site-packages [name of directory]. Once you've done that, use the Python executable in bin to run bootstrap.py as described above.
Then run ./bin/buildout.
You can now run tests with ./bin/test. Use ./bin/test --help to read about the many options.
To gain access to a Python interpreter with the package and its dependent eggs available, use ./bin/py.
You can generate ctags and idutils files for a variety of editors using ./bin/tags (see ./bin/tags --help). The advantage of the files generated from this utility is that they include the sourcecode from the eggs used in this buildout.
To generate distributions, use ./bin/buildout setup . SETUP_CMD [...]. That is, you can use ./bin/buildout setup . as if it were python setup.py. The intended advantage is easy access to a pristine, local version of setuptools.
Global Cache
Every project using zc.buildout will keep its own collection of eggs by default. You may want to cache these eggs and their downloaded distributions globally so you can share them across all the projects you work with.
To do this we create a .buildout/ directory in our home folder. Buildout will look for the file .buildout/default.cfg and load settings from it.
mkdir -p ~/.buildout/eggs mkdir ~/.buildout/download-cache
Create ~/.buildout/default.cfg with the following content, replacing "$HOME" with the path to your home::
[buildout] eggs-directory=$HOME/.buildout/eggs download-cache=$HOME/.buildout/download-cache
Working with Unreleased Packages
You may want to develop with an unreleased distribution of another package. To do so, you have many options. Here are three.
Source
If you have a source package, you can add it to this buildout (temporarily or, if really appropriate, permanently) as follows.
NOTE: We'll use "this software" to refer to the software/buildout to which you want to add a source package; and "the other software" to refer to the source package that you want to add.
- Put the top-level directory (the one with the setup.py) of the other software somewhere accessible to this software. The best thing to do is to locate it in a sibling tree so that there is no confusion about it being separate.
Every time you run buildout and want the other software to be available you need to tell buildout where the other piece of software is located. To do that set the develop key: bin/buildout buildout:develop=". ../path/to/other/package" You can set the key persistently in buildout.cfg but that will get committed and as such leads to mistaken commits which no longer work for other people.
When you want to switch back to using an egg for that other component just run bin/buildout
- Barring version conflicts in your setup.py or buildout configuration (which will be reported as errors when you run bin/buildout), you should now be using the source version of the dependency. You can verify this by running bin/py` and importing the library - check its version or some similar aspect.
Distribution
If you have a source distribution or an egg, and you set up the download-cache as described in the Global Cache_ section above, you can put the distribution (sdist or egg) in the download-cache's dist directory. Then rerun bin/buildout.
Alternatively, if you have an egg, you can put it in your eggs directory, or in the shared eggs directory as described in the Global Cache_ section, if you are using that. The rerun bin/buildout.
In both cases, to undo, you must remove the egg from the local or shared eggs directory. If you used the download-cache/dist directory, you must also remove it from there.
Private Source
If you want to distribute a source distribution privately, see http://pypi.python.org/pypi/zc.buildoutsftp/.
References
There is much, much more to learn about zc.buildout. It works well for small packages, but is even better for big projects. For more information about zc.buildout:
- http://grok.zope.org/documentation/tutorial/introduction-to-zc.buildout
- Presentation notes of an introduction to zc.buildout by its author, Jim Fulton.
- http://pypi.python.org/pypi/zc.buildout
- The user manual.
Bazaar Configuration
When hacking on LAZR it's useful to have the put something like this in your bazaar .location.conf file:
# This is the directory containing all the LAZR projects you are hacking on. # This could be a shared repository, or you can make each subdirectories a # shared repository. [/home/<yourname>/canonical] email = Me <me@example.com> pqm_email = Canonical PQM <launchpad@pqm.canonical.com> push_location = bzr+ssh://bazaar.launchpad.net/~yourlpnick push_location:policy = appendpath public_branch = bzr+ssh://bazaar.launchpad.net/~yourlpnick public_branch:policy = appendpath submit_to = merge@code.launchpad.net mail_client = editor # You then add the following two stanzas for each LAZR project [/home/<yourname>/canonical/lazr.config] submit_branch = /home/<yourname>/canonical/lazr.config/trunk [/home/<yourname>/canonical/lazr.config/trunk] public_branch = bzr+ssh://bazaar.launchpad.net/~lazr-developers/lazr.config/trunk # You should then be able to bzr push/send in any branches.
Landing your branches
If you're familiar with the Launchpad review process, the lazr process is very similar. If you're not familiar with the Launchpad review process, don't worry: the lazr process is simpler.
- Branch from the public branch at lp:lazr.[project] and do your development.
Make sure all the existing tests pass and that you have good test coverage for your new code. (Feel free to ask for help in #launchpad-dev on FreeNode.)
Once you're done developing a branch, push it to Launchpad. bzr push should work if you've set up your bzr configuration as per above. Otherwise, you'll need to specify a destination URL: bzr push lp:~[yourlpnick]/lazr.[project]/[your-branch-name]
Visit the Launchpad page for your branch (it'll be linked from http://code.launchpad.net/~[yourlpnick]). Propose it for merging into the public branch at lp:lazr.[project]. Describe the changes you've made, with links to any bugs you're trying to fix.
Get your merge proposal reviewed. The #launchpad-reviews channel has one or more reviewers on call, most of the time Monday-Friday. If you can't find anyone to review your branch, contact Leonard Richardson (IRC:leonardr, email:leonard.richardson@canonical.com).
- A review is an iterative process: the reviewer may ask you questions or ask you to make changes to your branch. Once the reviewer thinks your branch should be merged into the main branch, they'll vote 'approve' on your merge proposal.
If you are a Launchpad developer or a member of the lazr-developers team, it's your responsibility to land your own branch. If you're not, the person who reviewed your merge proposal is responsible for landing your branch. Don't let them forget, or your branch will just sit on Launchpad doing nothing!
Here's how to land a reviewed branch:
Check out a copy of the public branch. bzr co lp:lazr.[project] should do it. If you already have a copy, cd into the branch and use bzr up to pick up other peoples' changes.
bzr merge your development branch with the public branch. Resolve any conflicts (if these are serious, you might want to get the branch re-reviewed).
- Build the branch and run the tests to be sure the buildout succeeds and all the tests pass.
- Add an entry describing your changes to the NEWS.txt file.
bzr commit your changes to the development branch. Be sure to include the IRC nick of the person who reviewed your branch, and the bug number if appropriate Example:
bzr commit -m "[r=barry][bug=426323] Put 'title' attributes on sections in the API documentation."
If you're landing a branch for someone else, be sure to also mention the name of the person who did the original development.
bzr commit -m "[r=allenap] Add a credentials_file parameter to login_with(). Landed on behalf of Martin Pitt."
Use bzr log to look into the past and see what makes a good commit message.
Finally, push the updated development branch back to where it came from using bzr push lp:lazr.[project].
Releases
launchpadlib contains a script for uploading release tarballs to Launchpad. Here's how to do a release of a lazr package using this script. See also ReleaseChecklist.
This assumes you have made your desired changes in a branch, gotten them reviewed, and merged them to trunk. It also assumes that your changes include updating version.txt and NEWS.txt appropriately. The version number in version.txt should be bumped by a full release number (e.g., 1.X -> 2.0) rarely, and with the buy-in of the primary maintainer(s); by a major release number (e.g., 1.3 -> 1.4) for feature changes; and by a minor release number (e.g., 1.3 -> 1.3.1) for bugfix changes only. NEWS.txt should describe the changes for your release.
This also assumes (in step three) that you have a recent checkout of launchpadlib's trunk that you have prepared for development with buildout using a Python without site-packages. This can be a locally-built Python from python.org, or a virtualenv Python. (Work is under way to make it possible to use the system Python trivially.) Minimally, where "python" indicates the executable without site-packages, use bzr branch lp:launchpadlib && cd launchpadlib && python bootstrap.py && bin/buildout. You may want to see the discussion of the global cache, above.
1. Check the commit log and NEWS.txt file to be sure the important changes are reflected there.
2. Set the correct date and version for this release in the NEWS.txt file.
3. Set the correct version for this release in the version.txt file.
4. Run the buildout and tests to be sure everything is in working order.
5. Commit the above changes.
6. Create the tarball. (These steps use ./bin/buildout setup . rather than the more standard python setup.py because the first is more likely to work without effort. If you prefer python setup.py and it works for you, go ahead.)
$ ./bin/buildout setup . sdist
7. The tarball will be created in dist/lazr.[package]-[version].tar.gz. Sign it with your GPG key.
$ gpg --armor --sign --detach-sig dist/lazr.[package]-[version].tar.gz
8. That will create a signature file dist/lazr.[package]-[version].tar.gz.asc. Now, upload the tarball and its signature.
$ path/to/launchpadlib/bin/py path/to/launchpadlib/contrib/upload_release_tarball.py lazr.[package] [release series] [version] dist/lazr.[package]-[version].tar.gz
9. The tarball is now available from Launchpad. Now, register the package with PyPI.
$ ./bin/buildout setup . register
10. Tag the version.
$ bzr tag [version]
Now the world knows about the new release.
