launchpad development
Help on Foundations Components
Ideas on how to build this document (summary: if you think something needs to be done, Just Do It; no discussion is necessary unless you deem it so):
- Please fill in existing incomplete listings. If you think it is a bad listing (including irrelevant or too small to bother with), feel free to delete it or adjust it or comment on it.
- Feel free to add new components and new sections that you believe are missing.
- Feel free to rearrange existing components and sections. Discuss first if you deem necessary.
- For every component, try to answer these questions briefly, with only a sentence or sentence fragment answer for each one:
- What is it? Include links if appropriate.
- What components use it?
- To read and understand the code, where do you start?
- To read documentation, where do you start?
- If you need more information, who should you ask?
(Bonus question, if you can do this very quickly) To debug, where do you start?
- If you think we need to say more or less about a particular component, Just Do It.
These need to be quick to write. We don't have time to dig in. These are pointers. Each topic should take, say, no more than five minutes to write. If you have taken ten minutes on a single topic, please step away from the computer :-).
I didn't include a lot of DB stuff in my own initial sketch, because stub is still the DBA. Stub or anyone else, please Just Do It if you disagree.
Webservice
Webservice integration in Launchpad
- This is done by annotating our Zope interfaces with lazr.restful annotations.
To understand the annotations, start with the annotations in launchpad/lib/lp/bugs/interfaces/, and also see the developer wiki documentation.
The implementation of the web service layer is in lib/canonical/launchpad/webapp/servers.py. See especially WebServicePublication, whose getPrincipal method implements OAuth authorization.
Most exceptions on startup are caused by annotation problems. Try finding the annotations you're using in src/lazr/restful/declarations.py and adding breakpoints.
- If you need information, ask leonardr, benji, or gary.
Testing the webservice in Launchpad
The launchpadlib_for method gives you a launchpadlib client object, which you can use in a unit test to browse your newly created factory resources.
- If you need information, ask leonardr, benji, or gary.
lazr.restful
- Library to publish Zope interfaces as web service resources.
- Used throughout Canonical, not just in Launchpad.
To understand it, look at the example service in src/lazr/restful/example/base
- If you need information, ask leonardr, benji, or gary.
lazr.restfulclient
- Client-side library to consume WADL and JSON generated by lazr.restful.
- Most of the functionality you think of as belonging to launchpadlib, actually belongs to lazr.restfulclient.
To understand it, look at the Resource class in src/lazr/restfulclient/resource.py. This class binds the JSON representation of a resource to its WADL representation and provides a Python interface to the data.
- If you need information, ask leonardr, benji, or gary.
launchpadlib
- Client-side library to consume the Launchpad web service.
- Includes only Launchpad-specific code.
To understand how to use it, see the Launchpad help wiki.
Most of the code has to do with OAuth tokens. Look at AuthorizeRequestTokenWithBrowser to see how they're authorized, and look at CredentialStore and its subclasses to see how they're stored locally. Both classes are in src/launchpadlib/credentials.py. (Turning an OAuth token into a signed request is handled in lazr.restfulclient.)
- If you need information, ask leonardr, benji, or gary.
wadllib
- Client-side library to parse WADL files into Python data structures.
- Changes very slowly.
The best way to understand it is to read the WADL standard, and to see how lazr.restfulclient uses wadllib.
- If you need information, ask leonardr, benji, or gary.
Building and integrating dependencies
Buildout
See https://dev.launchpad.net/Hacking#Buildout . It gives all pointers to docs.
- Gary, maybe Benji or Francis
meta-lp-debs and Launchpad's PPA
See https://dev.launchpad.net/Hacking#Changing%20the%20launchpad%20dependency%20debs
In particular, see https://dev.launchpad.net/LaunchpadPpa . It describes meta-lp-debs and the procedure for updating it.
- maxb, Gary, ...
Sourcecode
- Necessary for shipit; used by others
- utilities/update-sourcecode in tree is the starting point for the code.
- utilities/sourcedeps.conf in tree is the configuration
- Gary, Curtis (?)
Testing and landing
ec2 [test/land/...] script
- utilities/ec2 in tree
- Jono, Gary, Maris, many others
Buildbot
configuration is in its own project: https://launchpad.net/lpbuildbot
- Maris, Gary
Tarmac and tarmac-land
Post-release QA and analysis
OOPS tools
Code is here: https://edge.launchpad.net/oops-tools/
- Installation instruction is in the README.txt file in the root of branch.
- Diogo, Gary, Ursula, Maris
Performance reports
- Stuart, Francis
DB reports
- Stuart
qa tagger and qa report
Code is here: https://edge.launchpad.net/qa-tagger
- Installation instruction in the README.txt file in the branch.
More info about the process: https://dev.launchpad.net/QAProcessContinuousRollouts
- Ursula, Diogo, Maris
++profile++
- Gary, Maris, Benji
Authentication and identity
openid
- Benji, Stuart, Salgado, Curtis, Gary
session db
- Stuart
other operational conveniences and tools
haproxy pages (+opstats and the new one flacoste did)
Feature flags
- Martin, Robert
Jobs system
(none of us did anything on this, but the page is in the Foundations section...)
- Aaron, Julian, ...
Javascript/Windmill
(?)
- Maris, Paul, Deryck
Memcached
- Stuart, Gary
Chameleon
- Gary, Sidnei
