||This page is about how Launchpad's Code Hosting component. Please [[Help|ask for help]] right away if you have any questions. || = Launchpad Code Hosting = Launchpad hosts source code using [[https://git-scm.com/|Git]] and [[https://bazaar.canonical.com/|Bazaar]] branches. However dealing with code in Launchpad covers many more areas than just hosting branches. Code for the web application is only part of the Launchpad codehosting system. The major sub-systems are: * The `git` and `bzr`/`brz` clients (neither of which is part of Launchpad, but their behaviours are important to us) * Connectivity to Launchpad (git, git+ssh, and https for Git; sftp and bzr+ssh for Bazaar) * Hosting infrastructure * The underlying object model * The web application * Email processing * Code imports (from CVS, Subversion, git and Mercurial) * Branch source code browser ([[https://git.zx2c4.com/cgit/|cgit]] for Git; [[/Loggerhead|loggerhead]] for Bazaar) * Source package recipes (`git-build-recipe`/`bzr-builder` integration with [[Soyuz|Soyuz]]) Each of these subsystems also have multiple moving parts and some have other asynchronous jobs associated with them. The [[attachment:codehosting.png|codehosting overview diagram]] summarizes how some of these systems interact. You can [[/HowToUseCodehostingLocally|run the codehosting system locally]]. We no longer put significant effort into bzr hosting beyond making sure it remains functional. The future of bzr is [[https://www.breezy-vcs.org/|Breezy]]; Launchpad has migrated to it for code hosting, and will migrate to it for code imports in due course. == The bzr/brz client == This is what users install on their systems to use Bazaar. The bzr application is also installed on the server side for Launchpad to use to access the information in the Bazaar branches. '''Parts [and responsibilities] ''' * bzr/brz client [maintained by the [[https://www.breezy-vcs.org/|Breezy]] community] * bzr.plugins.launchpad [shared with the [[https://www.breezy-vcs.org/|Breezy]] team] * server side of lp name resolution == Connectivity to Launchpad == === Git === `turnip` implements several frontends for different transports. '''Parts [and responsibilities]''' * haproxy configuration (encoded in the [[Mojo spec|https://git.launchpad.net/launchpad-mojo-specs/tree/mojo-lp-git/services]]) * frontends (`turnip-pack-frontend-git`, `turnip-pack-frontend-ssh`, `turnip-pack-frontend-http`) * `turnip-pack-virt` for path virtualisation (translating from the URL path namespace to physical storage on disk) * `turnip-pack-backend` for backend storage, calling out to `git upload-pack` and `git receive-pack` to do most of the hard work * `nfs-ganesha` to allow horizontally scaling `turnip` * git hooks which call out to Launchpad, for e.g. push notifications or to check per-ref permissions * Launchpad's XML-RPC interfaces in `lp.code.xmlrpc.git` === Bazaar === Connecting to the code hosting system from the outside world is done either through SSH using SFTP or the bzr+ssh protocol, or through HTTP. Apache handles the HTTP routing using a number of mod-rewrite rules. '''Parts [and responsibilities] ''' * HTTP apache configuration [shared with LOSAs] * branch location rewrite script (called by mod-rewrite rule) * ssh server * authentication * sftp implementation * smart server launching * smart server * lp-serve plugin * bzr's smart server [shared with [[https://www.breezy-vcs.org/|Breezy]] team] * codehosting transport implementations * codehosting interfaces on xmlrpc.lp.internal == Hosting infrastructure == This really groups together the bits around what happens once we have a branch that a user has pushed to us, and the associated jobs that get kicked off. '''Parts [and responsibilities] ''' * puller (Bazaar only) * scanner * revision email jobs * scalability concerns - efficient storage and responsiveness [shared with LOSAs] * stacking * reclamation cleanup jobs * backup support scripts == The web application == Code that is executed as part of the Launchpad web application. The core [[/Concepts|concepts]] are documented on the [[/Concepts|Code/Concepts]] page. '''Major features''' * general information * listings for various registry objects - people, teams, projects, distros, packages * privacy * code reviews * merge management * branch gardening * linking to other launchpad apps (bugs, blueprints, rosetta (soyuz soon)) * translation jobs [rosetta team primarily responsible for this] * asynchronous generation of review diffs == Email processing == There is also the processing of incoming email to the code.launchpad.net domain. Currently there are two main things are triggered with incoming email: * creating a new merge proposal (an possibly a source branch) from a merge directive * processing email comments (with possible commands) == Code imports == Launchpad provides a way for users to get access to source code from other systems as Bazaar branches (from CVS, Subversion, git and Mercurial). This also supports mirroring Git repositories from elsewhere into Git repositories in Launchpad. * code import jobs * integration of import tools * cscvs for CVS (and legacy Subversion imports) * bzr-svn and subvertpy for all new Subversion imports * bzr-git and dulwich for git * bzr-hg for mercurial imports == Git repository source code browser (cgit) == Launchpad uses [[https://git.zx2c4.com/cgit/|cgit]] to provide a web view of the repository contents. We use an unmodified package of `cgit`; Launchpad's customizations are in [[https://git.launchpad.net/turnip/tree/turnip/pack/http.py|turnip.pack.http]]. == Bazaar branch source code browser (loggerhead) == Launchpad uses [[https://launchpad.net/loggerhead|loggerhead]] to provide a web view of the branch contents. We try not to have any Launchpad specific code in loggerhead itself, but instead keep that in the [[https://code.launchpad.net/~launchpad-pqm/loggerhead/devel|lp:~launchpad-pqm/loggerhead/devel]] branch. * loggerhead itself - community project but with major contributions from Canonical See [[/Loggerhead|Loggerhead for Launchpad developers]] for details on how to land changes to Launchpad loggerhead. == Source package recipes == See [[https://help.launchpad.net/Packaging/SourceBuilds|the help wiki]] for more details on what recipes are.