Note: this is an unofficial/wip LEP.

Improve our Launchpad setup scripts

Replace rocketfuel-* to produce a new script more easily maintainable, more informative for the user, better tested, and more flexible--specifically, flexible enough to support both LXC and non-LXC development environments.

lpsetup is a Python project currently used by the Yellow Squad to create a Launchpad testing environment inside a container, that is then used as a template for ephemeral instances. Starting isolated ephemeral instances, each one containing a full Launchpad environment, is actually the preferred way to run tests in parallel. lpsetup can be improved in order to let developers create and update a full Launchpad environment, inside an LXC or just in the host. This way lpsetup can be considered a replacement of rocketfuel-* scripts.

Contact: frankban
On Launchpad: https://launchpad.net/lpsetup

Rationale

Launchpad has a lot of dependencies, and the process of setting up and building the code can be difficult and painful. The rocketfuel-* scripts do not have automated tests, take over the developer's system in many ways, have minimal interaction with the user as they are run, and are not written in Python.

However, the effort the yellow squad made to code an automated Python script for parallel tests can be reused, with small improvements, to help developers run, fix and develop Launchpad itself. It can provide a tool to get started with Launchpad with much less pain, and in a way that leaves the host relatively untouched, because almost all of the dependencies can be installed inside a Linux container (LXC) if desired.

We hope that this will help community contributors begin Launchpad development more easily. We hope that this will also give core developers an improved development experience, as well as a more maintainable set of tools.

As written above, we are doing this now because it seems to us a natural consequence of what we produced in the parallel testing project. We started creating the testing environment with setuplxc, a standalone script first developed for parallel tests. As the time passed, it became evident that a re-factor was needed to let the process become more reliable and reusable. This re-factor was completed as a slack time project, and now lpsetup has fully replaced the deprecated (and, actually, deleted) setuplxc.

Stakeholders

All Launchpad developers.
The Launchpad contributors.

As a Developer
I want to easily create and update Launchpad development environments inside LXC containers
so that I can concentrate on fixing bugs and adding features without spending time and resources on the environment set up and configuration.

As a Developer
I want to have a full test suite for the code used to create and update Launchpad instances
so that when I change it, I can be confident that it still works.

As a Developer or Contributor
I want to be informed about all the changes needed in my system to set up Launchpad instances
so that I can easily revert them, and acquire knowledge about the environment and the technologies involved.

As a Contributor
I want a tool that lets me dig into Launchpad code and run the application
so that I can quickly start helping with development and contributing, without even dirty my base system.

Constraints and Requirements

Must

• Optionally create Launchpad environments in the host machine.
• Optionally create Launchpad environments inside LXC containers.

• Replace rocketfuel-get: update the Launchpad environment.

• Replace rocketfuel-branch: create a new branch of devel.

• Warn about all the changes made by the program to the system, giving the user the chance to quit the process.
• Continue supporting our parallel testing story, including options for including tweaks, helpers, generated scripts, and a non-interactive execution.
• Add banners to all created and modified files in the system.
• Store the initial configuration in a file, so that a saved state can be used when other commands are invoked.

Nice to have

• A way to store different configurations for different environments (e.g. adding a --remember option).
• A full integration test (e.g. using juju) supporting different options and use cases.
• Update and branch from the host, even if the container is not running.

• A bzr-like UI (i.e. lp-setup subcommand [options])

Must not

• Change the host environment without warning the user and without giving the user a chance to opt-out.
• Create or modify a file without specifying the creation/modification source and date-time.

Out of scope

• Replace scripts under utilities/ other than rocketfuel-*.

• Support all the customized environments that can possibly be used by Launchpad developers (i.e. lightweight checkouts).

• Create Launchpad environments in ec2 instances (we have the ec2 scripts, juju, etc.).

• Create Launchpad environments in systems != current Ubuntu release or current production release.

Success

How will we know when we are done?

Developers can successfully use the project to create their development environment.
lpsetup is accepted as part of the Launchpad project, ready to be maintained by the Launchpad team.
The removal of rocketfuel-* from the tree is accepted by the Launchpad team.

How will we measure how well we have done?

One month without critical bugs after the first release.
Feedback from developers and contributors.
When lpsetup is used as part of the parallel tests process, being able to reliably start test runs using buildbot is a good evidence that everything is set up well.

Thoughts?