Parallel EC2 Command

Our ec2 command is used for three main tasks.

• Testing (ec2 test)
• Testing and landing (ec2 land)
• Providing a demo machine (ec2 demo)

Other commands (help, images, kill, list, update-image) are helpers or administrative.

The ec2 command was developed before juju, and before parallel testing. This proposal is to update and rethink our ec2 command to take advantage of these two technologies.

ec2 demo could be replaced simply with a Launchpad developer charm that is set up for juju expose. Updating the image could also be part of that story, as mentioned in the linked LEP's integration discussion.

Given a Launchpad developer charm that supports parallel testing, ec2 test and ec2 land can be drastically simpler. Specifically, the ec2 script could simply drive the charm, and then merge the desired branch, and then arrange to run the tests, send the results by email, and send a PQM submission if requested and appropriated.

This would give us parallel testing, it would push a lot of the EC2 code into Juju and out of Launchpad developer maintenance, and it would be a part of a more flexible environment that integrated with Canonical's Juju push.

Contact: Gary Poster
On Launchpad: XXX Link to a blueprint, milestone or (best) a bug tag search across launchpad-project

Rationale

We have spent a fairly large effort to get Launchpad's tests to run in parallel. This can make our automated testing in well under an hour, but it is still typically preceded by an "ec2 land" run of a few hours. To be able to code something in the morning and see it deployed in the evening, we need ec2 land to be fast also.

We have active experience in parallel testing right now. If we want to take best advantage of that experience, we make this effort now.

Stakeholders

Launchpad developers, and especially Launchpad team leads and the IS TA.

User stories

Developer Lands Branch If Tests Pass

As a Launchpad Developer
I want to be able to get my tests run and my branch automatically landed in less than an hour
so that (with LEP/ParallelTesting) I can code a branch in the morning and have it deployed in the afternoon, in an agile coding cycle.

Developer Maintains Less Code

As a Launchpad Developer
I want to reduce the amount of code in the ec2 command
so that I can spend more time on valuable tasks, rather than maintaining code that has a significant overlap elsewhere.

Constraints and Requirements

Must

• Have a single command that fires off an ec2 land.

• Have a single command that fires off an ec2 test without a landing. I almost made this a nice to have, because a Juju charm that starts up quickly and runs tests relatively quickly might make this less valuable. However, the ability to have the machine automatically shut down after the tests is very valuable.

• Have good test coverage.
• Be less code than what we have now.

Nice to have

• ec2 list
• ec2 kill

Must not

• Leave machines running unexpectedly

Out of scope

• ec2 demo and ec2 update-images should be handled by the juju charm

Subfeatures

LEP/LaunchpadJujuCharmForDevs, which in its proposed implementation depends on LEP/LaunchpadSetupScripts

Success

How will we know when we are done?

When we can test our branches reliably in under an hour and have them automatically submitted for merging.

How will we measure how well we have done?

• Launchpad team determines that we are willing to remove the old command within two weeks of release of the tool.

Thoughts?

Put everything else here. Better out than in.