NOTE: there are now public EC2 images available for anyone to use; see Michael Hudson's 28 Sep 2009 mail about it. That information should probably be incorporated better into this page!
In order to make developer testing easier, we have created a way to automate testing instance on EC2. You can use the tools for running the test suite, or LandingChanges to the Launchpad code. To use the tool, you have to do a few initial one-time setup steps, and then running tests on EC2 should be easy.
There is one big warning. We are trying to be sure that the EC2 instance shuts down, but there are rare but real circumstances that we simply can't handle.
Developers are responsible for verifying that their EC2 test instances shut down.
Where can I find ec2test?
In lp:lp-dev-utils. The command name is ec2.
You'll need to have the python-boto, python-paramiko, python-tz and bzr-pqm packages installed. (You'll need bzr-pqm r89 or later.)
You'll probably want to put it in your path, e.g. by symlinking: ln -s $(pwd)/ec2 ~/bin/ec2
Then, follow the steps for setting up EC2.
What does it do?
Here's an example usage of running the test suite. Let's say you want to test my-great-branch.
$ cd ~/canonical/lp-branches/my-great-branch $ ec2 test
This will check that you have no uncommitted working tree changes and no unpublished locally-committed changes, a la pqm-submit. It will then start an instance, install the branch, and run the tests. Once the tests start, it disconnects, and the tests run headlessly on the ec2 box. The output will show you a link to a webpage near the point that ec2test exits. If you go to it, it will show you all branches used, and it will give you a full-length or summary view of the output so you can follow along (only from ec2test's originating IP, unless you tweak the security through, say, ElasticFox). At the end, a summary email will be sent to the address you configured for bzr. It should cost you about a few dollars, which you can expense.
Here's a variation.
$ ec2 test --attached -s '[r=kermit] Fix the frobnob.'
Then it will start an EC2 instance and run the tests for you. You can see the output on your terminal. Once the test is completed, if the test run was successful, ec2test will submit your branch to PQM with the message supplied. You will get an email with information on the branches used, summary results, and confirmation that the PQM message was sent (or not) if you requested one.
Use ec2 test --help for more information.
If you want ec2 to automatically land your changes after the test suite has run, you should try the ec2 land command.
Using EC2 to demo branches
If you run ec2 demo, a copy of Launchpad will be run on the EC2 instance, which can be used to demo your branch, possibly to another developer for UI reviews. Anyone wanting to see the demo will need to hack their /etc/hosts to point the launchpad.dev etc domain names at the EC2 instance. You'll need to pass the IP of anyone you want to see the instance to --demo-networks option.
See ec2 demo --help for more.
Setting Up EC2 and ec2test
- Get an Amazon developer account, and sign up for EC2.
Go to http://aws.amazon.com/ and click to "Sign Up Now" for an AWS account. Canonicalers: Consider creating a new Amazon account with your Canonical email address. You will need to keep track of the costs for reimbursement, and this might make it easier.
- Once you are logged into your account, you need to sign up for EC2. Instructions for how to do this have changed over time because Amazon changes their website, so the best advice is to hunt for it. After signing up for EC2, it may say it wants you to upload an x.509 cert. Ignore that--you don't need it for ec2test.
- You must enter a valid credit card before you will be able to use EC2. Do that under 'Payment Method'.
Make sure you're signed up for EC2 by going to 'Your Account'->'Account Activity' and verifying EC2 is listed.
- Get the necessary account information in the right places.
While logged into your AWS account, find the "Access Credentials" link (via "Account" -> "Security Credentials".
On the page, you'll see "Access Key Id:" and "Secret Access Key:". Make a ~/.ec2 directory in your dev box with a file called aws_id. In the first line, put your access key id. In the second line, put your secret access key id.
Make sure that your ~/.bazaar/bazaar.conf is configured to send mail using a publicly accessible server. Note that this works like usual bzr configuration, so settings in locations.conf specific to the location you are working with will override the global smtp settings. Therefore, if you have set the global smtp values and you are still getting an error about the smtp settings, check your locations.conf. However, ec2 ignores credentials in authentication.conf. This is bug #949104.
- Get the tools.
Add the ~launchpad PPA ("deb http://ppa.launchpad.net/launchpad/ppa/ubuntu lucid main") to a launchpad-ppa.list into /etc/apt/sources.list.d.
Perform sudo aptitude update and sudo aptitude install python-boto or equivalent. You should get version 1.5c of python-boto (about which aptitude will warn you about the lack of key), and Hardy users will get an upgraded version of paramiko.
(Recommended) Install ElasticFox (http://developer.amazonwebservices.com/connect/entry.jspa?externalID=609). If you do, it will give you an ability to easily check to make sure that your AWS EC2 instances are shut down and not leaking money away! Otherwise you'll need to use another tool, like the AWS command-line tools.
Note that dead instances will appear to hang around for up to an hour in ElasticFox. Don't be alarmed. Also note that the views often do not refresh without a click on the refresh buttons above each pane. Alternatively you can use Amazon's AWS console.
Now you are done with the necessary one time steps. You should be good to go to run tests.
If you want to review the output and your terminal scrollback is not sufficient for whatever reason, do the standard UNIXy things of sending the output to a file and so on. Alternatively, you can look at the first line of output to see the DNS for your instance. In your browser you can go to http://THE_PUBLIC_DNS/current_test.log (e.g., http://ec2-67-202-29-53.compute-1.amazonaws.com/current_test.log) to see how it is going, but this will be gone once the instance is finished (though see the -p hint below).
The -p option is useful sometimes. After the test run, it will dump you to a Python console. If you are trying to debug the ec2test.py script itself that might be nice. But even if you are just trying to debug a test, it is nice because it keeps the EC2 instance alive until you close the console (CTRL-D), and gives you hints on how to look at the test output in your browser, and how to ssh to the instance. Make sure you close the console when you are done, because you are leaking money (US $.40 an hour or part thereof as of this writing).
* For information about updating the AMI that is used by ec2test, please read EC2Test/Image.