EC2Test

Not logged in - Log In / Register

Revision 6 as of 2010-02-01 13:26:18

Clear message

NOTE: there are now public EC2 images available for anyone to use; see Michael Hudson's 28 Sep 2009 mail about it. That information should probably be incorporated better into this page!

EC2 Testing

In order to make developer testing easier, we have created a way to automate testing instance on EC2. To use it, you have to do a few initial one-time steps, and then running tests on EC2 should be easy.

There is one big warning. We are trying to be sure that the EC2 instance shuts down, but there are rare but real circumstances that we simply can't handle.

Developers are responsible for verifying that their EC2 test instances shut down.

Where can I find ec2test?

In the launchpad tree, as utilties/ec2 test.

Unlike other launchpad scripts it runs with the system Python and the system-wide Python libraries (at least until Launchpad is using Python 2.5). You'll need to have python-boto and python-paramiko installed.

If you want more detailed, step-by-step instructions, with EC2 account setup and software setup, see the "One-Time Steps" section below. If you encounter problems, please review that section before reporting errors.

What does it do?

Here's an example usage. Let's say you want to test my-great-branch.

$ cd ~/canonical/lp-branches/my-great-branch
$ ./utilities/ec2 test

This will check that you have no uncommitted working tree changes and no unpublished locally-committed changes, a la pqm-submit. Then it will start an EC2 instance and run the tests for you. You can see the output on your terminal. At the end, a summary email will be sent to the address you configured for bzr. It should cost you about a few dollars, which you can expense.

Here's a variation.

$ ./utilities/ec2 test --headless -s '[r=kermit] Fix the frobnob.'

This will run ec2test tied to your machine up through getting all source code. Once the tests start, it disconnects, and the tests run headlessly on the ec2 box. The output will show you a link to a webpage near the point that ec2test exits. If you go to it, it will show you all branches used, and it will give you a full-length or summary view of the output so you can follow along (only from ec2test's originating IP, unless you tweak the security through, say, ElasticFox). Once the test is completed, if the test run was successful, ec2test will submit your branch to PQM with the message supplied. You will get an email with information on the branches used, summary results, and confirmation that the PQM message was sent (or not) if you requested one.

Use ./utilities/ec2 test --help for more information.

Using EC2 to demo branches

If you run ./utilities ec2 demo, a copy of Launchpad will be run on the EC2 instance, which can be used to demo your branch, possibly to another developer for UI reviews. Anyone wanting to see the demo will need to hack their /etc/hosts to point the launchpad.dev etc domain names at the EC2 instance. You'll need to pass the IP of anyone you want to see the instance to --demo-networks option.

See ./utilities/ec2 demo --help for more.

One-Time Steps

  1. Get an Amazon developer account, and sign up for EC2.
    • Go to http://aws.amazon.com/ and click to "Sign Up Now" for an AWS account. Canonicalers: Consider creating a new Amazon account with your Canonical email address. You will need to keep track of the costs for reimbursement, and this might make it easier.

    • Once you are logged into your account, you need to sign up for EC2. Instructions for how to do this have changed over time because Amazon changes their website, so the best advice is to hunt for it. After signing up for EC2, it may say it wants you to upload an x.509 cert. Ignore that--you don't need it for ec2test.
    • You must enter a valid credit card before you will be able to use EC2. Do that under 'Payment Method'.
    • Make sure you're signed up for EC2 by going to 'Your Account'->'Account Activity' and verifying EC2 is listed.

  2. Get the necessary account information in the right places.
    • While logged into your AWS account, find the "Access Identifiers" link (either on the left, or via "Your Account" -> "Access Identifiers".

    • On the page, you'll see "Your Access Key Id:" and "Your Secret Access Key:". Make a ~/.ec2 directory in your dev box with a file called aws_id. In the first line, put your access key id. In the second line, put your secret access key id.

    • Make sure that your ~/.bazaar/bazaar.conf is configured to send mail using a publicly accessible server. Note that this works like usual bzr configuration, so settings in locations.conf specific to the location you are working with will override the global smtp settings. Therefore, if you have set the global smtp values and you are still getting an error about the smtp settings, check your locations.conf.

  3. Get the tools.
    • Add the ~launchpad PPA to a launchpad-ppa.list into /etc/apt/sources.list.d.

    • Perform sudo aptitude update and sudo aptitude install python-boto or equivalent. You should get version 1.5c of python-boto (about which aptitude will warn you about the lack of key), and Hardy users will get an upgraded version of paramiko.

  4. (Recommended) Install ElasticFox (http://developer.amazonwebservices.com/connect/entry.jspa?externalID=609). If you do, it will give you an ability to easily check to make sure that your AWS EC2 instances are shut down and not leaking money away! Otherwise you'll need to use another tool, like the AWS command-line tools.
    Note that dead instances will appear to hang around for up to an hour in ElasticFox. Don't be alarmed. Also note that the views often do not refresh without a click on the refresh buttons above each pane. Alternatively you can use Amazon's AWS console.

Now you are done with the necessary one time steps. You should be good to go to run tests.

Hints

* For information about updating the AMI that is used by ec2test, please read EC2Test/Image.