• Immutable Page
• Info
• Attachments
• More Actions: Raw Text Print View Render as Docbook Delete Cache ------------------------ Check Spelling Like Pages Local Site Map ------------------------ Rename Page Delete Page ------------------------ Subscribe User ------------------------ Remove Spam Revert to this revision Package Pages Sync Pages ------------------------ Load Save SlideShow

A user's guide to QA for Continuous Rollouts

This page is a quick guide to landing your code using the new QA flags and process. You need to follow this process to keep our new system of continuous rollouts flowing.

The Really Important Part

Every branch that lands in our merge workflow has to have some sort of QA status associated with it, like 'needs testing' or 'does not require QA'. When you land a branch, you need to pass some switches into the bzr lp-land and ec2 land commands to tell them what the QA for your branch should look like. The land commands will set the commit message for your branch so that our automated QA tools know what to do next.

As a developer you have the following responsibilities:

1. It is your responsibility to set the QA status of your branches properly so that we do not deploy untested code.
2. It is your responsibility to QA your code in a timely manner so that your QA does not block the continuous rollout of everyone's code.

Robert is regularly using cherry picks to roll out every revision of 'stable' that has passed QA. *It is really important that you set the QA status correctly because your code will soon be in production!*

Changes to 'db-stable' are not part of this new process. Rollout of the db-stable branch will take place as normal at the end of the cycle.

Landing your branch

There are a few different situations you will meet in your day-to-day development cycle and we have a set of QA tags that are appropriate to each. Below is a list of the most common situations and how you set the QA status appropriately.

The branch fixes one or more bugs, and it needs QA

What you do:

1. Link the bug(s) to the branch with --fixes or in the web UI.

2. Land as usual using ec2 land or bzr lp-land

3. QA as normal (tag the bug as qa-ok, qa-bad, or qa-untestable).

The bug(s) will be tagged qa-needstesting. After doing your QA you must mark the bug(s) as qa-ok, qa-bad, or qa-untestable. More below about what each QA tag does.

The branch does NOT need QA

What you do:

1. If there is a bug for this branch, link it; if not, that is fine too.

2. Use ec2 land --no-qa or bzr lp-land --no-qa.

The bug will be tagged as 'qa-untestable', and the branch will be deployed. The bug will be closed.

The branch is one in a series, and it needs QA

What you do:

1. Link the bug.

2. Use ec2 land --incremental or bzr lp-land --incremental to land the branch.

3. QA as normal.

The bug will remain open.

The branch is one in a series, but does NOT need QA

What you do:

1. Link the bug.

2. Use ec2 land --no-qa --incremental or bzr lp-land --no-qa --incremental.

The branch will be deployed. The bug will remain open.

The qa-ok, qa-untestable, and qa-bad tags

To finish normal QA for a branch you must choose one of three tags: qa-ok, qa-untestable, or qa-bad. If you use qa-ok or qa-untestable, then your branch will be deployed. If you use the qa-bad tag, then you have to back your code out of the mainline.

Bugs marked as qa-bad or qa-needstesting block the rollout of every revision that comes after them!

Backing out a revision

What you do:

1. Back out the code by reversing a patch or 'bzr merge -r Y..X'. For instance if rev 101 needs to be reverted you would reverse it with 'bzr merge -r 101..100'.
2. Commit your change, linking to each bug the original branch was linked. (NOTE: using multiple --fixes=lp:bug_number on the commit does not seem to work.)
3. Create a merge proposal and get it reviewed, or review it yourself.

4. Land the code using ec2 land --rollback=BADREVNO --testfix or bzr lp-land --rollback=BADREVNO --testfix. In our example, ec2 land --rollback=101 --testfix or bzr lp-land --rollback=101 --testfix. --testfix is not required for the backout but you're probably doing it because buildbot is in testfix mode.

The bug will be tagged as bad-commit-*, all the qa-* tags will be removed, and all revisions up to and including the rollback revision will be deployed.

You may also want to use this procedure to back out untested code that is blocking too many other user's deployments. This will become more of an issue when we start doing daily rollouts.

Has my branch been deployed yet?

We have created HTML reports that shows the QA status of all of the revisions on stable and db-stable:

• https://devpad.canonical.com/~lpqateam/qa_reports/deployment-stable.html

• https://devpad.canonical.com/~lpqateam/qa_reports/deployment-db-stable.html

The reports show whose change is blocking rollouts, links to the blocking Bug, and a list of all undeployed revisions that have passed QA. The reports update every 20 minutes. If you do not see your revision on the list, then it it is either blocked by someone else, or it has already been deployed.

Who to contact if you have questions

If you have any questions about the new process please feel free to contact anyone on the Launchpad Foundations team (Ursinha, matsubara, or mars).