launchpad development|
⇤ ← Revision 1 as of 2011-02-22 04:42:46
Size: 1891
Comment:
|
Size: 2222
Comment: draft
|
| Deletions are marked like this. | Additions are marked like this. |
| Line 1: | Line 1: |
| The purpose of this template is to help us get ReadyToCode on features or tricky bugs as quickly as possible. See also LaunchpadEnhancementProposalProcess. The bits in ''italics'' are the bits that you should fill in. '''Delete the italic bits.''' '''''Talk to the product strategist soon after cutting a first draft of this document''''' = $HEADLINE = |
= Single machine parallel testing of single branches = |
| Line 11: | Line 5: |
| '''Contact:''' ''The primary contact for this LEP. Normally the drafter or the implementer.'' <<BR>> '''On Launchpad:''' ''Link to a blueprint, milestone or (best) a bug tag search across launchpad-project'' |
'''Contact:''' RobertCollins '''On Launchpad:''' https://bugs.launchpad.net/launchpad-project/+bugs?field.tag=paralleltest |
| Line 14: | Line 8: |
| '''As a ''' $PERSON<<BR>> '''I want ''' $FEATURE<<BR>> '''so that ''' $BENEFIT<<BR>> |
'''As a ''' Technical Architect<<BR>> '''I want ''' our automated test runs to parallelise effectively within a single machine<<BR>> '''so that ''' the qa and branch promotion pipeline moves faster<<BR>> |
| Line 18: | Line 12: |
| ''Consider clarifying the feature by describing what it is not?'' ''Link this from [[LEP]]'' |
'''As a ''' Developer<<BR>> '''I want ''' ec2 runs to parallelise effectively within the ec2 instance<<BR>> '''so that ''' landings happen faster<<BR>> |
| Line 24: | Line 18: |
| ''Why are we doing this now?'' | Our test suite execution time is a crucial factor in the minimum cycle type to make a change and deploy it. Similarly our ability to recover from intermittently failing tests is defined by the time it takes the test suite to run. |
| Line 26: | Line 20: |
| ''What value does this give our users? Which users?'' | Having a dramatically faster test suite will reduce the latency for bugfixes, gathering of profile data and repetition/solving of intermittent failures. |
| Line 30: | Line 24: |
| ''Who really cares about this feature? When did you last talk to them?'' | All Launchpad developers. |
| Line 36: | Line 30: |
| ''What MUST the new behaviour provide?'' | * Fix our tests such that we can run in parallel mode no less reliably than single processing mode. (With bin/test --parallel - '''not''' bin/test -j, as -j only parallelises by layer. * Update our ec2 test environment to use the ec2 machine type that will run our parallelised test suite as rapidly as possible. * Determine the needed disk/cpu bandwidth per test process so that we can reasonably project the performance of different size machines for buildbot (or tarmac/pqm in future) * Permit developers to reliably run parallelised as well. |
| Line 40: | Line 37: |
| * testr --parallel support for Launchpad working. |
|
| Line 41: | Line 40: |
''What MUST it not do?'' |
|
| Line 46: | Line 43: |
| Changing the landing technology is out of scope: thats something the SimplifiedMergeMachinery project will evaluate and act on. |
|
| Line 48: | Line 47: |
| ''Other LaunchpadEnhancementProposal``s that form a part of this one.'' |
|
| Line 51: | Line 48: |
''What are the workflows for this feature? Even a short list can help you and others understand the scope of the change.'' ''Provide mockups for each workflow.'' '''''You do not have to get the mockups and workflows right at this point. In fact, it is better to have several alternatives, delaying deciding on the final set of workflows until the last responsible moment.''''' |
|
| Line 61: | Line 53: |
| ec2 and buildbot test runs are dramatically shorter than they are today: down to less than 50% of the current time, preferrable 15%-20% |
|
| Line 64: | Line 58: |
''Put everything else here. Better out than in.'' |
Single machine parallel testing of single branches
Short description of feature
Contact: RobertCollins On Launchpad: https://bugs.launchpad.net/launchpad-project/+bugs?field.tag=paralleltest
As a Technical Architect
I want our automated test runs to parallelise effectively within a single machine
so that the qa and branch promotion pipeline moves faster
As a Developer
I want ec2 runs to parallelise effectively within the ec2 instance
so that landings happen faster
Rationale
Our test suite execution time is a crucial factor in the minimum cycle type to make a change and deploy it. Similarly our ability to recover from intermittently failing tests is defined by the time it takes the test suite to run.
Having a dramatically faster test suite will reduce the latency for bugfixes, gathering of profile data and repetition/solving of intermittent failures.
Stakeholders
All Launchpad developers.
Constraints and Requirements
Must
Fix our tests such that we can run in parallel mode no less reliably than single processing mode. (With bin/test --parallel - not bin/test -j, as -j only parallelises by layer.
- Update our ec2 test environment to use the ec2 machine type that will run our parallelised test suite as rapidly as possible.
- Determine the needed disk/cpu bandwidth per test process so that we can reasonably project the performance of different size machines for buildbot (or tarmac/pqm in future)
- Permit developers to reliably run parallelised as well.
Nice to have
- testr --parallel support for Launchpad working.
Must not
Out of scope
Changing the landing technology is out of scope: thats something the SimplifiedMergeMachinery project will evaluate and act on.
Subfeatures
Workflows
Success
How will we know when we are done?
ec2 and buildbot test runs are dramatically shorter than they are today: down to less than 50% of the current time, preferrable 15%-20%
How will we measure how well we have done?
