Diff for "Testing"

Not logged in - Log In / Register

Differences between revisions 29 and 30
Revision 29 as of 2010-07-26 20:36:48
Size: 5036
Editor: gary
Comment:
Revision 30 as of 2010-08-17 15:56:08
Size: 5031
Editor: bac
Comment:
Deletions are marked like this. Additions are marked like this.
Line 74: Line 74:
You can load the file from ec2 test runs. To do so, you need to uncompress the file, so {{{gunzip < attachment.log.gz | testr load}}} (where {{{attachment.log.gz}}} is the log file attached to the ec2 email, whatever the name) will do the trick. After a successful load, you will get a line like {{{id: 0 tests: 8956 failures: 5 skips: 23}}}. The word on the street is that you don't have to worry about the "skips" part of that line. Now you can use {{{testr run --failing}}} to re-run the tests. You can load the file from ec2 test runs. To do so, you need to uncompress the file, so

{{{gunzip < attachment.log.gz | testr load}}}

(where {{{attachment.log.gz}}} is the log file attached to the ec2 email, whatever the name) will do the trick. After a successful load, you will get a line like {{{id: 0 tests: 8956 failures: 5 skips: 23}}}. The word on the street is that you don't have to worry about the "skips" part of that line. To re-run the tests:

{{{testr run --failing}}}

This page is an introduction to writing and running test. All pages dedicated to testing can be found in CategoryTesting.

Writing tests

For each part of an application there are 2 level of testing:

  • unit testing
    • comprehensive testing, including cornerstone cases
    • written in python files (try to avoid doctest)
  • smoke/functional/integration testing
    • testing normal use cases
    • written using doctest format

Model

  1. Unit testing
    • Files locate in lib/lp/<application>/scripts/tests

  2. Integration testing
    • Files locate in lib/lp/<application>/doc

View

  1. Unit/integration testing
    • Files locate in lib/lp/<application>/browser/tests

    • More details: ViewTests

  2. Smoke testing
    • Files locate in lib/lp/<application>/stories

    • More details: PageTests

Javascript

  1. Unit testing
  2. Smoke testing
    • Files locate in lip/lp/<application>/windmill (if written in Python) and lib/canonical/launchpad/windmill/jstests (if written in JS)

    • More details: Windmill

Scripts

  • Files locate in lib/lp/<application>/scripts/tests

API

  • Files located in lib/lp/<application>/stories/webservices

  • More details: TestingWebServices

Testing Your Launchpad Changes

This page is very much still in progress.

You might also be interested in Debugging Launchpad.

General usage

The normal pattern for testing your changes is to run the tests you think will be affected locally, and then (possibly just before submission) run all the tests in the cloud before landing your changes on trunk.

Iterating with testrepository

apt-get install testrepository
#cd $yourlaunchpaddevdir
testr init

Don't worry about creating a .testr.conf file; the defaults created for you works fine.

To run all the tests: testr run

To run an individual test using the -t PATTERN option: testr run -- -t foo

To see the current known failures: testr failing

To run just the known failing tests: testr run --failing

You can load the file from ec2 test runs. To do so, you need to uncompress the file, so

gunzip < attachment.log.gz | testr load

(where attachment.log.gz is the log file attached to the ec2 email, whatever the name) will do the trick. After a successful load, you will get a line like id: 0 tests: 8956 failures: 5 skips: 23. The word on the street is that you don't have to worry about the "skips" part of that line. To re-run the tests:

testr run --failing

testr is moving and bug reports and patches are accepted :).

Running old skool

To run the tests, you run the ./bin/test script, which is produced by make build.

You can see all the options you have by running ./bin/test --help.

Usually you will run ./bin/test -vvct PATTERN where PATTERN is a regular expression that is used to filter the tests that will be run.

You can use '!PATTERN' to match all expression not matching that patter. Ex, to run all test, except the one at Windmill layer you can use:

./bin/test -vvc --layer '!Windmill'

Speed up the tests

Since Librarian and MemCache does not change often and they take a long time to be started and shutdown, ./bin/test can leave them started by using this command:

LP_PERSISTENT_TEST_SERVICES=1 ./bin/test PATTERN 

You can kill them using  ./bin/kill-test-services 

When running tests written in python files, and you only want to test a file, you can speed up the test by specifying the full path to the python file:

LP_PERSISTENT_TEST_SERVICES=1 ./bin/test PATH/TO/PYTHON/TEST/FILE.py

See this mail for more.

Headless Windmill tests

  • In case you are running windmill test on a machine without X server, you can use xvfb-run to run WindmillTests:

xvfb-run -s '-screen 0 1024x768x24' bin/test YOUR_TEST_ARGUMENTS

Testing Launchpad Translations

  • Rosetta admin user - translations-deity@example.com

  • Pagetest browser setup for Rosetta Administrators rosetta_admin_browser = setupRosettaExpertBrowser()

  • Pagetest browser setup for Ubuntu Translation Administrators ubuntu_admin_browser = setupDTCBrowser()

Performance and stress tests

== Populate the db ==

To add object into the database you can use:

& env LP_DBNAME="launchpad_dev" make iharness
from canonical.lp import initZopeless
zl = initZopeless()
#
# use the factory here
#
zl.commit()


CategoryTesting

Testing (last edited 2020-07-16 16:51:31 by doismellburning)