Windmill for JS Integration Tests

For integration testing that covers JS workflows, our tool of choice is Windmill.

Contents

Windmill for JS Integration Tests
1. Using Windmill to Test Launchpad
Test Writing Tips
External resources
Troubleshooting

Using Windmill to Test Launchpad

Setting up Windmill

Windmill is included in the Launchpad source tree. Its dependencies are installed by the launchpad-developer-dependencies ubuntu package.

Running the Launchpad Windmill test suite

To run all Windmill tests:

```
    bin/test --layer=WindmillLayer
```

To run all Windmill tests for a single application (e.g. the Bugs application):

```
    bin/test --layer=BugsWindmillLayer
```

To run a single Windmill test (e.g. test_bug_commenting() in the Bugs application):

    bin/test --layer=BugsWindmillLayer --test=test_bug_commenting

The Windmill tests are integrated into our normal test suite, and the tests has a layered named FooWindmillLayer, where Foo is the name of the application (e.g. BugsWindmillLayer. The Windmill layers are excluded by default, so when running a Windmill tests you always have to specify the --layer option, otherwise the test runner won't find it.

How are the tests organized

Tests written in python are placed in lib/lp/<application>/windmill/tests.

Tests using the JS API are in lib/canonical/launchpad/windmill/jstests.

Test Writing Tips

Test the Happy Path, and one or two error paths
Edge cases are best pushed into the unit tests
Setup and teardown are expensive, so there will be a trend towards testing more per test
Prefer element ids to XPath for locating page elements
- XPath makes your tests brittle and dependent on page structure
Try to pull test actions out into their own functions, creating a Python Windmill testing DSL
Use the PageObject pattern to encapsulate the actions available through specific pages
Error: [Exception... "Component returned failure code: 0x80004005 (NS_ERROR_FAILURE) [nsIXMLHttpRequest.setRequestHeader]" nsresult: "0x80004005 (NS_ERROR_FAILURE)" -- you likely started your windmill shell using the wrong hostname, ie. launchpad.dev instead of bugs.launchpad.dev

When using open(), make sure to use waits.forPageLoad() before any waits.forElement() (unless you are absolutely sure that you can't be on a page that already contains that element).

When using the lazr-js slide-in/slide-out effects, use the following assertions to test whether the element is visible or not.

    # Test for visibility.
    client.asserts.assertProperty(
        id=u'search-results',
        validator='className|lazr-opened')

    # Test for invisibility.
    client.asserts.assertProperty(
        id=u'search-results',
        validator='className|lazr-closed')

When hiding or revealing elements using POJS, use the unseen class to control this behavior. It's easy to test for too. E.g.

    // Hide this element.
    element1.addClass('unseen');
    // Reveal this element.
    element2.removeClass('unseen');

    # Test for visibility.
    client.asserts.assertNotProperty(
        id=u'search-results',
        validator='className|unseen')

    # Test for invisibility.
    client.asserts.assertProperty(
        id=u'search-results',
        validator='className|unseen')

External resources

Windmill Book
Full-stack webapp testing with Selenium and Rails - has a number of tips that apply to any automated testing framework

Troubleshooting

If you are running on a slow computer or inside a VM, you may randomly trigger timeout errors.
In the Windmill IDE window, you can test windmill's ability to find objects by going to the Tools menu (not the Firefox Tools menu) and clicking on "Firebug Lite". In Firebug Lite, you will be able to call the lookupNode() function with the same locator parameters that windmill uses. For example:
- lookupNode( {classname: 'yui-picker'} )
- lookupNode( {xpath: '//table[@id=foo]//button'} )
- Locator Docs
Run make schema if you're having an issue that ends in something like:

  File "/usr/lib/python2.4/urllib.py", line 82, in urlopen
    return opener.open(url)
  File "/usr/lib/python2.4/urllib.py", line 190, in open
    return getattr(self, name)(url)
  File "/usr/lib/python2.4/urllib.py", line 316, in open_http
    errcode, errmsg, headers = h.getreply()
  File "/usr/lib/python2.4/httplib.py", line 1137, in getreply
    response = self._conn.getresponse()
  File "/usr/lib/python2.4/httplib.py", line 866, in getresponse
    response.begin()
  File "/usr/lib/python2.4/httplib.py", line 336, in begin
    version, status, reason = self._read_status()
  File "/usr/lib/python2.4/httplib.py", line 294, in _read_status
    line = self.fp.readline()
  File "/usr/lib/python2.4/socket.py", line 317, in readline
    data = recv(1)
IOError: [Errno socket error] (104, 'Connection reset by peer')

CategoryJavaScript CategoryTesting

launchpad development

Windmill