launchpad development
Results Tracker - Implementation
The implementation of the Results Tracker is mostly concerned with storing and retrieving data, as opposed to exposing a web interface. So, the implementation emphasises sequence diagrams for interacting with the data and database schemas for representing the data internally.
Sequence Diagrams
This feature is concerned with interacting with the data which is expressed in the form of sequence diagrams defined below.
Submission Diagram
A submission consists of a session during which data is streamed from the client to the Results Tracker.
As researched in Streaming Test Results, the interface should make it possible to submit results from the client in constant memory.
Report Diagram
The report diagram consists of the RESTful interface which will be exposed by the Results Tracker.
- Assuming the Results Tracker becomes a core feature of Launchpad, it should be completely accessible through launchpadlib.
Database Schemas
This feature is also concerned with the models which will determine the extent of the data being stored. To make sure this data is at least complete, the database schemas are defined below.
- Note that a double arrow indicates a many to many relationship which is usually expressed with a join table, but this clutters the schema.
Device Schema
The device schema must take special consideration to grow in respect to different system configurations rather than number of submissions. In other words, if the same system submits the same information repeatedly, this should not result any growth whatsoever.
The purpose of the DeviceGroup table is to normalize sets of devices which will be common across multiple submissions from similar system.
The distinction between the Device and DeviceState tables is that the former provides information about standalone devices whereas the latter about connected devices which includes driver details and its relationship with other connected devices.
The parent_id column in the DeviceState table represents a tree of device states as an adjacency list, see the following article for a comparison with nested sets.
The reason why the DeviceProduct and DeviceVendor tables are separated from the main Device table is not only to increase normalization but also to be updatable from authoritative locations for PCI ids, USB ids and PNP ids.
Processor Schema
The processor schema should be distinguished from the device schema because a processor contains more specific information than consistently available across devices.
- The columns in the Processor table have been determined to be consistent across many architectures, including ARM. See the cpuinfo_resource script in Checkbox for more details.
Distribution Schema
This provides distribution specific information about the system being tested.
- The content of the distribution tables can be provided from any LSB compliant system. See the lsb_resource script in Checkbox for more details.
- Launchpad already contains the concept of a distribution series but should be extended to include the concept of a distribution release. The distinction is that a release contains descriptive information like "Ubuntu maverick (development branch)".
Test Schema
In addition to test results, the test schema also defines the necessary tables for test definitions. This is important so that some context about the test can be persisted within the same database as the corresponding result.
- The value column in the Result table actually represents both a string column, for the output of a test, and a float column, for the measurement of a benchmark.
The value of the status column in the Result table is an enumeration of the Test Result Codes researched across several test suites: PASS, FAIL, UNINITIATED, UNRESOLVED, UNSUPPORTED, UNTESTED.
- The reason why a Test Result does not contain a foreign key reference to a Test is to increase normalization. As an example, it can be quite common for tests to have essentially the same result consisting of a PASS status and not output. As another example, it can be quite common for a test version to be incremented while the result remains the same in most situations.
- In order to link a Test Result to a Test, this can be accomplished by referencing the Submission for the same Test Identity values. See the Submission Schema for more details.
Submission Schema
The submission schema is where all the data streamed by the test client is stored. This not only consists of the test results but also resources from the system under test such as hardware devices and package releases.
The project column in the Test Identity table provides a generic name inspired from the Job Submission Description Language (JSDL) Specification which can represent a suite in the context of a test.
The annotation column in the Test table provides a generic name, also inspired from the JSDL Specification, which can represent various information.
- The alias_id column in both the Submission and Attachment tables point to the reference returned by the Launchpad Librarian.
Session Schema
The session schema is where the state of the client is persisted when using the API for reporting purposes.
