Diff for "ArchitectureGuide"

Not logged in - Log In / Register

Differences between revisions 2 and 3
Revision 2 as of 2010-09-01 19:36:44
Size: 1934
Editor: lifeless
Comment: more
Revision 3 as of 2010-09-01 21:52:15
Size: 3562
Editor: lifeless
Comment: start to expand
Deletions are marked like this. Additions are marked like this.
Line 27: Line 27:
== Related documents ==

Of particular note the PythonStyleGuide is specifies coding style guidelines.
Line 41: Line 45:
Transparency speaks to the ability to analyse the system without dropping into pdb or taking guesses.

Some specific things that aid transparency:

 * fine grained just-in-time logging (e.g. bzr's -Dhpssdetail option)

 * live status information
  * (+opstats, but more so)
  * cron script status
  * migration script status

 * regular log files
 * OOPS reports - lovely

We already have a lot of transparency. We can use more.

Aim for automation, Developer usability, minimal losa-intervention, on-demand access.

When adding code to the system, ask yourself 'how will I troubleshoot this when it goes wrong without access to the machine it is running on'.
Line 42: Line 66:

The looser the coupling between different parts of the system the easier it is to change them. Launchpad is pretty good about this in some ways due to the component architecture.

But its not the complete story and I think decreasing the coupling more will help the system.

I've seen some recent work on this such as the jobs system and the buildd queue refactoring, which is excellent - generic pieces that can be used and reused.

The acid test for the coupling of a component is 'how hard is it to reuse?'

And we can go further. For instance, the job system is nice, but its tightly coupled to the launchpad DB, perhaps we could make it possible to use it for other Canonical projects, or other Zope projects. Or perhaps move it to MQ and just have an adapter instead? Tasks running in a job could still talk to the DB.

Architectural Guide

In this guide you will find some expansion and clarification on the architectural values I presented in the Launchpad Architectural Vision 2010

All the code we write will meet these values to a greater or lesser degree. Where you can, please make choices that make you code more strongly meet these values.

Some existing code does not meet them well; this is simply an opportunity to get big improvements - by increasing e.g. the transparency of existing code, operational issues and debugging headaches can be reduced without a great deal of work.

This guide is intended as a living resource : all Launchpad developers, and other interested parties, are welcome to join in and improve it.

Goals

The goal of the recommendations and suggestions in this guide are to help us reach a number of big picture goals: We want Launchpad to be:

  • Blazingly fast
  • Always available
  • Change safely
  • Simple to make, manage and use
  • Flexible

(See the presentation for more details).

However its hard when making any particular design choice to be confident that it drives us towards these goals : they are quite specific, and not directly related to code structure or quality.

Of particular note the PythonStyleGuide is specifies coding style guidelines.

Values

There are a number things that are more closely related to code, which do help drive us towards our goals. These are values I (RobertCollins) hold dear, and which the more our code meets these values, the easier it will be to meet our goals.

The values are:

  • Transparency
  • Loose coupling
  • Highly cohesive
  • Testable
  • Predictable

Transparency

Transparency speaks to the ability to analyse the system without dropping into pdb or taking guesses.

Some specific things that aid transparency:

  • fine grained just-in-time logging (e.g. bzr's -Dhpssdetail option)
  • live status information
    • (+opstats, but more so)
    • cron script status
    • migration script status
  • regular log files
  • OOPS reports - lovely

We already have a lot of transparency. We can use more.

Aim for automation, Developer usability, minimal losa-intervention, on-demand access.

When adding code to the system, ask yourself 'how will I troubleshoot this when it goes wrong without access to the machine it is running on'.

Loose coupling

The looser the coupling between different parts of the system the easier it is to change them. Launchpad is pretty good about this in some ways due to the component architecture.

But its not the complete story and I think decreasing the coupling more will help the system.

I've seen some recent work on this such as the jobs system and the buildd queue refactoring, which is excellent - generic pieces that can be used and reused.

The acid test for the coupling of a component is 'how hard is it to reuse?'

And we can go further. For instance, the job system is nice, but its tightly coupled to the launchpad DB, perhaps we could make it possible to use it for other Canonical projects, or other Zope projects. Or perhaps move it to MQ and just have an adapter instead? Tasks running in a job could still talk to the DB.

Highly cohesive

; focused components

Testable

Predictable

; in isolation, load tests, broken-backend tests

ArchitectureGuide (last edited 2021-11-25 16:19:00 by cjwatson)