PolicyAndProcess/ZeroOOPSPolicy

Not logged in - Log In / Register

Revision 8 as of 2011-03-24 19:23:42

Clear message

Policy Overview

In a nutshell, this policy is about moving the tolerance-level for OOPSes to zero. This mean that any user-visible error happening in production is a stop-the-line event and should be fixed ASAP. This includes javascript errors even though we do not currently record OOPS for them: 741991.

The burndown charts are now private, sadly: https://lpstats.canonical.com/graphs/LPQA/ https://lpstats.canonical.com/graphs/LPQAByTeam/

Why this policy?

We should be proud of the service we build and deliver, and we cannot take pride in a low-quality product. Everytime an OOPS page reaches a user, whether because of a time out or an unhandled exception, we failed on the measure of quality. An OOPS page means that a user was prevented from completing their work, that's really bad.

Having zero tolerance for OOPSes in production means that we are putting actions behind our mantra of quality. An OOPS is basically an escaped defect, and we cannot tolerate that.

Daily we have between tens and hundreds of OOPS. This policy is basically about making sure that the Exceptions and Timeouts section of the report are empty.

What should be done about OOPSes

Once we achieve Zero-OOPS status:

But All OOPSes are not the same

All OOPSes in the "Exceptions" and "Time outs" sections should be eliminated. If an OOPS isn't important - because it's only triggered by robots, or for whatever reason, then the system should be changed to not record an OOPS.

One way to prevent an OOPS being visible is to change the exception type so that it doesn't trigger the OOPS code.

The end goals are:

The expected result of achieving these goals is that the system will generally be in good shape and if an OOPS is recorded its something important we should work on immediately - no sifting through many false positives.

When

We are starting this policy now.

Coming Soon

Burn down chart of the bugs with the "oops" and "timeout" tags.