Not logged in - Log In / Register

Bug triage > Process background

Background to our bug triage process

Triage is the act of sorting bugs into different priority groups.

There are many conflicting sorts; everyone has their pet bug that should be 'first'. The sort order we choose is from the project's perspective: we try to balance the needs of our users.

So, bug triage is: sorting bugs by importance-to-the-project, and these are the influences we try to strike a balance between in assessing that importance:

When we have triaged a bug, it has status triaged and an importance other than unknown.

Why triage

This may be obvious, but having just a big bucket of open bugs isn't very efficient: there are more genuinely important issues to fix than engineers and, as such, engineers will forget what things are urgent and what aren't.

Secondly, each of the groups of users whose needs we're trying to compromise between are interested in when things will get done. By sorting the bugs we provide a proxy metric for when tasks will be worked on.

How much triage is needed?

The world is dynamic and constantly changing; as such any sort we come up with for our bugs will be outdated pretty quickly. We could make the sort complete (so all bugs are ranked) and constantly refresh it. However this is inefficient: the only times the sort actually matters are:

So how much sorting is enough? Two interesting metrics are freshness and completeness.

If the sort is too old, bugs will be indicated as 'should be next to work on' that are not valid as that any more. Our priorities may change month to month but they rarely change faster than that : so we can tolerate things being months (or more) stale.

The sort is complete enough if the answers to 'what is an important bug to work on now' and questions that users may ask (like 'how long till this will be worked on') get answers accurate enough... and how accurate do we need?

We answer that question in the bug triage policy itself.

Bug Importance

Bug importance in Launchpad is where we record the result of the triage process; we have 5 buckets we can use in Launchpad: critical/high/medium/low/wishlist.

We don't actually ever block a release based on having a particular importance bug - we block releases based on having regressions, which any commit can have - and we mark that on the bug mapping to the commit.

The buckets combine to give a partial sort: bugs in the critical bucket are sorted before bugs in the high bucket.

We can choose to use some or all of these 5 buckets.

How many do we need? A good way to answer that is to consider our hypothetical complete, fresh sort, and consider how many slices we'd need to make in it to answer questions well; we also need to consider what would change to those slices when things change (such as new things coming that sort to the front).

Also buckets have a cost : we need a ruleset for triage that will let us assign bugs to buckets: every bucket makes the heuristics more complex.

Given that we have a freshness tolerance for most bugs of some months, that we don't want to update many bugs when a single bugshuffles in front, and that because we have more bugs coming in than we fix - we need three or perhaps four buckets:

We map these buckets into:

This has a clear tension: time-till-we-start-work is a good metric for what bucket to put in, but given a bug with some symptoms how do we decide what bucket it should go into.

BugTriage/Background (last edited 2011-06-03 10:52:07 by matthew.revell)