Mini OOPS Tools sprint, taking place in Campinas, SP, Brazil, 20-24 September 2010.

We're discussing

• Bug 461269: oops reports should be grouped by oops signature not exception type and exception value (better grouping of oopses; better relation oopses-real problems)

• How to move this information to be easily accessible via web
• Improve oops-tools to handle queries for a given pageid

How to reduce all OOPS emails into one with grouped information

Send only one email to the list including information broke down by team:

• stats section is global, for all production instances
• each team section shows top exception and top timeout and the percentage of oops reports the team is responsible for.
  • maybe show top offending pageid (i.e. the pageid with the greatest number of oopses) instead?
• all team sections show link to a full team report
• remove unnecessary sections from the report, such as soft time out, informational only, user generated errors
• provide a way to find oopses that would appear in those sections through the web ui

• remove unnecessary oopses from the report (e.g bug 540890 and 251896)

Subject: Oops report for 2010-09-22

= Stats for 2010-09-22 =

* 10000 Exceptions
* 50000 Time Outs

== Top 5 Exceptions per Page ID ==

* 300 Person:+index                      - ConjoinedBugTaskError: 'Foo bar is foobared'
* 120 BugTask:+index                     - ConjoinedBugTaskError: 'Foo bar is foobared'
* 100 Distribution:+search               - ConjoinedBugTaskError: 'Foo bar is foobared'
* 80  DistributionSourcePackage:+filebug - ConjoinedBugTaskError: 'Foo bar is foobared'
* 78  Archive:+packages                  - ConjoinedBugTaskError: 'Foo bar is foobared'


== Bugs 20 % ==
Full report: https://lp-oops.canonical.com/summary/?team=Bugs&date=2010-09-22

 * 230 ConjoinedBugTaskError: 'Foo bar is foobared'
 * 440 TimeOut: 'some broken page'

== Code 50 % ==
Full report: https://lp-oops.canonical.com/summary/?team=Code&date=2010-09-22

 * 100 TooNewRecipeError: 'recipe is foobared'
 * 1140 TimeOut: 'some broken page'


== Foundations 10 % ==
Full report: https://lp-oops.canonical.com/summary/?team=Code&date=2010-09-22

 * 330 LibrarianDiskError: 'librarian is down'
 * 666 TimeOut: 'some other page is broken'

• Better team reports

  • PageId to become an object, not just an attribute

    • Find out PageId 'owners'

    • Use django admin interface to set 'orphan' PageIds teams

    • Will have to migrate data to the new model
  • Change how an oops' team is calculated:

    • Prefix > PageId > vhost

We're doing

• Bug 540890: exclude robot posts from reports

We did

• Bug 612354: fix oops-tools bootstraping

• Bug 251896: oops-tools should filter out not found errors referred from non-local domains