= Create a cache of statistics for bugtasks =

The time it takes to calculate structural aggregates of bugs in Ubuntu is overly expensive and growing as more bugs are opened. This shows up as:
 * bug portlets update up to 60 minutes after changes happen
 * When they update they will sometimes time out : slow times to calculate bug summary portlets(series task counts, milestone task counts and untriaged etc counts.)

This LEP is about introducing an aggregate store which will let those counts be calculated more cheaply, at a potential loss of accuracy. Doing them more cheaply will allow:
 * Immediate portlet updates
 * No timeouts on bug portlets {for another 4-5 years}

'''Contact:''' RobertCollins <<BR>>
'''On Launchpad:''' [[https://bugs.launchpad.net/launchpad/+bug/758587|bug 758587]]
'''List discussions:''' [[https://lists.launchpad.net/launchpad-dev/msg06914.html|initial thread]]

'''As a ''' User<<BR>>
'''I want ''' Launchpad bug portlets to be fast<<BR>>
'''so that ''' they show up quickly<<BR>>

This is a primarily technical change but has one distinct and possibly unsettling user visible change. Specifically, because of our model for visibilty of private bugs, the summary table can overcount visible private bugs.

The way this overcounting will happen is that when a user has multiple subscriptions to a private bug, they will see the bug counted once per subscription. For instance, if someone on the Ubuntu security team files a security bug on Ubuntu, they will see their 'open bug' portlet count go up by 2 - once for their direct subscription, and once for their indirect subscription via the Ubuntu security team.

This has the following properties:
 - public bug counts will be completely accurate.
 - counts of 0 will always show as zero.
 - The error rate for developers with access to private bugs is about 1 in 200. [We will show 201 when the actual figure should be 200].
 - We should be able to render the existing portlets in 1/2 second.

While its not ideal to ever show erroneous information we have been showing erroneous data in the past and still do (the 60 minute memcache time mentioned above).

== Rationale ==

Open bugs for Ubuntu constitute 1/7th of the entire bug database but all bugs for Ubuntu constitute 1/2 the bug database. As a result many aggregate queries end up processing the entire bug database (which requires about 2million rows to be processed).

Bringing in a table which preaggregates bugs that are the same from the perspective of the aggregations we do will permit us to provide broad aggregate statistics.

We can in principle query separately for private bugs but this will require optimisation of those queries vs what we do today and in addition to use the aggregate table for public bugs.

== Stakeholders ==

Users. 

== Constraints and Requirements ==

=== Must ===

=== Nice to have ===

=== Must not ===

Show a non-zero count where a zero-count is expected: This would stop people driving bug counts to zero and be very frustrating.

=== Out of scope ===

Complete accuracy and precision in the short term. Medium/long term we can revisit and see if a schema can be come up with to count private bugs accurately and quickly.

== Subfeatures ==

== Success ==

=== How will we know when we are done? ===

=== How will we measure how well we have done? ===

== Thoughts? ==