Bug Notifications and Subscriptions: Less Noise, More Control

Note: this LEP was approved a long time ago. It is now being submitted for re-approval by the product strategist, so we have a clearer definition of success.

We should make subscribing to bugs something people want to do, not dread.

Contact: GaryPoster and the Yellow squad
Originally Drafter: Deryck Hodge <deryck.hodge@canonical.com>
On Launchpad: The story-better-bug-notification tag.

As an Ubuntu developer
I want control over my Launchpad subscriptions
so that I can have an easier time following exactly what I care about in Launchpad.

As an upstream developer
I want control over what information I get from Launchpad
so that I can easily find bugs related to my software.

We are taking two primary approaches to these goals.

• Forward-looking action: Provide more control over subscriptions, so you get fewer unwanted notifications.

• Experience-based reaction: Provide a way to understand why you got a particular notification, and a way to react to it so you don't get notifications like it again.

In addition, we will make a small effort to do better batching of notifications.

If people get fewer Launchpad notifications that do not interest them, they will have an easier time finding and following the bugs and other events that they do care about.

Rationale

Why now?

This story was taken up after consultation with Brian Murray, representing Ubuntu Platform QA. Launchpad's current focus is Getting Bugs Off Ubuntu. As a feature slot came open for development, we asked Brian what would help Ubuntu best manage the amount of bugs they have. He said, better bug subscribing and notifications.

Brian says, "people subscribe mostly to packages, but still get too much mail"

Stats from DB confirm packages are the most subscribed to: https://pastebin.canonical.com/29229/

What value does this bring?

Fixing bug notifications to be less noisy and bug subscribing to allow finer grained controls will allow people to manage the high volume of bugs against packages. This brings value to both Ubuntu and upstream developers by allowing people to better find and follow bugs that interest them.

Stakeholders

Who will care about this work?

In particular, Ubuntu developers who are subscribed to large numbers of packages. In general, anyone who receives a lot of bug mail.

• bdmurray
• bryce
• leanne
• cjwatson
• pitti
• seb128
• ubuntu server team
• ubuntu kernel team
• keybuk (Note: too much bug mail became a serious problem to the end of 10.04 LTS)
• Elliot Murphy (or someone on his team)

When did we last talk to these stakeholders?

Only bdmurray has been consulted so far, and only in preparation for this story.

We hope to use iterative user interface testing to get early input into the work from selected stakeholders, and alpha testing for later input. The alpha testing group is malone-alpha.

Constraints and Requirements

Must

• Users can subscribe to individual bugs, choosing to filter by events. They can receive all notifications as they do now ("the bug receives any change"), or to only receive notifications if the "bug is opened, closed, or reopened," or to only receive notifications if the "bug is changed in any way other than a simple comment." (See further discussion on this in "Nice to have.")

• Users can subscribe to notifications on the bugs of projects, packages, and distributions ("structural subscriptions") with any of the following filters, intersected. Bug 674422

  • Only subscribe to certain events that trigger a notification: choose one among these options "bug is opened, closed, or reopened", "the bug is changed in any way other than a simple comment", or "the bug receives any change." (See further discussion on this in "Nice to have.")

  • Users can filter out notifications generated because of a change they made themselves. Bug 548

  • Only subscribe to notifications that happen on bugs that are tagged with any of one or more tags or all of one or more tags. Users can include negative assertions about tags--so, for instance, you can subscribe to bugs that have one tag and do not have another. (See further discussion on this in "Nice to have.") Bug 151129

  • Only subscribe to notifications that happen on bugs that have any of one or more selected statuses.

  • Only subscribe to notifications that happen on bugs that have any of one or more selected importances.

• Users can make multiple bug notification subscriptions for the same project, package, or distribution. Each can have different fliters as described above. Multiple subscriptions that overlap result in only a single notification. This means that, for instance, you can subscribe to all bug notifications in the Launchpad project that have the UI tag, and also bug notifications in the Launchpad project that show when a bug is opened or closed with the tag "bugjam2011". If a notification matches both subscriptions, you only get one copy of the notification.

• Users can edit and delete these subscriptions.

• Users can "mute" their notifications on individual bugs, so even if they would normally get a notification for it because of a subscription on a package, for instance, they will not hear notifications about that particular bug. Bug 204980

• Each email notification will include a link to a page with all subscriptions that led to the receipt of that email. You can mute notifications from that particular bug, or edit or delete the subscriptions that made you receive the email. Bug 649252

• Notifications will batch attachments that are added to the same bug by the same person within a certain amount of time (five minutes now, AIUI). Bug 424849

Must not

• Actions that are done and then quickly undone will not generate notification emails Bug 164196

Nice to have (Maybe)

• To add extra flexibility to the structural subscriptions filters described above in the "Must" section, the subscriptions allow the following approaches.

  • Users can choose to also receive notifications when a bug no longer matches the filter's description. For instance, if you are subscribed to a tag named "user-experience" because that is your domain in a given package, you may want to know when a bug loses that tag. (Note that this may cause confusion if you have made this gesture and yet are not subscribed to metadata events, because then you will not receive the notification you expect.) (Implementation note: after discussion with Deryck, sending notifications on a bug's diff like this sounds doable but possibly tricky.)

  • Finer-grained event filtering (please note that the behavior described in the "Must" section is already mostly implemented, via an enumeration.). The following two options are mutually exclusive.

    • (Option 1: Extend the existing enumeration.) In addition to being able to choose to receive events as described in "Must" (no event filtering, or "only when a bug is opened, closed, or reopened", or "if the bug is changed in any way other than a comment alone"), include two other options: "only when a bug is first opened" and "only when a bug is opened or the status changes". As before, users must choose only one of the full set of available event filters; they cannot be combined. The first new one is the most minimal optional; the second one is a bit more inclusive than "only when a bug is opened, closed, or reopened" but still does not trigger notifications for changes to other metadata, like importance changes and linked branches, or for comments.

    • (Option 2: Replace the enumeration with individually selectable choices.) Users can subscribe to fine-grained changes, like choosing explicitly to include or omit changes to each of initial creation, status, importance, linked branches, and comments. Please note the discussion about this feature in the "Thoughts?" section, titled "Unlimited configuration for notifications?"

  • Tag filtering:

    • Users can subscribe to events that happen on bugs that have no tags (this is partially implemented).

    • Users can subscribe to events that happen on bugs that have any tags (this is partially implemented).

    • Instead of a single broad "any" or "all" choice on the tag filter as described in the "Must" section, users can combine filter specifications with parentheses and boolean ANDs and ORs.

• The extra filtering described above for events is also available for direct subscriptions to bugs. This would almost certainly need to involve UI changes for the direct subscriptions.

• Users can have a gesture to "mute" a direct subscription. The only reason this would make sense, as opposed to simply deleting the direct subscription, would be because we (ab)use subscriptions to provide security access to private bugs. You could mute a direct subscription in order to keep access to the bug while stopping notifications from that bug. A better solution would be to change how we manage the security of private bugs, which I believe is something that Curtis Hovey is leading.

• Email notifications include a X-Launchpad-Subscription-Description header with the (optional, user-defined) names of the subscriptions that caused the notification to be sent.

• We provide a single, central page for a user to see and manage all of their subscriptions. This might be limited to bug subscriptions, or limited further to structural bug subscriptions.

• We provide a single, central page for a team to see and manage all of the related subscriptions.

• Structural subscriptions to a distribution are currently disallowed except for a team (to allow mailing list subscriptions to all bug notices). Deryck reports that this was done for two reasons. The most important one was performance: sending out bugs for the structural subscriptions on Ubuntu was causing timeouts. A secondary reason was social: people were somehow subscribing to all of the bugs in Ubuntu, receiving tons of mail, and publicly complaining about the receipt of all those notifications (Launchpad was "spamming" them). We'd have to figure out a way to reduce the risk of a performance problem, but we could re-allow structural subscriptions on distributions. For instance, I could imagine people wanting to receive all bugs that match a certain tag. Perhaps we could allow subscriptions as long as they were constrined to certain tags?

Out of scope

Some of the "Nice to have" elements above may move here.

• Subscribe to search results (related to Bug 49752). This was originally part of the LEP, and advocated by mpt and jml. However, Deryck informed me that this was investigated and rejected for technical reasons. jml confirmed verbally that he understood this and had accepted it.

Subfeatures

(none)

Workflows

Please see the more detailed workflows, including mockups, described in yellow/Subscriptions. The following are the workflows from the original LEP.

Bug page

New subscription:

• A user clicks "Subscribe"
• An overlay appears with the various options
• The user makes choices, clicks "subscribe" on the overlay
• The user is subscribed to the bug with those options

Existing subscriptions:

• A user clicks "Edit Subscription"
• An overlay appears with the various options
• The user changes the choices and clicks "Update subscription"
• The changes take effect

Packages, Projects

New subscription:

• A user clicks "Subscribe"
• An new page is loaded with the various options
• The user makes choices, clicks "subscribe" on the form
• The user is subscribed to the bug with those options and redirected back to the bugs homepage for the object

Existing subscriptions:

• A user clicks "Edit Subscription"
• An new page is loaded with the various options
• The user changes the choices and clicks "Update subscription"
• The changes take effect and the user is redirected to the bugs homepage for the object

Distributions

As mentioned above in the "Nice to have" section, structural subscriptions to a distribution are disallowed except for a team (to allow mailing list subscriptions to all bug notices). We might consider re-opening this, as discussed above.

Mockups

The current mockups can be seen in yellow/Subscriptions.

These are the original mockups: lp:~gmb/+junk/subscription-widget-mockups/

Success

How will we know when we are done?

All critical and high bugs for the story-better-bug-notification tag are closed. These bugs should either be related to the features in the Must or Must Not section, or technical bugs about the implementation of these features.

Features in the "Nice to Have" section may be promoted to "Must" by the Product Strategist at any time, but we are hoping that these will only be promoted when we have delivered, or are about to deliver, the full set of "Must" features. Then the product strategist with the yellow squad can determine if any further increments are desired before the LEP is closed and the squad moves to a bug rotation.

How will we measure how well we have done?

• We see an increase of bug subscriptions after the features are released, because users now want to use bug subscriptions more because they are more usable.
• We see significant use of the new features--the filters in particular.

Any other ideas? Other possibilities from the original LEP, which are currently not intended to be used:

• less bug mail for pitti, seb128
• less mail to ubuntu-bugs
• how many packages lack subscribers

  • On staging, 6818 Ubuntu source packages out of 34423 have subscribers [2010-04-21]

    • SELECT COUNT(DISTINCT sourcepackagename) FROM structuralsubscription WHERE distribution = 1;

    • SELECT COUNT(*) FROM sourcepackagename;

  • See also http://bazaar.launchpad.net/~ubuntu-bugcontrol/ubuntu-qa-tools/master/annotate/head:/launchpadlib-scripts/packages-without-subscribers.py

Useful info to have:

• what filters/options are selected?

Thoughts?

Unlimited configuration for notifications?

The following discussion was in an earlier version of the draft. It apparently happened before the team decided that subscription-from-search was out of scope. It seems that the first speaker is Deryck.

Dividing filtering

We decided against the following line of thinking, because it led to dividing "filters" off away from "subscriptions". We felt providing a unified interface to what was called was both more understandable and more flexible. In our revised design, you can subscribe multiple times to the notifications on a particular bug target. On each subscription, you can filter based on both the kind of events (which was available on "subscriptions") and on the state of the bug (importances, statuses, and tags, which were available on "filters").