Add the report to https://dev.launchpad.net/QA/ExploratoryTesting/

BetterBugSubscriptionsAndNotifications

Summary

Describe in a few phrases the main findings of the testing. If someone read only this part of the report, what would you want them to know?

Findings

Users can filter out notifications generated because of a change they made themselves

• Bug 770267: There is a full-stop (period) at the end of the option text. This is inconsistent with the other options on that page.

• After changing the configuration option, there is no notification of what change you've made.

• Supporting material: .

Actions that are done and then quickly undone will not generate notification emails

• Feature appears to work as intended.
• I made sure that I had selected the "Send me bug notifications for changes I make." configuration option and then I added a tag of "test-test-test" to bug 770267. Immediately, I removed the tag. I have not received an email.

Subscribe to individual bugs, choosing to filter by events

• "this bug is fixed or re-opened": this gave me pause. The bug I was looking at was open. I didn't understand how it could be re-opened. why would this bug be re-opened? It i's not closed. It seems pretty clear that this means that, should the bug ever be closed, this option would give you an email if were then to be opened again. However, I had to pause to think about what it might mean as it seemed to imply that the bug was currently closed. Could we improve this by saying, "this bug is closed or later re-opened"? Do we really mean "fixed" here and not "closed"?

• Unsubscribing using the "minus sign" button beside my name on the bug page didn't work. My name disappeared but when I clicked on "Subscribe" I was given the option of . Also, screen cast attached to bug report.

"mute" their notifications on individual bugs

• Bug 770315: The help pop-up could be clearer and use words that are more familiar to our users.

• I muted bug 674422 on staging, as I was subscribed via a structural sub. It timed out first time round. Second time it worked but took four or five seconds. This was much quicker on production with bug 770293, to which I have a direct subscription.

• Bug 770342: When unmuting a bug, the overlay confusingly talks about subscribing to the bug that I'm already subscribed to.

• Bug 770345: when muting a bug, your name is removed from the subscribers list. When unmuting the same bug, your name is not put back in the subscribers list.

Recommendations

• Short recommendation that describes the outcome, rather than the implementation detail.

• Consider things that would make testing easier.

• Consider patterns that would result in systematic quality improvements (e.g. fixing 758976)

Don't add these to the final report.

Checklist

Keep these in mind while doing the exploratory testing session

• Does it match the values?

• Be very conscious of permissions
  • anonymous user?
  • owner?
  • admin user?
  • un-priv user?
• Are the process and scope of the feature easy to understand? If not, why? Does it need help, UI text improvements, other docs?
• How people would discover the feature?
• Focus on things not caught by automated tests (e.g. cohesion of UI elements, small usability glitches, confusing workflows, lack of documentation)
• Is the end to end usage of the feature consistent? Is it following UI guidelines?
• What are the riskiest areas of the feature?
• Try to break the feature (e.g. double posting forms, try XSS vulnerabilities, etc)
• How's the feature going to be deployed? Is there a plan for a rollback?
• What are the most interesting bug found during this session?