15 October 2008: Marjo and Karl met up in New York
Before we met, Marjo sent an email with a sort of agenda:
1. https://dev.launchpad.net/Ubuntu/InfrastructureNeeds
2. Interviews
1) Very slow in performance.
2) Difficult to use for bug management due to the user interface.
3) Can't make batch changes with GUI. Must use scripts.
4) Documentation not very usable, is confusing and inaccurate. Need
better examples.
5) Can’t search easily
6) Can't close bugs until all related tasks are closed
7) Can only get estimates of per release quality
3. There's some concern that the UI doesn't require comments or reason
why a bug report is being updated or commented on.
4. Bugs submitted by Ubuntu platform team members
5. Can we display the "This Affects Me Too" number by default?
6. Can we have the ability to "lock down" bugs to only allow certain
people the ability to change them? That is, once our developers
have accepted them and are working on them, we don't need random
people changing the priority or packages they affect. Commenting
would still be allowed.
7. Bugs submitted by the canonical-qa team members. We've been using
the tag 'ubuntu-qa' for the important malone bugs[1] that affect
the Ubuntu QA team.
[1] https://bugs.edge.launchpad.net/malone/+bugs?field.tag=ubuntu-qa
Some of the most important ones to us are:
http://launchpad.net/bugs/436267
http://launchpad.net/bugs/151129
http://launchpad.net/bugs/201015
8. The whole bug nomination process is a bit of a mess and has been
for quite a while. Any bug can be nominated for a release and once
a nomination is declined it is not possible to renominate the bug.In the spirit of faithful brain-dumping, I'm going to write up everything we talked about, even though much of it is not related to the "bridging the gap" theme. However, there are some very relevant things below.
The Meeting
I told Marjo about Launchpad's new strategic direction. He understood that it meant that some of his team's needs would be prioritized lower than before, while others would be very high priorities (e.g., those relating to forwarding/receiving bugs and patches to/from upstreams).
I won't reproduce https://dev.launchpad.net/Ubuntu/InfrastructureNeeds here, since it's constantly changing anyway, but Marjo and I noticed that even though much of that page's prioritization was done before the Sep 2009 Launchpad TL sprint, the items in its "High Priority" category were still very much in line with whath we want Launchpad to do.
They use Blueprints a lot. UDS-L will produce various requirements -- high-level actionable Platform team items. Each req will involve various lower-level tasks; they file bugs for those. He feels there's a missing "middle level" interface between Blueprints and bugs.
When testing a release, the testers discover bugs, and each bug may get N tasks associated with it. Some tasks link up to Debian upstream package, others to different releases of Ubuntu, etc. They find it frustrating that they cannot close the overall bug until every subtask is closed; they want the psychological satisfaction of closing it. I said I thought the software was probably behaving okay, since the bug really isn't closed until all the subtasks are done, but that there should be some other closeable item so they can get a sense of progress. They want the thing "off their plate", and the problem is that we offer a too-rigid definition of "plate". We didn't spend a lot of time on this; he understood it was sort of outside the new direction.
He's talked to a bunch of engineering managers and tech leads, throughout Platform (not just in QA) and it turns out they're all writing launchpadlib scripts to give them what the UI does not. Therefore, API deficiencies are a big deal to them. Responsiveness to API concerns would be a win, from their point of view. I said that while it would always be great if the web UI directly supported everything they needed, in practice that degree of custom-fitting probably wouldn't be possible, so I thought writing scripts was actually a good solution. He thought so too, but only as long as they get the API support they need.
ACTION: (DONE) Sent Marjo an email about our effort to consolidate a community around launchpadlib script development/maintenance. It turns out that Platform is just now starting an effort to collect all their own such scripts together, to avoid duplicated effort, and they didn't even know about our existing effort to do this. I've sent him details on this now, so we can join forces.
Re UI issues: he mentioned there are some idiosyncracies, for example, that clicking on a column header does not cause a list to be sorted by that column (in many other UIs it commonly does a sort). I don't remember other idiosyncracies, but might be worth asking again, and/or asking others in Platform.
He said the bug tracker is good for developers, but not so good for managers, as its reporting capability is slim. (Again, scripting probably the answer, but for reports that the community and/or upstreams would like, it may be that building them into the UI is appropriate.)
He feels thet bug triagers have poor tool support; too many clicks and page-load waits in their lives. Also, much better automated dup searching (especially if including Debian bugs and/or other upstream bug databases) would help them a lot. He estimated that they spend 30% or more of their time on finding dups, partly because human dup-finding comes after triaging. So if LP could search harder for dups, or if the UI could make it quicker and easier for humans to spot and link dups, then they'd save a lot of time.
ONE NON-METRIC, AND ONE POSSIBLY-USEFUL METRIC: We don't yet have any way to get metrics on "failure-to-dup", that is, bugs that are really duplicates of existing bugs but we never find out about it. This might be impossible to measure. We could, however, measure which packages tend to have dups filed most often; that knowledge might be very useful in prioritizing a debian-dup-search or upstream-dup-search feature.
Talked a bit about the problem that any logged-in user can do bug management. It's frustrating for Canonical engineering managers that their job requires them to manage bug flow, but their work can be undone by anyone who comes along and decides to set a bug's metadata differently. He acknowledged tension between openness to community participation and ability for Canonical to control its workflow; he also pointed out how in the one case where they locked something down (I think it was Importance), the Canonical people didn't always step up and a lot of bugs were left as "Undecided", when perhaps the community could have contributed to triaging them.
POSSIBLE METRIC: I hypothesized that we could get metrics on community mis-marking or mis-packaging of bugs, by examining how often a Canonical person comes along and changes/undoes a metadata change made by a non-Canonical person. He thought that might be interesting.
PROCESS ISSUE, POSSIBLY RESOLVED NOW: He talked about about how they'd had a longstanding problem with the "+filebug" redirect, and despite lots of correspondence it didn't get fixed for a long time, and then eventually Deryck took a look at it and it turned out to be an easy fix and the bugs team Just Did It. I agreed that sucks, and said very best way to prevent it is to make sure when talking to any Launchpad developer about a specific issue that you get a bug number for it right away; the Launchpad developers should have requested that Platform file a bug, but also Platform should have said "Why aren't you asking us to file a bug?". Once there is a bug report, you can push on it, track it, etc. Just having an email correspondence isn't enough. He totally grokked this; hopefully we won't have this problem again.
He works a lot with Bryce Harrington, Leann Ogasawara, and Brian Murray (names we also heard from Jorge Castro, by the way). We'll want to talk to them.
ACTION WE SHOULD TAKE SOMETIME SOON: Look through https://dev.launchpad.net/Ubuntu/InfrastructureNeeds and evaluate the priorities there based on Launchpad's new direction.
29 October 2008: Marjo and jml had a call
On Sun, Nov 1, 2009 at 11:07 AM, Jonathan Lange <jml@canonical.com> wrote: > Hey guys, > > I'm a rotten note-taker, here are the things I wrote down. I'll add > them to the wiki in a bit. > > * LP performance slow enough to seriously hinder QA team work > * UI sucks for _managing_ bugs, mostly because you can't batch, so > they use API, but API is slow. > * Would be great to have Python examples for launchpadlib, maybe > even a cookbook. > * Can we use Answers as a way of doing this? > * LP is all about working on one bug at a time. Sucks for managing & QA. > * There's a difficulty with marking bugs as closed. Marjo can't > really explain it clearly: bdmurray & ogasawara are the two people to > talk to, esp. ogasawara. > * Stronger per-series views of bugs would be great. > > jml