Jorge Castro and Karl Fogel have a semi-weekly call (Tuesdays, 1400 UTC). We try to focus on one story per call. The goal is to understand it in depth, capture it in notes that any developer can understand without having to call Jorge for clarification :-), and come out with some action items and suggested metrics that get us closer to turning that story into a Launchpad improvement.
02 December 2009: Conversation about "patch report", after UDS
Karl and Jorge talked about the "patch report" idea, which came out of UDS:
Upstreams' number 1 complaint is not being able to find Ubuntu patches. So show which bugs have patches attached, when/where those patch have been forwarded, do same for other distro's trackers and for upstream trackers as well.
Many people were at the meeting, including (I think) jml and martin. What came out of it was this Blueprint
which points to this wiki page for the full specification:
Bryce Harrington has a protoype page that does this (beautifully) already:
For a bonus -- "icing on the cake", as Jorge puts it -- on top of Bryce's page, expose the patches as bzr branches.
Jorge says: "Nobody is providing a service where you as an upstream project can see, for every distro, where you have patches. We can make patches for every distro more visible." Launchpad can be that service. Having this knowledge would help upstreams and distro. For example, distro could say "I see this patch is in 5 of 6 other distros, maybe I should look at it!"
In other trackers, we can also programmatically determine which bugs have patches attached. See Daniel Holbach's harvest:
Daniel knows the full list of trackers in which we can programmatically determine which bugs have patches attached. So far:
- Launchpad (obviously)
- Debian (?)
ACTION: Karl to get full list from Daniel. (DONE, 2009-12-07. It's complicated: Daniel explained how harvest works, and how it just pulls from CSV files that are generated by custom scripts and all the source-specific knowledge is in the scripts. At http://bazaar.launchpad.net/~harvest-dev/harvest-data/trunk/annotate/head%3A/opportunities you can see the current sources.)
The most important things:
- exposing the patches
- exposing their ages
- exposing how many places have the same patch
More notes from Jorge:
See https://wiki.ubuntu.com/LucidReleaseSchedule for when freezes are.
On Bryce's page, people like having stuff sorted by package, but would even more like it if could be divvied by team. Like the personal page's "bugs related to me" thing.
Feature request: an RSS feed, or some notification system, so that as soon as a patch hits your package, you as upstream know about it.
Also wants to know amount of time a patch sat in Launchpad, then when it is forwarded, then how long it sits upstream. Let's them determine which upstreams are responsive, which will help with prioritizing which patches to handle.
28 October 2008: Conversation about the Sandy Armstrong (of Tomboy) mail, and upstreams.
Today we used the Sandy Armstrong data as a jumping off point.
Sandy asked for automatic bug forwarding. Jorge says it's very rare for an upstream maintainer to request that. However, if we did dup-finding against upstream, more might want it. Two ways to do that: import their bugs on a regular basis, or have an upstream dup search API (either upstream tracker has that natively, or we have it in our Launchpad plugin).
Regarding getting new releases into SRU: if Launchpad had a PPA-from-upstream-trunk-tip feature (or -branch-tip, whatever), upstreams would like that. It gives them better testing on what goes into SRU, and also takes some of the pressure off them in the sense that when a bug report comes in, they would often be able to respond by offering the user a convenient way to install a fix.
ACTION: Karl to ask QA guys (specifically Steve Beattie) how they determine what goes in SRU. They have tags (regression-potential which then becomes regression, or something like that). Specifically: "How do you determine what's safe to release in an SRU? Do users test it first? If so how do they get the stuff to test? Do they often not get it because it's not available conveniently enough (e.g., not in a PPA)?" How can we measure this? DONE: see this internal mail.
Jorge suggested some upstream devs we could talk to at UDS:
(This part of my notes now lives at Launchpad/UDS/L#upstreams; please see there.)
POSSIBLE METRIC: not just number of upstreams using the Launchpad bug plugin, but number of bugs thus exposed (i.e., measure it by "size" of upstream tracker's database).
ACTION: Karl to find out exactly what that plugin does (is it just comment forwarding?), where it's installed, and if upstream dup finding or if that's on the roadmap. ASKED in this internal mail.
POSSIBLE BIG IDEA: plugin for Debian bug tracker that allows Launchpad to be used as UI for the tracker. Remember, users in Launchpad always have an email address associated with them anyway -- they're contactable. ACTION: Karl to figure out who to talk to about that. Like, jml in our one-on-one in seventeen minutes as of this writing. ASKED, on canonical-launchpad@ in this internal mail
21 October 2009: Forwarding bugs upstream is too hard.
Notes from Karl Fogel's call with Jorge Castro on 2009-10-21.
* Linking Launchpad bugs to upstream bugs is harder than it should be. Jorge gave an example. Start with these two entities: 1) launchpad.net/ubuntu/+source/evolution "Evolution (ubuntu)" This represents Evolution *in Ubuntu*. (Note that distro-specific versions of this are, for example, https://edge.launchpad.net/ubuntu/karmic/+source/evolution, https://edge.launchpad.net/ubuntu/jaunty/+source/evolution, https://edge.launchpad.net/ubuntu/intrepid/+source/evolution, etc). 2) launchpad.net/evolution This represents upstream Evolution; nothing to do with Ubuntu. Now consider this common scenario: Pedro sees a bug report come in for Evolution-in-Ubuntu, and determines that it is an upstream problem. He then files a bug directly in upstream Bugzilla (does his dup-searching there, etc). Next he goes back to the Evolution-in-Ubuntu bug in Launchpad, so he can link it to the upstream report he just filed. Here's where the problems start: To link it upstream, he has to click on "Also affects project". PROBLEM #1: Using "Also affects project" to link a Launchpad bug report to an upstream report is not intuitive. Pedro works for Canonical, so he knows it, but Jorge says community people consistently say they're surprised that this link is used for that purpose. Instead, they often leave upstream bug URLs in comments, because they don't realize what this link does. Note that people do understand (correctly) that the link can be used to indicate, say, "This bug in lib_theora also affects that mp3player project". It's just the fact that it *also* means "link this bug to upstream via the upstream's representation in Launchpad" that is totally non-obvious. FWIW, Jorge says the link next to it, "Also affects distribution", is fine & effective, because it's clear what it does and that's the only thing it does. Anyway, moving on... Pedro clicks "Also affects project", but unfortunately, someone had set the wrong upstream link for Evolution-in-Ubuntu. To be very specific about it: In https://edge.launchpad.net/ubuntu/+source/evolution, look at the right column, where it says "Set Upstream Link" for various distros but the top one (Karmic, right now) links to the head series for Evolution upstream: https://edge.launchpad.net/evolution/head When that upstream link is set correctly, and its destination -- the Launchpad project representing upstream -- is configured correctly, then Pedro's next step is very easy: he pastes in the URL of the upstream bug and he's done. But if that *isn't* all configured correctly, life gets much harder. In this case, someone had set the upstream link on https://edge.launchpad.net/ubuntu/+source/evolution to some totally unrelated project. We don't know who did it (probably it's in a log someplace, but nothing in the UI says who; no matter). So at this point, Launchpad got confused and said "What project do you want to link to?" and offered up some random project. Now if Pedro wants to forward his bug upstream, he has to fix this entire situation, configure the upstream link correctly... yet he has the upstream bug URL in his hand! He feels it would be so simple to just let him do the minimal thing he needs to do and not block on this other stuff! PROBLEM #2: There are situations where someone is looking at a Launchpad bug report, and has the URL of the known-related upstream bug report at hand, but can't easily link the two. Why can't people just enter it no matter what? Why do people have to set up all this other information first -- go find the project in Launchpad, then set all its upstream information correctly, including the series, which can be quite difficult to get right -- just to do this simple thing? SUGGESTION: If user has a Launchpad bug report in hand, and an upstream bug report URL, they should be able to link them in one step, always, no matter what. We can also offer them the *opportunity* to set up the upstream correctly, it just shouldn't block their immediate task. METRICS: Brian Murray has a script that finds bugs where someone has put an upstream bug URL in the comments, instead of linking properly to upstream. This is very useful for finding out how big the problem is, and measuring improvement. (Of course, we'll have to filter out the comments where someone is saying "I think this bug might be related to / the cause of this other bug over here", but we can just do a random sample to get the incidence rate for that and compensate.) ACTION: Karl to get Brian's script. ([[https://lists.launchpad.net/private-launchpad-strategy/msg00028.html|Sent him internal email asking for it on 2009-10-26]]) Other interesting things Jorge mentioned: * The extra page load on "Also affects project" is brutal, especially at this point in the cycle. Some people touch 2500 bugs in a cycle. If there's _any_ way we could inline that or speed up upstreaming of bugs, it would be a huge win. * Any time someone on the platform team has run into a Launchpad bug that affects upstream relations, they tag it with 'ubuntu-upstream-relations'. Look at those. * Canonical (specifically Pedro and Sebastien) is known to be the largest contributor to Gnome Testing: we upstream more bugs than anyone else. More counterevidence to the "Canonical doesn't contribute back" meme on the Net. * Jorge is going to do another time-and-motion study of bug triagers at UDS. ACTION: Jorge to get other lp devs in the room when he runs this study, and to videotape it.
13 October 2009: Intro call, many random stories here.
Notes from Matthew Revell's and Karl Fogel's conversation with Jorge Castro on 2009-10-13. These were originally mailed to the launchpad-strategy@ list, but then the archives got blown away in the Great Archive Blow-Away of 2009. It's okay; the notes below are better anyway, as they're a bit more organized than the email was:
People to speak to: Talk to Graham Binns, who has talked to Ubuntu team a lot. Brian Murray, Pedro V. (Ubuntu QA) Ken VanDine (Ubuntu Desktop team (integration engineer)) Matthias Gug (Ubuntu Server Team) Leann Ogasawara (Kernel QA) David Planella (for translations stuff, but note he just sprinted with Danilo et al) Bryce Harrington (Ubuntu X Windows) James Westby (esp re Debian connection) Steve Beattie (Ubuntu QA (Stable Release) team) Karl is meeting with Marjo Mercado (QA) on Thursday. Pedro and Ken are hugely about linking to upstreams. Jorge is already watching upstream linkages report: https://edge.launchpad.net/ubuntu/+upstreamreport ACTION: pull over time to track improvement/non-improvement Graham implemented what's there. Daniel has done some screen-scraping already on this "We want our developers marking bugs as upstreamable. As that number goes up, we can add more Watches." Row goes green if over 90% of bugs upstreamed have a Watch. We might want to go green based on percentage of total product bugs upstreamed instead! Right now it shows top 100 products with open bugs. Teams want team-specific reports, like server team, desktop team; or just for a group of packages (e.g. those handled by a particular MOTU), and so on. Pending bug watches -- "these are possible targets: bugs marked as possibly upstream but that do not have a link". https://bugs.edge.launchpad.net/ubuntu/+source/rhythmbox/+bugs?search=Search&field.status_upstream=pending_bugwatch Jorge would love stats on upstream rejects and the reasons for them; we don't have that now. ACTION: Brian Murray ran a query on who's forwarding bugs upstream; he got numbers per person, and it was a lot of non-Canonical people. ACTION: Hey, we don't have stats on who accesses these reports nor how often. Wouldn't that be nice to know? Jorge uses these reports to push sub-communities to do things. No way to remove task from a bug! "open an upstream task on firefox. A week later, ffox upstream says it's distro's fault. So distro opens up a null task -- that's their workaround." Yes, Jorge would love a way to mark a bug a "determined to be not upstreamable"! (How can we measure it?) Daniel Holbach wrote http://daniel.holba.ch/harvest/handler.py?pkg=f-spot, which examines patches shipped by other distros for packages. FEATURE: track patches shipped by any distro for your package then click a button to integrate it into a Bazaar branch so I (upstream) can mess with it ACTION: Talk to Banshee to find out how they talk to Launchpad, since Jorge says they use it a lot even though they don't host there. Another one is Tomboy (Sandy Armstrong); maybe shoot for that first -- he's worked with our Online Services people before. Jorge says areas he's been concentrating on bugs and patches. Also said James Westby did a little query that showed the differences between what we're shipping and what Debian's shipping. E.g., how different is our Firefox from Debian's. Looking at diff^2 size turns out to be useless; you really have to look at the patches (it might be language packs, for example). Jorge wants to know what changes we're maintaining that we don't have to, because they could be either upstreamed or Debianed. ACTION: is there a way to measure this? Or measure how long a patch sits in a package? ACTION: Jorge to forward Karl conversation about server patches we were carrying that it turned out Debian had. Process of upstreaming is ad hoc and manual, when sending to upstreams. Debian has an automated query that they can do on a per-package basis (see http://packages.qa.debian.org/f/f-spot.html), but we don't have stats on how often they run that query nor how often they use the results. (Lucas Nussbaum is the DD and UD who maintains this, btw.) Linking bugs upstream is still difficult. Jorge did a little study on how many clicks it takes for someone to forward a bug upstream, and it was pretty bad. ACTION: get details, or better yet ask Jorge to re-do the study, since Launchpad has changed in a year. Upstream Gnome Bugzilla (as Bugzillas get upgraded this will be in more places) now has a field for URL, where we put the URL field. We want to automate this workflow, but automated only for people we trust (probably), to avoid spamming upstream. Jorge wishes that once he's determined that someone's contention that a bug is upstreamable is correct, that he had a nice AJAX-y way to send it upstream in one click. Says he could get a lot more done if he had that. Even better, pre-empt bug reports! IMPORTANT: We have an import of the entire Debian bug database. Jono is really keen on sucking in all these upstream bugs, and then using them as part of dup detection. We could do that with other bug trackers as well. Gnome, KDE, OpenOffice, Firefox, what others? Graham Binns was eager to work on this, too! ACTION: Karl and Matthew to call Deryck today. (Note: I didn't get to this -- got Skype working instead. Small triumphs, small triumphs.) ACTION: Plan a strategy sprint with Ubuntu people? For questions about Bazaar/codehosting and upstreams, talk to James Westby. Broken or obsolete VCS imports is a problem. For example, old Gnome SVN -- but that's a case of "wrong" not "broken". ACTION: Do we have any procedure for autodetecting truly broken imports? Jorge working on "upstream ambassador" project, whereby someone could recommend a PPA to try when a package is having a problem but SRU means upstream doesn't have anything we can put into standard distro yet. Martin Pitt is working on this thing where if your Python project uses setup.py, then a one line command can create .deb (that you could then put in a PPA). Right now, when you upload a source package to a PPA, it will build for whatever release you say (Karmic, etc), for all archs. Would be nice to build Debian (e.g., sarge) packages. Mark may have been against this; bigjools and/or jml might know for sure. ACTION: get one person from each team to sit down at UDS with jorge and mrevell and jml. Schedule these soon. When / where is next UDS? https://wiki.ubuntu.com/UDS-L exists, but https://wiki.ubuntu.com/UDS-M does not yet.