Strategy/RawData/JorgeCastro

Not logged in - Log In / Register

Jorge Castro and Karl Fogel have a semi-weekly call (Tuesdays, 1400 UTC). We try to focus on one story per call. The goal is to understand it in depth, capture it in notes that any developer can understand without having to call Jorge for clarification :-), and come out with some action items and suggested metrics that get us closer to turning that story into a Launchpad improvement.

02 December 2009: Conversation about "patch report", after UDS

Karl and Jorge talked about the "patch report" idea, which came out of UDS:

Upstreams' number 1 complaint is not being able to find Ubuntu patches. So show which bugs have patches attached, when/where those patch have been forwarded, do same for other distro's trackers and for upstream trackers as well.

Many people were at the meeting, including (I think) jml and martin. What came out of it was this Blueprint

which points to this wiki page for the full specification:

Bryce Harrington has a protoype page that does this (beautifully) already:

For a bonus -- "icing on the cake", as Jorge puts it -- on top of Bryce's page, expose the patches as bzr branches.

Jorge says: "Nobody is providing a service where you as an upstream project can see, for every distro, where you have patches. We can make patches for every distro more visible." Launchpad can be that service. Having this knowledge would help upstreams and distro. For example, distro could say "I see this patch is in 5 of 6 other distros, maybe I should look at it!"

In other trackers, we can also programmatically determine which bugs have patches attached. See Daniel Holbach's harvest:

Daniel knows the full list of trackers in which we can programmatically determine which bugs have patches attached. So far:

ACTION: Karl to get full list from Daniel. (DONE, 2009-12-07. It's complicated: Daniel explained how harvest works, and how it just pulls from CSV files that are generated by custom scripts and all the source-specific knowledge is in the scripts. At http://bazaar.launchpad.net/~harvest-dev/harvest-data/trunk/annotate/head%3A/opportunities you can see the current sources.)

The most important things:

More notes from Jorge:

See https://wiki.ubuntu.com/LucidReleaseSchedule for when freezes are.

On Bryce's page, people like having stuff sorted by package, but would even more like it if could be divvied by team. Like the personal page's "bugs related to me" thing.

Feature request: an RSS feed, or some notification system, so that as soon as a patch hits your package, you as upstream know about it.

Also wants to know amount of time a patch sat in Launchpad, then when it is forwarded, then how long it sits upstream. Let's them determine which upstreams are responsive, which will help with prioritizing which patches to handle.

28 October 2008: Conversation about the Sandy Armstrong (of Tomboy) mail, and upstreams.

Today we used the Sandy Armstrong data as a jumping off point.

Sandy asked for automatic bug forwarding. Jorge says it's very rare for an upstream maintainer to request that. However, if we did dup-finding against upstream, more might want it. Two ways to do that: import their bugs on a regular basis, or have an upstream dup search API (either upstream tracker has that natively, or we have it in our Launchpad plugin).

Regarding getting new releases into SRU: if Launchpad had a PPA-from-upstream-trunk-tip feature (or -branch-tip, whatever), upstreams would like that. It gives them better testing on what goes into SRU, and also takes some of the pressure off them in the sense that when a bug report comes in, they would often be able to respond by offering the user a convenient way to install a fix.

ACTION: Karl to ask QA guys (specifically Steve Beattie) how they determine what goes in SRU. They have tags (regression-potential which then becomes regression, or something like that). Specifically: "How do you determine what's safe to release in an SRU? Do users test it first? If so how do they get the stuff to test? Do they often not get it because it's not available conveniently enough (e.g., not in a PPA)?" How can we measure this? DONE: see this internal mail.

Jorge suggested some upstream devs we could talk to at UDS:

(This part of my notes now lives at Launchpad/UDS/L#upstreams; please see there.)

POSSIBLE METRIC: not just number of upstreams using the Launchpad bug plugin, but number of bugs thus exposed (i.e., measure it by "size" of upstream tracker's database).

ACTION: Karl to find out exactly what that plugin does (is it just comment forwarding?), where it's installed, and if upstream dup finding or if that's on the roadmap. ASKED in this internal mail.

POSSIBLE BIG IDEA: plugin for Debian bug tracker that allows Launchpad to be used as UI for the tracker. Remember, users in Launchpad always have an email address associated with them anyway -- they're contactable. ACTION: Karl to figure out who to talk to about that. Like, jml in our one-on-one in seventeen minutes as of this writing. ASKED, on canonical-launchpad@ in this internal mail

21 October 2009: Forwarding bugs upstream is too hard.

Notes from Karl Fogel's call with Jorge Castro on 2009-10-21.

* Linking Launchpad bugs to upstream bugs is harder than it should be.

  Jorge gave an example.  Start with these two entities:

    1) launchpad.net/ubuntu/+source/evolution
       "Evolution (ubuntu)"  This represents Evolution *in Ubuntu*.

       (Note that distro-specific versions of this are, for example,
        https://edge.launchpad.net/ubuntu/karmic/+source/evolution,
        https://edge.launchpad.net/ubuntu/jaunty/+source/evolution,
        https://edge.launchpad.net/ubuntu/intrepid/+source/evolution,
        etc).

    2) launchpad.net/evolution
       This represents upstream Evolution; nothing to do with Ubuntu.

  Now consider this common scenario:

  Pedro sees a bug report come in for Evolution-in-Ubuntu, and
  determines that it is an upstream problem.  He then files a bug
  directly in upstream Bugzilla (does his dup-searching there, etc).
  Next he goes back to the Evolution-in-Ubuntu bug in Launchpad, so he
  can link it to the upstream report he just filed.

  Here's where the problems start:

  To link it upstream, he has to click on "Also affects project".

  PROBLEM #1: Using "Also affects project" to link a Launchpad bug
              report to an upstream report is not intuitive.  Pedro
              works for Canonical, so he knows it, but Jorge says
              community people consistently say they're surprised that
              this link is used for that purpose.  Instead, they often
              leave upstream bug URLs in comments, because they don't
              realize what this link does.  

              Note that people do understand (correctly) that the link
              can be used to indicate, say, "This bug in lib_theora also
              affects that mp3player project".  It's just the fact that
              it *also* means "link this bug to upstream via the
              upstream's representation in Launchpad" that is totally
              non-obvious.

              FWIW, Jorge says the link next to it, "Also affects
              distribution", is fine & effective, because it's clear
              what it does and that's the only thing it does.

  Anyway, moving on...

  Pedro clicks "Also affects project", but unfortunately, someone had
  set the wrong upstream link for Evolution-in-Ubuntu.  To be very
  specific about it:

  In https://edge.launchpad.net/ubuntu/+source/evolution, look at the
  right column, where it says "Set Upstream Link" for various distros
  but the top one (Karmic, right now) links to the head series for
  Evolution upstream: https://edge.launchpad.net/evolution/head

  When that upstream link is set correctly, and its destination -- the
  Launchpad project representing upstream -- is configured correctly,
  then Pedro's next step is very easy: he pastes in the URL of the
  upstream bug and he's done.

  But if that *isn't* all configured correctly, life gets much harder.

  In this case, someone had set the upstream link on
  https://edge.launchpad.net/ubuntu/+source/evolution to some totally
  unrelated project.  We don't know who did it (probably it's in a log
  someplace, but nothing in the UI says who; no matter).  So at this
  point, Launchpad got confused and said "What project do you want to
  link to?" and offered up some random project.  Now if Pedro wants to
  forward his bug upstream, he has to fix this entire situation,
  configure the upstream link correctly... yet he has the upstream bug
  URL in his hand!  He feels it would be so simple to just let him do
  the minimal thing he needs to do and not block on this other stuff!

  PROBLEM #2: There are situations where someone is looking at a
              Launchpad bug report, and has the URL of the known-related
              upstream bug report at hand, but can't easily link the
              two.  Why can't people just enter it no matter what?  Why
              do people have to set up all this other information first
              -- go find the project in Launchpad, then set all its
              upstream information correctly, including the series,
              which can be quite difficult to get right -- just to do
              this simple thing?

  SUGGESTION: If user has a Launchpad bug report in hand, and an
              upstream bug report URL, they should be able to link them
              in one step, always, no matter what.  We can also offer
              them the *opportunity* to set up the upstream correctly,
              it just shouldn't block their immediate task.

  METRICS:    Brian Murray has a script that finds bugs where someone
              has put an upstream bug URL in the comments, instead of
              linking properly to upstream.  This is very useful for
              finding out how big the problem is, and measuring
              improvement.  (Of course, we'll have to filter out the
              comments where someone is saying "I think this bug might
              be related to / the cause of this other bug over here",
              but we can just do a random sample to get the incidence
              rate for that and compensate.)

  ACTION:     Karl to get Brian's script.  ([[https://lists.launchpad.net/private-launchpad-strategy/msg00028.html|Sent him internal email asking for it on 2009-10-26]])

Other interesting things Jorge mentioned:

  * The extra page load on "Also affects project" is brutal, especially
    at this point in the cycle.  Some people touch 2500 bugs in a cycle.
    If there's _any_ way we could inline that or speed up upstreaming of
    bugs, it would be a huge win.

  * Any time someone on the platform team has run into a Launchpad bug
    that affects upstream relations, they tag it with
    'ubuntu-upstream-relations'.  Look at those.

  * Canonical (specifically Pedro and Sebastien) is known to be the
    largest contributor to Gnome Testing: we upstream more bugs than
    anyone else.  More counterevidence to the "Canonical doesn't
    contribute back" meme on the Net.

  * Jorge is going to do another time-and-motion study of bug triagers
    at UDS.

    ACTION: Jorge to get other lp devs in the room when he runs this
            study, and to videotape it.

13 October 2009: Intro call, many random stories here.

Notes from Matthew Revell's and Karl Fogel's conversation with Jorge Castro on 2009-10-13. These were originally mailed to the launchpad-strategy@ list, but then the archives got blown away in the Great Archive Blow-Away of 2009. It's okay; the notes below are better anyway, as they're a bit more organized than the email was:

   People to speak to:

   Talk to Graham Binns, who has talked to Ubuntu team a lot.
   Brian Murray, Pedro V. (Ubuntu QA)
   Ken VanDine (Ubuntu Desktop team (integration engineer))
   Matthias Gug (Ubuntu Server Team)
   Leann Ogasawara (Kernel QA)
   David Planella (for translations stuff,
                   but note he just sprinted with Danilo et al)
   Bryce Harrington (Ubuntu X Windows)
   James Westby (esp re Debian connection)
   Steve Beattie (Ubuntu QA (Stable Release) team)

   Karl is meeting with Marjo Mercado (QA) on Thursday.

   Pedro and Ken are hugely about linking to upstreams.

   Jorge is already watching upstream linkages report:

   https://edge.launchpad.net/ubuntu/+upstreamreport
   ACTION: pull over time to track improvement/non-improvement
   Graham implemented what's there.
   Daniel has done some screen-scraping already on this
   "We want our developers marking bugs as upstreamable.  As that
    number goes up, we can add more Watches."
   Row goes green if over 90% of bugs upstreamed have a Watch.  We
   might want to go green based on percentage of total product bugs
   upstreamed instead!
   Right now it shows top 100 products with open bugs.
   Teams want team-specific reports, like server team, desktop team;
     or just for a group of packages (e.g. those handled by a
     particular MOTU), and so on.

   Pending bug watches -- "these are possible targets: bugs marked
   as possibly upstream but that do not have a link".
   https://bugs.edge.launchpad.net/ubuntu/+source/rhythmbox/+bugs?search=Search&field.status_upstream=pending_bugwatch
     
   Jorge would love stats on upstream rejects and the reasons for them;
   we don't have that now.

   ACTION: Brian Murray ran a query on who's forwarding bugs upstream;
           he got numbers per person, and it was a lot of non-Canonical
           people.
   ACTION: Hey, we don't have stats on who accesses these reports nor
           how often.  Wouldn't that be nice to know?
     
   Jorge uses these reports to push sub-communities to do things.

   No way to remove task from a bug! "open an upstream task on
   firefox.  A week later, ffox upstream says it's distro's fault.
   So distro opens up a null task -- that's their workaround."

   Yes, Jorge would love a way to mark a bug a "determined to be not
   upstreamable"!  (How can we measure it?)

   Daniel Holbach wrote http://daniel.holba.ch/harvest/handler.py?pkg=f-spot,
   which examines patches shipped by other distros for packages.
   FEATURE: track patches shipped by any distro for your package
            then click a button to integrate it into a Bazaar branch
            so I (upstream) can mess with it

   ACTION: Talk to Banshee to find out how they talk to Launchpad,
           since Jorge says they use it a lot even though they don't
           host there.  Another one is Tomboy (Sandy Armstrong);
           maybe shoot for that first -- he's worked with our
           Online Services people before.

   Jorge says areas he's been concentrating on bugs and patches.
   Also said James Westby did a little query that showed the
   differences between what we're shipping and what Debian's
   shipping.  E.g., how different is our Firefox from Debian's.
   Looking at diff^2 size turns out to be useless; you really have
   to look at the patches (it might be language packs, for example).
   Jorge wants to know what changes we're maintaining that we don't
   have to, because they could be either upstreamed or Debianed.
   ACTION: is there a way to measure this?  Or measure how long a
   patch sits in a package?

   ACTION: Jorge to forward Karl conversation about server patches
   we were carrying that it turned out Debian had.
             
   Process of upstreaming is ad hoc and manual, when sending to
   upstreams.  Debian has an automated query that they can do on a
   per-package basis (see
   http://packages.qa.debian.org/f/f-spot.html), but we don't have
   stats on how often they run that query nor how often they use the
   results.  (Lucas Nussbaum is the DD and UD who maintains this, btw.)

   Linking bugs upstream is still difficult.  Jorge did a little
   study on how many clicks it takes for someone to forward a bug
   upstream, and it was pretty bad.  ACTION: get details, or better
   yet ask Jorge to re-do the study, since Launchpad has changed in
   a year.

   Upstream Gnome Bugzilla (as Bugzillas get upgraded this will be
   in more places) now has a field for URL, where we put the URL
   field.  We want to automate this workflow, but automated only for
   people we trust (probably), to avoid spamming upstream.

   Jorge wishes that once he's determined that someone's contention
   that a bug is upstreamable is correct, that he had a nice AJAX-y
   way to send it upstream in one click.  Says he could get a lot
   more done if he had that.

   Even better, pre-empt bug reports!

   IMPORTANT:
   We have an import of the entire Debian bug database.  Jono is
   really keen on sucking in all these upstream bugs, and then using
   them as part of dup detection.  We could do that with other bug
   trackers as well.  Gnome, KDE, OpenOffice, Firefox, what others?
   Graham Binns was eager to work on this, too!

   ACTION: Karl and Matthew to call Deryck today.  (Note: I didn't get
   to this -- got Skype working instead.  Small triumphs, small triumphs.)

   ACTION: Plan a strategy sprint with Ubuntu people?
     
   For questions about Bazaar/codehosting and upstreams, talk to
   James Westby.

   Broken or obsolete VCS imports is a problem.  For example, old
   Gnome SVN -- but that's a case of "wrong" not "broken".
   ACTION: Do we have any procedure for autodetecting truly broken
           imports?
     
   Jorge working on "upstream ambassador" project, whereby someone
   could recommend a PPA to try when a package is having a problem
   but SRU means upstream doesn't have anything we can put into
   standard distro yet.

   Martin Pitt is working on this thing where if your Python project
   uses setup.py, then a one line command can create .deb (that you
   could then put in a PPA).

   Right now, when you upload a source package to a PPA, it will
   build for whatever release you say (Karmic, etc), for all archs.
   Would be nice to build Debian (e.g., sarge) packages.  Mark may
   have been against this; bigjools and/or jml might know for sure.

   ACTION: get one person from each team to sit down at UDS with
           jorge and mrevell and jml.  Schedule these soon.

   When / where is next UDS?  https://wiki.ubuntu.com/UDS-L exists,
   but https://wiki.ubuntu.com/UDS-M does not yet.

Strategy/RawData/JorgeCastro (last edited 2009-12-09 04:36:55 by kfogel)