* '''Launchpad entry:''' https://launchpad.net/launchpad/+spec/malone-and-blueprint
 * '''Created:''' <<Date(2006-05-11T06:41:52Z)>> by MatthewPaulThomas
 * '''Contributors:''' MatthewPaulThomas

Merging Bugs and Blueprints into a single tracker will make Launchpad more functional and easier to use, make scheduling easier for developers, simplify the codebase, and make Launchpad development faster. Bug reports, feature requests, and other to-dos can be created, processed, scheduled, and resolved the same way. This should not be implemented as a whole new tracker, but by porting missing features from Blueprints to Bugs, then rebranding and migrating. This specification covers (1) the tracker itself, (2) how representative projects would use it, and (3) how we will orchestrate the implementation and migration.

Our current top-level chronological plan:
 1. bug dependencies
 1. burndown charts
 1. work items
 1. nomination of bugs for sprints
 1. names?
 1. migration (data and socialization)

Some changes probably need to be chunked with visual design changes and data (e.g. bug status) changes.

<<TableOfContents()>>

== Rationale ==

Tracking bug reports and feature requests in separate systems makes tasks hard to report, hard to track, and hard to schedule, subtly discourages quality in projects that use Launchpad, and impedes development of Launchpad itself. These points are explained, with many examples, in [[/Rationale|the full rationale]].

== Research ==

We have carried out [[https://wiki.canonical.com/Launchpad/Strategy/Research/BlueprintBugMerger|interviews and statistical analysis]] of the projects that most use blueprints. (Unfortunately, because some of these projects are private, the data is also private. The following user examples capture the publishable details.)

== User examples ==

 * In '''Ubuntu''', people propose hundreds of blueprints for discussion at the next Ubuntu Developer Summit; that number is helpfully limited by the obscurity of the function. Managers [[https://launchpad.net/sprints/uds-n/+specs|approve]] or [[https://launchpad.net/sprints/uds-n/+specs?acceptance=declined|decline]] blueprints for UDS, renaming them to fit into semi-arbitrary [[http://uds.ubuntu.com/tracks/|tracks]]. The [[https://launchpad.net/summit|Summit]] scheduler arranges sessions based on who has registered their attendance and marked themselves as “essential” subscribers. During UDS sessions, participants assemble work items with other notes as plain text in [[http://en.wikipedia.org/wiki/Gobby|Gobby]], then up to a week or two later, transfer them to the blueprint whiteboard. Sometimes a blueprint links to a [[https://wiki.ubuntu.com/SpecSpec|specification]] on the Ubuntu wiki. The Unity Design team avoids blueprints, instead using [[https://bugs.launchpad.net/ayatana-design|a dummy project]] to track feature design separately from implementation in bug reports. For all other teams, Ubuntu drivers approve or decline the finished blueprints. If approved, a blueprint’s work items and linked bug reports appear on [[http://people.canonical.com/~platform/workitems/|burndown charts]]. As work items are updated during the cycle, Launchpad mails whiteboard diffs to the subscribers. If a feature takes more than six months, sometimes the blueprint is reused for the next UDS, and sometimes it is superseded.
  || <rickspencer3> jcastro, where can I see some organized lists of the blueprints registred to date for Natty?<<BR>>...<<BR>><jcastro> https://blueprints.edge.launchpad.net/sprints/uds-n <<BR>>...<<BR>><jcastro> the link at the bottom takes you to the "pending" ones people are submitting<<BR>>...<<BR>><jcastro> which is the link you'll want to check regularly<<BR>><rickspencer3> jcastro, can I see them listed by track?<<BR>><rickspencer3> also, I'd like to be able to list them by team if possible<<BR>><rickspencer3> jcastro, does stuff like this work: https://blueprints.edge.launchpad.net/sprints/uds-n?searchtext=*-foundations-* <<BR>><rickspencer3> ?<<BR>><jcastro> rickspencer3: clicking on the column sorts them by name, which is are the tracks<<BR>><jcastro> yes, it does<<BR>>...<<BR>><jcastro> rickspencer3: as long as you stick to the naming convention it should be fine||

 In the new tracker, people (would this need restricting?) could nominate any issue to be discussed at UDS. (Would the sprint tracker still be part of Launchpad?) Drivers could approve or decline issues for UDS based not on whether they were “features”, but on whether they were complex enough to need in-person discussion. During a UDS session, participants would update the issue description directly and add/remove work items. Drivers could approve or decline issues and/or individual work items for the release, and allocate them to milestones if desired. Launchpad itself would display burndown charts for the Ubuntu series as a whole, for milestones, for teams, and for individuals.

 * '''Linaro''' processes are complex but their presentation is in flux. As with Ubuntu, features are proposed, and the [[http://www.linaro.org/how-linaro-works/|Technical Steering Committee]] approves or declines them, for discussion at six-monthly [[https://wiki.linaro.org/Process/ProjectManagement|developer summits]]. Overall changes targeted for a release are described as [[https://wiki.linaro.org/Process/linaro1011ReleaseCycle#References|technical requirements]] ([[https://wiki.linaro.org/Releases/1011/TechnicalRequirements|example]]). [[https://wiki.linaro.org/Process/Blueprints|Blueprints]] representing these have a `tr-` prefix, sometimes depending on other blueprints from both Linaro and Ubuntu. As with Ubuntu, [[https://wiki.linaro.org/Process/linaro1011ReleaseCycle#Planning|blueprints have work items tracked on burndown charts]], and also on [[https://wiki.linaro.org/WorkingGroups/KernelConsolidation/Status|elaborate external dashboards]] that track both definition status and implementation status. [[http://people.canonical.com/~jamesw/workitems/natty.old/|Separate charts]] track [[https://wiki.linaro.org/Process/ReleaseManagement|weekly]] progress for each technical requirement. Project members are currently debating whether to request [[LEP/EffortEstimation|fields for person-hours estimated vs. expended]] in Launchpad for more precise tracking.

 In the new tracker, people could nominate issues for discussion at an LDS. The Technical Steering Committee could have a [[http://launchpad.net/bugs/193585|team-controlled “tr” tag]] for marking issues as technical requirements, so that they could be displayed as a separate list, and those issue reports could depend on other issue reports that listed the actual work items. Launchpad could show overall progress of the release, but progress of technical requirements would still need to be displayed using an external tool. Estimated person-hours could be entered in Launchpad either for a whole issue, or for its individual work items.

 * '''Drizzle''' heavily uses both work items and milestones, but [[http://drizzle.org/workitems/|its “burndown chart”]] does not track progress towards a milestone.

 In the new tracker, contributors could easily see a burndown chart for each milestone, as well as an issue status chart for the project as a whole. Work items would be entered inside issue reports.

 * The '''Unity design team''' does not use blueprints. It assesses incoming Unity bug reports, and if they need design work, tags them as “UDT” and files them as “Confirmed” for [[https://launchpad.net/ayatana-design|Ayatana Design]]. A report bounces between “In Progress” and “Triaged” for whatever kinds of design work (interaction, visuals, animation) are needed. When design is complete, the report is marked “Fix Committed” for Ayatana Design, and “Confirmed” for [[https://launchpad.net/unity|Unity itself]]. Once “Fix Released” for Unity, it is (at least in theory) tagged as “needsdesignreview” and then “reviewedbydesign”.

 In the new tracker, the Ayatana Design project would be retired. Instead, a Unity issue report would be marked as “Ready” (a.k.a. “Triaged”) only when the desired behavior was either (a) obvious or (b) included in a specification linked to from the report. Unity managers or engineers could [[#Delegation|delegate]] an issue to whichever designers or testers were necessary.

 * '''A small project''' typically uses blueprints sporadically, if at all. When it does, Definition status is used to track whether the idea is ready to implement. Releases are made when they are ready, so there is no date for a burndown chart to reach.

 In the new tracker, the “Ready” status would be used to track whether an idea is ready to implement.

 * Various hardware vendors publicly contribute to Ubuntu core packages. The same bug, in the same package, may be of different importance to each of these vendors.

== The tracker ==

Bugs and Blueprints should be merged into Issues, a unified application for handling bug reports, feature requests, and other to-dos. Projects should be able to choose what kind of issues they want to use the issue tracker for, without having to learn separate applications.

=== Status ===

Issues should have six possible statuses.
 * '''Unconfirmed:''' Not yet investigated by someone qualified to determine its coherence.
 * '''Declined:''' Not appropriate or relevant for this project.
 * '''Incomplete:''' May be appropriate, but is missing information about the precise problem — for example, reliable steps to reproduce, a testcase, or hardware details.
 * '''Ready:''' The issue is described well enough, and is approved if necessary, so that work can begin. (''Cf.'' [[http://wiki.mozilla.org/BugzillaWorkflowImprovements|BugzillaWorkflowImprovements]].)
 * '''In Progress:''' Actively being designed, implemented, or fixed. (Some projects and/or assignees may not bother with this status.)
 * '''Done:''' Fixed, implemented, or achieved. Any unresolved issues have been recorded separately. For versioned things (such as software), the version/versions in which the issue is resolved should be recorded.

Issues marked as "Declined" or "Done" should continue appearing in search results for a period afterward (defaulting to six months, but configurable per-project to suit the typical upgrade cycle of the project's users). This would perform almost exactly the same duplicate-reducing function as "Fix Committed" was intended but failed to do, and as "Won't Fix" does, but in a much simpler way.

Data migration:
 * “New” -> “Unconfirmed” (because “New” is misleading)
 * “Invalid” -> “Declined” (because [[https://lists.canonical.com/archives/launchpad-users/2007-September/002289.html|“Invalid” is needlessly harsh]])
 * "Won't Fix" -> "Declined"
 * “Opinion” -> “Unconfirmed” (because some triagers have used this for things that the project maintainer hasn’t even considered)
 * “Confirmed” -> “Unconfirmed” (superseded by users-affected count)
 * "Triaged" -> "Ready" (because the name "Triaged" is both misleading and over-specific)
 * "Fix Committed" -> "Done" with "''context-target-name''-committed" tag (for easing migration of projects that were purposefully distinguishing between Committed and Released)
 * "Fix Released" -> "Done" with "''context-target-name''-released" tag

=== Importance ===

Issues should have the same set of importance values as Bugs had previously, except without Wishlist. Whether something is a wishlist item is orthogonal to how important it is.

=== Delegation ===

For effective issue tracking, it is important for ''all'' issues assigned to someone to represent things they are able to work on right now. When an issue is blocked on someone else, it should appear in that person's task list, and be downplayed in your own. Therefore Launchpad should know about ''delegation'', or temporary reassignment.
 * A specification in the Ready state may be delegated to drivers for approval, before being returned for implementation.
 * A smaller bug fix that affects the user interface may be delegated to a UI designer for sign-off, before being returned for implementation.
 * A code change may be implemented and then delegated to the review team for review, before being returned for committal.
 * Close to a major release, a reviewed code change may be delegated to a drivers team to gauge risk vs. reward, before being returned for committal to the release branch and/or the trunk.

=== Work items ===

As a workaround for poor tracking in Blueprint, the separate [[https://wiki.ubuntu.com/WorkItemsHowto|work items tracker]] has been grafted on top for large projects (Ubuntu, Linaro, Drizzle), with burndown charts. Work items are entered, and edited, as lines of text in blueprint whiteboards. Bug reports are (at least currently) too much work to use as work items. Any combined issue tracker will need to satisfy the same requirements.

=== No dependencies ===

Some bug and issue trackers include dependency tracking. Unfortunately dependencies are used to express a variety of different relationships:
 * Work on X can't begin until Y is finished.
 * Work on X can begin, but can't finish until Y is finished.
 * X could be finished without Y being finished, but the software wouldn't make sense to users in that state.
 * X might still be an issue, but it's difficult to tell because of Y.
 * Fixing Y would be one way of fixing X, but not the only way.
 * X is divided into two tasks, Y and Z.
When people assume that all dependencies are of the first sort, development is slowed unnecessarily.

[[http://fogcreek.com/FogBugz/blog/post/FogBugz-and-Dependencies.aspx|FogBugz doesn't record dependencies, because]] "on software teams, it's almost always the case that the team can keep working even when there's a dependency. Software developers have a great deal of flexibility in the order in which things are done, unlike, say, a construction crew building a house. Because it's so easy to create simple stub functions as placeholders, with software, it's perfectly reasonable to build the roof before the foundation is poured."

There are also other interesting relationships between issues, that cannot reasonably be tracked as dependencies:
 * If X was fixed, Y would become invalid.
 * Y is similar to X, and testers should be careful not to confuse them.
 * Y is a regression of X.
 * Y is the unfixed remainder of X.

With all these relationship types, the benefit of having a specific field for them in Launchpad is outweighed by the cost of the extra complexity. And this applies exactly as much to feature requests as it does to bugs. Therefore issue relationships should be recorded using free-form text in the issue's description, with Launchpad automatically linking issue numbers as it does now for bug numbers.

Data migration: Once blueprints are assigned issue numbers, blueprint dependencies should be converted to sentences at the end of the issue description.

== Implementation ==

'''This would not be implemented as a whole new tracker.''' The overall implementation plan would be:
 * port missing features from Blueprints to Bugs, rolling them out as ready
 * implement the rebranding of "Bugs" to "Issues" and the hiding of "Blueprints"
 * implement migration of all current blueprints to issue reports
 * simultaneously roll out the rebranding and perform the migration
  * possibly the migration can be done in the hours after the rollout
 * once it's all working, implement the removal of the dead code and DB records.

=== Code changes ===

=== Schema changes ===

=== Migration ===

 * blueprint "names" become issue nicknames
  * requires uniqueifying nicknames
 * bugs with "Wishlist" importance reset to "Undecided" for proper triage
 * URL redirects (`https://launchpad.net/bugs/nnn` should continue to work)
 * "Blocked" should be represented by an unresolved dependent issue

== Unresolved issues ==

 * What to do with "informational" specifications?
 * [[http://launchpad.net/bugs/136103|Bug 136103: Suggestion about bug types]]
 * [[http://launchpad.net/bugs/176431|Bug 176431: Better organization/segregation of wishlist items]]
 * Could "Declined" and "Done" be merged into "Closed"? Is there a functional distinction between the two? For example, if a bug is reported about a feature that is later removed, should the bug be considered invalid or fixed? Google Code Hosting makes no distinction by default. And Ubuntu package changelogs "Close" bugs, rather than fixing some and invalidating others.
 * How should "Does not affect software X" (or "Does not affect software X version Y") be represented?

=== Robert(TA)'s issues ===

 * "Therefore issue relationships should be recorded using free-form text in the issue's description, with Launchpad automatically linking issue numbers as it does now for bug numbers. " Seems an unjustified conclusion. We -are- doing bug relationships with semantic values, for other reasons; why shouldn't this build on that?

 * There seems to be no space for a confirmed unapproved issue, nor for an issue which has been triaged but is not ready to go. (Triaged as something inferred by 'has a priority and is confirmed' would be fine, but I can't tell if thats intended). Specifically, how are we expecting projects to reflect something that is real but not very important to the project? How would the project get that off their radar without declining it or doing -more- work to get it to ready? Currently Launchpad does this by triaging it: assigning an importance (of Low). 'Ready' -does not- reflect most of the triaged bugs in the Launchpad project's bug tracker. Doing 'just enough' work on an item to get it out of the way defers doing stuff up-front that may never be needed - thats the whole point of triage (which point Ubuntu currently fails at because there is nothing between Triaged and In Progress).

 * The section on work items seems incomplete - it says we will need them to meet some spec, but not what means.

== Notes ==

''Transcription of jml's notes made at the Launchpad Thunderdome 2011-01-19''

 * Want bugs and blueprints on a single list
 * Need to have "work items" for issues that can be burned down for a release
 * Need to be able to say that issue is not finished until a set of issues and work items are finished
 * Want "issue cannot be started until this other issue is finished"
 * Want "issue is related to other issue"
 * Need to have separate queue / process for "features" instead of bugs
   * Need to change smoothly during course of their life
   * Need to avoid / discourage flag fights
   * e.g. How do you decide that something needs approval? How do you track that approval? e.g. LEP.
 * Must avoid swamping UDS planners with bugs proposed  for discussion
 * Work items currently have DONE, INPROGRESS, TODO and POSTPONED statuses
 * See [[http://people.canonical.com/~james-w/workitems/natty.old/|Linaro charts]]
 * https://bugs.launchpad.net/launchpad/+bug/655385