Diff for "Ubuntu/InfrastructureNeeds"

Not logged in - Log In / Register
Differences between revisions 1 and 28 (spanning 27 versions)
⇤ ← Revision 1 as of 2009-08-03 21:05:02 →
Size: 15006
Editor: kfogel
Comment: Bring over this page from internal wiki. Next change will be to update it for the present.
← Revision 28 as of 2011-11-16 23:37:01 → ⇥
Size: 8499
Editor: bryce
Comment:
Deletions are marked like this. Additions are marked like this.
Line 1: Line 1:
||<tablestyle="width: 100%;" colspan=3 style="background: #2a2929; font-weight: bold; color: #f6bc05;">This page discusses ways Launchpad could better serve the Ubuntu community. Please [[Help|contact us]] with thoughts or questions. ||

|| '''Launchpad contact''' || [[https://launchpad.net/~flacoste|Francis Lacoste]] ||
|| '''Ubuntu contact''' || [[https://launchpad.net/~bryce|Bryce Harrington]] ||
Line 3: Line 8:
 1. Monitoring of Debian bugs (shortcoming relative to former Bugzilla/debzilla infrastructure)

  * Rationale: Nearly all Debian bugs also affect Ubuntu, and Debian's larger base of developers can find and fix a higher volume of bugs. We should be informed of these bugs so that we can make a decision about whether to act on them.

  * Status:
   * A script exists to import bugs from debbugs into Launchpad ([[https://blueprints.launchpad.net/malone/+spec/debian-bug-import|DebianBugImport]])
   * In LP 2.0 we will have the ability to import comments from Debbugs, and send replies back to debbugs. A separate issue is whether or not we proactively import ALL debian bugs into LP, or whether we only import a subset of bugs (say, RC bugs), or whether we only import Debian bugs which have been manually correlated to Ubuntu bugs.
   * https://launchpad.canonical.com/MaloneLondonNotes200706#bugzilla-bug-pushing-dirt-cheap

  * Ideas:
   * Create non-Launchpad infrastructure to monitor bugs - Where to get notification addresses? Scrape launchpad?
   * MoM's bug notifications could be the basis of a partial solution
Line 17: Line 9:
  * Tracked at: '''???'''
Line 21: Line 13:
    * The "no upstream bug tracker exists" use case, which is described in https://blueprints.launchpad.net/malone/+spec/forwarding-to-email-address
    * Simplified/automatic filing of bugs in an external bug tracker
    * Comment synching to enable inter-bug-tracker replies		    * The "no upstream bug tracker exists" use case, which is described in '''https://blueprints.launchpad.net/malone/+spec/forwarding-to-email-address''' (Done?)
   * Comment synching to enable inter-bug-tracker replies (Done?)
   * Map remote bugtracker product/component to source package: '''https://dev.launchpad.net/LEP/BugzillaComponents''' (Nearly done)
   * Forwarding of selected attachments from LP bug to upstream - '''https://dev.launchpad.net/LEP/ForwardAttachmentsUpstream'''
   * Automatic user registration in remote tracker
   * Automatic fill-in of upstream bug report fields (title, description, priority, cc, etc.)
Line 25: Line 20:
 1. More XML-RPC APIs  1. Bug Q&A
   * Tracked at: https://dev.launchpad.net/Bugs/BugQ%26A
   * Status:
     * Is included on Roadmap - '''https://dev.launchpad.net/VersionFourDotO/Stories'''
Line 27: Line 25:
  * Rationale: many of our workflow and reporting concerns could be alleviated by the promised XML-RPC interface; since that would allow us to generate custom reports based on custom queries and information, or even create Python workflow programs. In particular, `bughelper` and `apport` require this functionality. apport attachments to an existing bug and guided bug filing could be implemented given good XML-RPC APIs.  1. Bug Bookmarking
   * Tracked at https://bugs.launchpad.net/bugs/193585
   * Status: Needs escalation to Launchpad
Line 29: Line 29:
  * Status:
   * Feeds and APIs are targeted to LP 2.0
   * LP now has APIs, although they are incomplete; further work needed in Ubuntu to identify and prioritise missing items		  1. Only Series Tasks
  * Tracked at:
    * https://dev.launchpad.net/LEP/OnlySeriesTasks
    * 20111103 - https://blueprints.launchpad.net/ubuntu/+spec/other-p-bugtaskseries
    * Bugs 314432, 777861, ...
  * Rationale: Many times bugs are specific to a particular version of Ubuntu, but this is not tracked by Launchpad. We work around this by tagging bugs with the release codename but this is done inconsistently (apport will include the tag automatically, but non-apport filed bugs don't). There are various bugs, features, and usability issues for which being able to segregate bugs by series would be a possible solution.
Line 34: Line 38:

 1. Rebuild testing in Launchpad
  * Rationale: Rebuild testing is currently performed using a dak instance. Failures must be processed manually by a buildd admin and the relevant developers notified. This requires maintaining redundant infrastructure and more manual work than should be necessary.

  * Status
   * Specified in LaunchpadWiki:Soyuz/FrequentRebuildTesting
   * https://rt.admin.canonical.com/Ticket/Display.html?id=26302
   * Targeted for LP 2.0
   * 2009-05-15: Largely done; test currently in progress at https://launchpad.net/ubuntu/+archive/test-rebuild-20090513 (summary at http://people.ubuntuwire.com/~wgrant/rebuild-ftbfs-test/). Remaining omission is that the rebuild archive is not published and therefore is unable to use its own output (https://rt.admin.canonical.com/Ticket/Display.html?id=34356).

 1. Web interface for syncing source packages in Soyuz, accessible to those with corresponding upload privileges

  * Rationale: "Syncing", i.e. copying source packages verbatim from one distribution to another, is a very common operation in Ubuntu, and is often requested by developers either after automatic syncs from Debian stop a couple of months into the release cycle, or when all changes have been incorporated into Debian and the Ubuntu changes may be discarded. At present, this operation is restricted to those with privileged shell access to the production archive. Conceptually, it should only require upload access to the corresponding component. Allowing uploaders to perform this task would save on the order of an hour a day of archive administrator time performing trivial requests. Syncing should also be possible from PPAs

  * Status:
   * Debian sources now available in Launchpad database, updated twice daily, making this more feasible
   * syncSource method available in API
   * Some problems with changelog presentation in web UI and in mails to -changes lists; https://bugs.launchpad.net/soyuz/+bug/55795 documents part of this

== Low priority ==

 1. Opening a new distrorelease before releasing the previous one (shortcoming relative to former dak infrastructure)

  * Rationale: When opening a new distrorelease, uploads must be temporarily blocked until the toolchain and other basic infrastructure are in place. Opening the new release early would allow this work to happen in parallel, so that the new release would be immediately open for development.
  * see https://bugs.launchpad.net/soyuz/+bug/87012

  * Status:
   * PPAs make it possible to do some of the preparatory work for a new distro series
   * Fully handling this is not targeted for LP 2.0

 1. Improved activity logging

  * Rationale: The current activity log in the Launchpad bug tracking system does not capture milestone changes. This presents a problem for release managers, who have no way to tell whether the milestone on a bug was set by somebody they trust, or whom to contact about it if the person who milestoned the bug omitted to leave a comment.

  * Note that milestones are not designed to be tightly controlled, we have tight control of release-critical bugs targeted to the specific series, where only drivers can approve the bug as RC.

  * Status: Apparently needs a [[https://blueprints.launchpad.net/malone/+spec/bug-history|rewrite of the activity log]].
  * https://bugs.launchpad.net/malone/+bug/65660 (Joey: Fix Released)
  * Not targeted for LP 2.0
Line 78: Line 43:
   * https://bugs.launchpad.net/malone/+bug/3882    * '''https://bugs.launchpad.net/malone/+bug/3882'''
Line 81: Line 46:
  * Tracked at: '''???'''
Line 84: Line 49:
   * original bug at https://bugs.launchpad.net/malone/+bug/81692
   * https://blueprints.launchpad.net/launchpad-registry/+spec/official-project-members covers this for Bugs and Answers
   * Launchpad claims this specification is related to the bug: https://blueprints.launchpad.net/launchpad-foundations/+spec/person-name-presentation
   * Not targeted for LP 2.0		    * original bug at '''https://bugs.launchpad.net/malone/+bug/81692'''
   * '''https://blueprints.launchpad.net/launchpad-registry/+spec/official-project-members''' covers this for Bugs and Answers
   * Launchpad claims this specification is related to the bug: '''https://blueprints.launchpad.net/launchpad-foundations/+spec/person-name-presentation'''
   * Not targeted in Launchpad 4.0

 1. Prohibit filing bugs on obsolete packages (Bug:46385)

 1. Subscribe Team to Tag, e.g. 'regression'

 1. Hiding comments or removing comments (Req'd by kernel team)

 1. Soyuz archive index
  * Tracked at: '''https://dev.launchpad.net/ArchiveIndex'''
  * Rationale: Req'd by Software Center

 1. Structured bug json data - aka "tags with values"

 1. PPA developer usability enhancements

== Low priority ==

 1. Opening a new distrorelease before releasing the previous one (shortcoming relative to former dak infrastructure)
  * Tracked at: '''https://bugs.launchpad.net/soyuz/+bug/87012'''
  * Rationale: When opening a new distrorelease, uploads must be temporarily blocked until the toolchain and other basic infrastructure are in place. Opening the new release early would allow this work to happen in parallel, so that the new release would be immediately open for development.

  * Status:
   * PPAs make it possible to do some of the preparatory work for a new distro series
   * Fully handling this was not targeted for Launchpad 4.0
Line 90: Line 79:
  * Tracked at: '''???'''
Line 94: Line 83:
   * Structural subscriptions are targeted for LP 2.0    * Structural subscriptions were targeted for Launchpad 2.0 (2009-08-03: update, anyone? Do we push notification now?)

 1. Search PPAs for version of app you want, for version of ubuntu you're on

 1. A new status for bugs between the in progress and fix committed, for when you fixed the bug and waiting for merge

 1. Ability to clone a bug

 1. Search across attachments
     * From previous discussion, sounds like this would be quite hard / resource intense

 1. Tarball visibility / navigation

== Undefined ==
Line 97: Line 99:
   * Not targeted for LP 2.0
Line 99: Line 100:
== Done! ==  1. Blueprints improvements:
  * Much work needed: 65922, 115158, 120942, 125377, 126522, 137397, 172532, 177519, 177520, 247672, 307495, 398604, 398605, 489288, 825523
  * Status: Has been proposed to merge blueprints and bugs?
     * If this work is schedule we need to understand how existing use cases will be transitioned. Need guidance and a plan.
Line 101: Line 105:
 1. Filing of bug reports with package metadata (cloakroom)  1. Answers needs either significantly improved, or scrapped in favor of just using AskUbuntu.com. Need guidance and a plan.
Line 103: Line 107:
 1. Copying of source and binaries between pockets (shortcoming relative to former dak infrastructure)  1. PPA improvements. Build status notification. Ability to host multiple versions of a given package (e.g. for bisection study purposes). Expose more of the internal API through the external Launchpad API.
Line 105: Line 109:
  * Rationale: Currently, packages are built and tested in `-proposed`, then re-uploaded to `-updates`, rebuilt and released to users. This process does not allow for the final build to be tested before being released to users, allowing the possibility of unexpected regressions after testing, and with large packages introduces an unnecessary build delay and mirror push. Additionally, this capability would allow packages to be copied from `-security` to `-updates`, allowing them to be mirrored and saving a substantial amount of archive bandwidth.  1. QA tracking. Some way to better flag bugs/branches/patches/etc. as needing testing, passed testing, failed testing, and so on. Ideally should also support hooking into automated testing in some fashion, with the goal of being able to delineate between code ready to be released from that needing additional work.
Line 107: Line 111:
  * Specified in LaunchpadWiki:Soyuz/NativeSourceSyncing  1. Some sort of integrated task tracking. Would replace the makeshift work-items currently done in blueprint whiteboards. Could also consolidate the use of team subscriptions, bug tasks, tagging, merge review requests, sync requests, and other makeshift task-workflow-management systems currently in use. 393117, 578263
Line 109: Line 113:
 1. Automatic closing of bugs via upload (shortcoming relative to Debian's infrastructure)  1. wiki-like functionality in various places throughout launchpad. E.g. PPA descriptions, project home/about pages, blueprints, whiteboards, etc. We need wiki-like markup (including tables), revision tracking, and easy cross-linking to other LP entities. 240067, 254167, 797331
Line 111: Line 115:
  * Rationale: Each bug must currently be closed manually in Launchpad after making an upload. Bug numbers are automatically extracted from changelogs and included in the .changes file, and making use of that information in Launchpad would avoid all of these manual steps. Debian has had such a feature since sometime in the 1990s and its developers generally consider this a useful invention.
  * Status: .changes already include a structured text field containing the list of bugs to close. What is needed is for this information to be acted upon by Launchpad.		 == Other stakeholder issues also relevant to Ubuntu: ==
Line 114: Line 117:
 1. Shorter turnaround for translating a new release  1. git support. Native git hosting.
Line 116: Line 119:
  * Rationale: Ubuntu 6.10 was not translatable until about 8 weeks prior to its release. Ubuntu 7.04 was not translatable until 5 weeks prior to its release. This does not leave enough time for Ubuntu to be well translated, and causes problems for upstreams attempting to translate in Launchpad (including frustrating bug reports about missing/incorrect Ubuntu translations).  1. [[https://dev.launchpad.net/VersionFourDotO/OutOfScope/Blueprints|Wiki markup in blueprints]] (OEM)
Line 118: Line 121:
  * Status: Ubuntu 7.10 translations opened 18 weeks in advance of release!  1. Task tracking in blueprints (aka [[https://dev.launchpad.net/VersionFourDotO/OutOfScope/Blueprints|Blueprint decomposition]]) (OEM)
Line 120: Line 123:
 1. Modeling Ubuntu bug workflow in Malone  1. Add a "workaround" field to bug (Bug:54652). (Corp Services)
Line 122: Line 125:
  * Rationale: The bug workflow employed in Ubuntu is not easily modeled in Malone, and the workflow used in Malone doesn't fit all of the needs of Ubuntu development. We believe that a small number of changes could make a big difference in closing this gap.  1. Launchpad doesn't support multiple attachment (Bug:82652). (Corp Services)
Line 124: Line 127:
  * Specified in UbuntuWiki:BugWorkflow See also: Launchpad's RoadMap and list of [[LEP|LEPs]].
Line 126: Line 129:
 1. Bug tracking for commercial repository ([[https://bugs.launchpad.net/bugs/58495|bug 58495]])

  * Rationale: Currently, bugs in packages distributed in the commercial repository cannot be properly filed in Launchpad because the packages are not included in the Launchpad database. This is because this repository is managed outside of Launchpad, by a dak instance.

  * Ideas:
   * Create a new 'commercial' component in Launchpad
   * Import package data from `-commercial` using gina
   * Manage `-commercial` in Launchpad entirely, using a PPA(?)

 1. Building unpublished source (shortcoming relative to former dak infrastructure)

  * Rationale: Source packages currently require at least two publisher runs in order to be published, because the source must be published before it can be built. There is no inherent reason for this, and critical fixes would be ready faster (especially near releases).

  * Specified in LaunchpadWiki:BuildUnpublishedSources
  * https://bugs.launchpad.net/soyuz/+bug/77853

 1. Web interface for Soyuz upload queue processing

  * Rationale: Processing upload queues currently requires privileged shell access to the production archive. The security implications of this (unrestricted write access to the archive) mean that we cannot effectively distribute certain tasks beyond distro team staff: for instance, we cannot delegate management of uploads to the universe and multiverse components to members of the community. This causes extra work and delays near release times.

  * Status:
   * A web interface to the queues is already [[https://launchpad.net/ubuntu/hardy/+queue|partly in place]], and it is possible to inspect uploads (although not as conveniently as it could be; see the package-diff item below).
   * It is not possible to override packages. Archive administrators without privileged shell access can now process the NEW queue to some degree by taking advantage of default component mappings. However, if any changes to the default overrides provided are required, a privileged administrator still needs to step in. A UI for overrides needs to be designed.
   * Targeted for LP 2.0

 1. Automatic diffs of new uploads, relative to the version being superseded

  * Rationale: We currently have no infrastructure for code review in Ubuntu. A system which automatically emailed diffs would provide a feed of changes which reviewers could read and comment on
  * Status: https://blueprints.launchpad.net/soyuz/+spec/package-diff

 1. Security updates in Launchpad (shortcoming relative to former dak infrastructure)

  * Rationale: Security updates are currently processed and built by a dak instance, then hand-fed into Launchpad through a manual process. This requires maintaining redundant infrastructure and limits developer participation.

  * Specified in LaunchpadWiki:Soyuz/SecurityPocketSupport

  * Status
   * Will be done by the end of 2007 (Kiko)
   * Targeted for LP 2.0

 1. Package-specific hints for reporting a bug
  * Rationale: Many packages have associated [[UbuntuWiki:DebuggingProcedures|written instructions]] in the wiki which explain how to debug problems, which are valuable to bug reporters and triagers. However, there is no link from Launchpad to this information, so it is only used when the user has prior knowledge of it. For triagers, this means an extra round-trip to ask the user to read the document and follow its instructions. Launchpad should contain a simple (1 paragraph) hint for a category of packages (desktop, xorg, kernel) and an optional link to the wiki page.
  * https://bugs.launchpad.net/malone/+bug/43893

  * Status:
   * Committed in RF 7300; will test once it lands in production
   * RF 7529 adds web UI, now available on edge (e.g. https://edge.launchpad.net/ubuntu/+source/man-db/+edit)
   * Unfortunately requires ubuntu-drivers to edit (https://bugs.launchpad.net/malone/+bug/315582)


 1. Support for attaching apport data to an existing bug

  * Rationale: Most bugs are reported directly to Launchpad, and so do not benefit from the automatic submission of system data to Launchpad. Some bugs are not reported from the system where the bug was observed (e.g., on servers).

  * Status:
   * Filed as bug [[https://bugs.launchpad.net/malone/+bug/85040|85040]], [[https://bugs.edge.launchpad.net/bugs/124338|124338]]
   * Under consideration for LP 2.0
   * Could be implemented by the distro given good APIs for Malone, and APIs are a LP 2.0 target		 [[/Archive|Historical infrastructure needs]]

This page discusses ways Launchpad could better serve the Ubuntu community. Please contact us with thoughts or questions.

Launchpad contact

Francis Lacoste

Ubuntu contact

Bryce Harrington

High Priority

  1. Semi-automatic bug forwarding to upstream bug trackers

    • Tracked at: ???

    • Rationale: Ubuntu receives a very high volume of bug reports which should be forwarded upstream. Making this process more efficient would improve both the quality of Ubuntu and its relationships with upstream projects.
    • Status: Some of the steps for this to happen are implemented. Additional work includes:

  2. Bug Q&A

  3. Bug Bookmarking
  4. Only Series Tasks
    • Tracked at:
    • Rationale: Many times bugs are specific to a particular version of Ubuntu, but this is not tracked by Launchpad. We work around this by tagging bugs with the release codename but this is done inconsistently (apport will include the tag automatically, but non-apport filed bugs don't). There are various bugs, features, and usability issues for which being able to segregate bugs by series would be a possible solution.

Medium Priority

  1. Temporarily adding the uploader as a bug contact for the package being uploaded
    • Rationale: Our development model is such that packages are often uploaded by a developer who has no ongoing relationship with the package. Because they do not receive bug reports for the package, it is easy for them to be unaware of having introduced a regression.
    • Status:
  2. Visual distinction between bug comments from authoritative Ubuntu people and bug comments from random Launchpad users

    • Tracked at: ???

    • Rationale: Users who view and file bugs in Launchpad are not always familiar with the way bug tracking works in a large community project like Ubuntu. When they receive a comment which is inappropriate, erroneous or poorly presented, they assume that it came from someone representing the project, when in fact anyone with an email address can post a response. Users who find these bugs via web searches have difficulty telling the difference between comments from users and authoritative information from developers and QA. We want to avoid this confusion and misrepresentation, while still allowing everyone to participate, by visually showing the user whether the commenter is a member of an official team (such as Ubuntu QA), perhaps by showing the team badge next to their name.
    • Status:

  3. Prohibit filing bugs on obsolete packages (46385)

  4. Subscribe Team to Tag, e.g. 'regression'
  5. Hiding comments or removing comments (Req'd by kernel team)
  6. Soyuz archive index
  7. Structured bug json data - aka "tags with values"
  8. PPA developer usability enhancements

Low priority

  1. Opening a new distrorelease before releasing the previous one (shortcoming relative to former dak infrastructure)

    • Tracked at: https://bugs.launchpad.net/soyuz/+bug/87012

    • Rationale: When opening a new distrorelease, uploads must be temporarily blocked until the toolchain and other basic infrastructure are in place. Opening the new release early would allow this work to happen in parallel, so that the new release would be immediately open for development.
    • Status:
      • PPAs make it possible to do some of the preparatory work for a new distro series
      • Fully handling this was not targeted for Launchpad 4.0
  2. Notifying the release team of new milestone targets

    • Tracked at: ???

    • Rationale: The release team tracks outstanding targets for milestones and their resolution. However, they currently must poll in order to obtain this information. Asynchronous notification would be more efficient.
    • Status:
      • Structural subscriptions were targeted for Launchpad 2.0 (2009-08-03: update, anyone? Do we push notification now?)
  3. Search PPAs for version of app you want, for version of ubuntu you're on
  4. A new status for bugs between the in progress and fix committed, for when you fixed the bug and waiting for merge
  5. Ability to clone a bug
  6. Search across attachments
    • From previous discussion, sounds like this would be quite hard / resource intense
  7. Tarball visibility / navigation

Undefined

  1. Package version tracking for bugs (shortcoming of Launchpad relative to debbugs)
  2. Blueprints improvements:
    • Much work needed: 65922, 115158, 120942, 125377, 126522, 137397, 172532, 177519, 177520, 247672, 307495, 398604, 398605, 489288, 825523
    • Status: Has been proposed to merge blueprints and bugs?
      • If this work is schedule we need to understand how existing use cases will be transitioned. Need guidance and a plan.

  3. Answers needs either significantly improved, or scrapped in favor of just using AskUbuntu.com. Need guidance and a plan.

  4. PPA improvements. Build status notification. Ability to host multiple versions of a given package (e.g. for bisection study purposes). Expose more of the internal API through the external Launchpad API.
  5. QA tracking. Some way to better flag bugs/branches/patches/etc. as needing testing, passed testing, failed testing, and so on. Ideally should also support hooking into automated testing in some fashion, with the goal of being able to delineate between code ready to be released from that needing additional work.
  6. Some sort of integrated task tracking. Would replace the makeshift work-items currently done in blueprint whiteboards. Could also consolidate the use of team subscriptions, bug tasks, tagging, merge review requests, sync requests, and other makeshift task-workflow-management systems currently in use. 393117, 578263
  7. wiki-like functionality in various places throughout launchpad. E.g. PPA descriptions, project home/about pages, blueprints, whiteboards, etc. We need wiki-like markup (including tables), revision tracking, and easy cross-linking to other LP entities. 240067, 254167, 797331

Other stakeholder issues also relevant to Ubuntu:

  1. git support. Native git hosting.

  2. Wiki markup in blueprints (OEM)

  3. Task tracking in blueprints (aka Blueprint decomposition) (OEM)

  4. Add a "workaround" field to bug (54652). (Corp Services)

  5. Launchpad doesn't support multiple attachment (82652). (Corp Services)

See also: Launchpad's RoadMap and list of LEPs.

Historical infrastructure needs

Ubuntu/InfrastructureNeeds (last edited 2011-11-22 03:34:55 by bryce)