Archive index

We want to replace app-install-data and related packages with an automated system. By March 2010, Launchpad should be producing for every archive it controls — including Multiverse, Canonical Partner, and every PPA — an index of names, icons, summaries, categories, and keywords for all software items in the archive. A “software item” in this sense mostly corresponds to a binary package, but in some cases one binary package contains multiple applications that should have separate information. Launchpad should put this index in a standard place in the archive, and rebuild it whenever a package is added or changed.

Rationale

app-install-data is something we want to get rid of:

• it's slow and manual to update
• the time we want to update this package is when the archive is frozen
• there are exceptions and bugs, so software shows up in Ubuntu Software Center that isn't installable
• it's difficult to update the data
• it works only for Main and Universe, not for PPAs or other archives

User stories

• Maree maintains 16 packages in a PPA. She wants these packages to show up, with proper names, icons, departments etc, in the Ubuntu Software Center for anyone who adds her PPA.
• Brian has packaged a new version of Adobe Reader and published it to the Canonical partner repository. It has a different icon from the previous version.
• Ben uses apt-get to install everything. He isn't interested in downloading icons etc for applications he is never going to install.

File format

• Opaque to apt, so doesn't really matter
• RFC-822
• Localization of categories and keywords (in the translations file?)

Soyuz process

• Strip the data out of the package when building it, store it somewhere
• Publish the metadata file along with everything else

Roadmap

• Maybe start with non-localized data for PPAs in Lucid, then localized in a future version?
• Maybe still have Ubuntu Software Center using appnstall-data instead for Main and Universe in Lucid
  • iteratively survey the differences between app-install-data and the metadata Soyuz is producing
    • fix bugs in the packages and/or in Soyuz

actions:

• - client: write scripts to extract the needed data to LP - client: provide examples what the file

Issues

• Often a .desktop file is in a separate package from the package you're actually interested in
  • e.g. wesnoth-data vs. wesnoth
  • e.g. emacs-common contains the icon for emacs22
  • maybe this should be fixed in the packages themselves
• Debian may or may not be interested this
  • e.g. keeping packages and debtags in sync
• If bulk of metadata is not in Packages file then we can be nicer to Launchpad
• Filling the Librarian with icons is necessary but annoying
  • garbage-collect them when done?
• Current list view description translations should be migrated from app-install-data (currently ca. 4000 strings)
  • [long description translations come from DDTP data in the archive, but list view's short descriptions come from .desktop file's Comment field which is translated in app-install-data's Rosetta template]

Two parts:

• 1) LP export 2) apt support for downloading this

LP:

• what to export?
  • per arch, per pocket (main, universe)

  • one pkg -> multiple apps, app names are not uniq

  • desktop data parts
    • appname
    • packagename
    • Comment (friendly summary) - multi language
    • popcon / rating
    • keywords
    • iconname
    • Categories
    • mime-type
  • command not found data
    • per arch, per pocket (main, universe)

    • packagename -> binaries

    • PROBLEM diverts etc, real world problem
  • icons ?
    • uuencode?
    • gkticoncache ?
    • size?
    • format?
  • tagfile/rfc822 just like Packages
• how many files?
  • command-not-found
  • software-center
  • icons
• we need to hook into the build process to extract the desktop file, command not found file data, icons etc (problem is diverts)

Initial Implementation

• Put icons into repository
  • options: big tarball or gtk-icon-cache
• something needs to go once the build is finished and extract the .desktop file and icons, then export that metadata
  • setup another script once build is finished
• need to create a standard for what kind of metadata can be supplied within the package
  • debian/control modifications
  • debian/something.desktop with X-App-Install tags
  • debian/something.something for command-not-found hints
• Things this metadata might contain:
  • icons
  • package descriptions (also translated)
  • restart required
  • not screenshots/movies/sounds (handled elsewhere as not downloaded up-front in software center)
  • mimetype
  • hardware/software requirements (opengl etc)
  • whether it's available in my language (may change after package upload since translation packages are different)
    • similar problem to ratings updates
  • keywords and keyword translations
• we have information that can be extracted from a upload (icon)
• and changes that happen after a upload (rating)

Launchpad team requirements

Needs doing by beta 1, or not in Lucid at all.

The Launchpad team needs from the Ubuntu Software Center team:

• File format for the metadata
• TODO - software center team: The code to inspect Debian packages and extract the metadata