Not logged in - Log In / Register

How to import bugs into Launchpad

Users almost always approach us via a question on Launchpad. Typically they provide a link to download their bug export, which must adhere to the import format. If they want the bugs imported into staging, then a LOSA has to run the script.

If they're using Trac they may have used the Trac to Launchpad Migrator to dump their XML. Bear this in mind if you encounter errors later on because there may be a bug (and a workaround) already filed about it, and you have somewhere to file one if not.


  1. Use jing to check the bug export against the schema:

    jing -c doc/bug-export.rnc downloaded.xml
  2. Use utilities/massage-bug-import-xml to check and possibly fix the bug export:

    utilities/massage-bug-import-xml \
      --project ${PROJECT_NAME} < downloaded.xml > massaged.xml
  3. Check that the massaged export has sensible nicknames:

    fgrep '<nickname>' massaged.xml

    These nicknames are very useful for migrating bug trackers. After the bugs have been imported, https://launchpad.net/bugs/NICKNAME will automatically redirect to the correct imported bug. This makes it easy for the project owners to set up a redirect from their old bug tracker to Launchpad.

    Note that these nicknames must be unique across all Launchpad bugs. If you suspect they won't be, massage-bug-import-xml accepts two switches that can help:

    --fix-nickname Normalize the nickname to ${project_name}-${bug-id}.
    --tag-nickname Add the original bug nickname as a tag.
  4. Import the bugs into a non-production instance of Launchpad. If you're a developer it's easiest to fire up a local instance, but there's no harm in using a staging environment.

    1. Ensure the librarian is running so that attachments in the bug export can be uploaded.

      For a development instance make run is the easiest way, because you'll want to check the bugs via the web UI later anyway.

    2. Create a new project into which to import the bugs:

      $ bin/harness
      >>> project = factory.makeProduct()
      >>> project.official_malone = True
      >>> print project.name
      >>> transaction.commit()

      Or do this through the web UI.

    3. Import the bugs:

      scripts/bug-import.py \
        --product product-name918523 massaged.xml \
          2>&1 | tee import.log

      Pay close attention to the import errors. Typically the log will link to librarian files which contain stack traces.

      All errors must be resolved. Often this will be an iterative process of asking the user to correct his or her bug export.

      Read about the cache file if you need to re-run the import for any reason, and update the bug nicknames so they remain unique:

      utilities/massage-bug-import-xml \
        --project project$RANDOM --fix-nickname \
          < massaged.xml > rerun.xml

      Then import rerun.xml instead of massaged.xml. Don't forget to use massaged.xml for production!

    4. Look at the bugs in the web UI and try to spot anything awry. Check that the totals on the project's bug page sound reasonable.

      If the imported bugs are all or mainly private, you won't be able to see them. Make yourself bug supervisor:

      $ bin/harness
      >>> from lp.registry.interfaces.person import IPersonSet
      >>> you = getUtility(IPersonSet).getByName(your_username)
      >>> from lp.registry.interfaces.product import IProductSet
      >>> project = getUtility(IProductSet).getByName(project_name)
      >>> project.setBugSupervisor(you, you)
      >>> transaction.commit()
    5. If you've imported into a staging environment you can ask the requesting user to take a look around too, but tell them that they have to look soon because the database gets refreshed regularly.

      If the imported bugs are all or mainly private, use the same procedure as for yourself to set the bug supervisor to the requesting user.

  5. If you've been doing this with a local, development, instance of Launchpad, attempt the import in a staging environment. The more realistic data in those envionments can uncover new issues to resolve.
  6. If the requesting user wants to have a quick switch from their old bug tracker to Launchpad you'll need to schedule a time to do this. They may also want to provide an export taken the moment after they freeze their bug tracker. You should still go through the steps above to sanity check for a clean import.

    Hint: The massaged XML files are pretty-printed, so a plain old textual diff can be useful for sanity checking too.

  7. Import into production.

    If the same or a very similar import has recently been done in to a staging environment you will not need explicit approval for this.

  8. Tell the requesting user the following:

    • If their bugs are not yet visible, how to configure their project to use Launchpad as the bug tracker (hint: the +configure-bugtracker page).
    • How the nickname redirection works for their project.

That's it: you're done. Hello? You look like you could do with a rest. Perhaps a cup of tea and sandwich wouldn't go amiss either...

About the cache file

A cache file, named bug-map.pickle by default, is always created by scripts/bug-import.py. It can be specified with the --cache option:

--cache=FILE Cache for bug ID mapping

This file is meant to make it easier to retry a failed bug import. It is used to see which bugs have already been imported so that they are not imported a second time.

This can be useful in production, though the uniqueness requirement of nicknames provides a safety net.

In a development environment it's better to retry the whole import again, into a new project. Remember to either remove this file before attempting another import, or specify a new cache file.

PolicyAndProcess/BugImportHowto (last edited 2011-11-08 15:33:55 by allenap)