#format rst
=====================================
  How to import bugs into Launchpad
=====================================

Users almost always approach us via a question on Launchpad. Typically
they provide a link to download their bug export, which must adhere to
the `import format`_.  If they want the bugs imported into staging, then
a LOSA has to run the script.

.. _import format: https://help.launchpad.net/Bugs/ImportFormat

If they're using Trac they may have used the `Trac to Launchpad
Migrator`_ to dump their XML. Bear this in mind if you encounter
errors later on because there may be a bug (and a workaround) already
filed about it, and you have somewhere to file one if not.

.. _Trac to Launchpad Migrator: https://launchpad.net/trac-launchpad-migrator


Steps
-----

#. Use ``jing`` to check the bug export against the schema::

     jing -c doc/bug-export.rnc downloaded.xml

#. Use ``utilities/massage-bug-import-xml`` to check and possibly fix
   the bug export::

     utilities/massage-bug-import-xml \
       --project ${PROJECT_NAME} < downloaded.xml > massaged.xml

#. Check that the massaged export has sensible nicknames::

     fgrep '<nickname>' massaged.xml

   These nicknames are very useful for migrating bug trackers. After
   the bugs have been imported, https://launchpad.net/bugs/NICKNAME
   will automatically redirect to the correct imported bug. This makes
   it easy for the project owners to set up a redirect from their old
   bug tracker to Launchpad.

   **Note** that these nicknames *must be unique* across all Launchpad
   bugs. If you suspect they won't be, ``massage-bug-import-xml``
   accepts two switches that can help:

   --fix-nickname        Normalize the nickname to ${project_name}-${bug-id}.
   --tag-nickname        Add the original bug nickname as a tag.

#. Import the bugs into a non-production instance of Launchpad. If
   you're a developer it's easiest to fire up a local instance, but
   there's no harm in using a staging environment.

   #. Ensure the librarian is running so that attachments in the bug
      export can be uploaded.

      For a development instance ``make run`` is the easiest way,
      because you'll want to check the bugs via the web UI later
      anyway.

   #. Create a new project into which to import the bugs::

        $ bin/harness
        ...
        >>> project = factory.makeProduct()
        >>> project.official_malone = True
        >>> print project.name
        product-name918523
        >>> transaction.commit()

      Or do this through the web UI.

   #. Import the bugs::

        scripts/bug-import.py \
          --product product-name918523 massaged.xml \
            2>&1 | tee import.log

      Pay close attention to the import errors. Typically the log will
      link to librarian files which contain stack traces.

      **All errors must be resolved**. Often this will be an iterative
      process of asking the user to correct his or her bug export.

      Read `about the cache file`_ if you need to re-run the import
      for any reason, and update the bug nicknames so they remain
      unique::

        utilities/massage-bug-import-xml \
          --project project$RANDOM --fix-nickname \
            < massaged.xml > rerun.xml

      Then import ``rerun.xml`` instead of ``massaged.xml``. **Don't
      forget** to use ``massaged.xml`` for production!

   #. Look at the bugs in the web UI and try to spot anything
      awry. Check that the totals on the project's bug page sound
      reasonable.

      If the imported bugs are all or mainly private, you won't be
      able to see them. Make yourself bug supervisor::

        $ bin/harness
        ...
        >>> from lp.registry.interfaces.person import IPersonSet
        >>> you = getUtility(IPersonSet).getByName(your_username)
        >>> from lp.registry.interfaces.product import IProductSet
        >>> project = getUtility(IProductSet).getByName(project_name)
        >>> project.setBugSupervisor(you, you)
        >>> transaction.commit()

   #. If you've imported into a staging environment you can ask the
      requesting user to take a look around too, but tell them that
      they have to look *soon* because the database gets refreshed
      regularly.

      If the imported bugs are all or mainly private, use the same
      procedure as for yourself to set the bug supervisor to the
      requesting user.

#. If you've been doing this with a local, development, instance of
   Launchpad, attempt the import in a staging environment. The more
   realistic data in those envionments can uncover new issues to
   resolve.

#. If the requesting user wants to have a quick switch from their old
   bug tracker to Launchpad you'll need to schedule a time to do
   this. They may also want to provide an export taken the moment
   after they freeze their bug tracker. You should still go through
   the steps above to sanity check for a clean import.

   **Hint:** The massaged XML files are pretty-printed, so a plain old
   textual diff can be useful for sanity checking too.

#. Import into production.

   If the same or a very similar import has recently been done in to a
   staging environment you will *not* need explicit approval for this.

#. Tell the requesting user the following:

   * If their bugs are not yet visible, how to configure their project
     to use Launchpad as the bug tracker (hint: the
     ``+configure-bugtracker`` page).

   * How the nickname redirection works for their project.

That's *it*: you're done. Hello? You look like you could do with a
rest. Perhaps a cup of tea and sandwich wouldn't go amiss either...


About the cache file
~~~~~~~~~~~~~~~~~~~~

A cache file, named ``bug-map.pickle`` by default, is always created
by ``scripts/bug-import.py``. It can be specified with the ``--cache``
option:

  --cache=FILE          Cache for bug ID mapping

This file is meant to make it easier to retry a failed bug import. It
is used to see which bugs have already been imported so that they are
not imported a second time.

This can be useful in production, though the uniqueness requirement of
nicknames provides a safety net.

In a development environment it's better to retry the whole import
again, into a new project. Remember to either remove this file before
attempting another import, or specify a new cache file.