Launchpad Production oops-tools setup

The Launchpad instance of OOPS-Tools is run on devpad under the lpqateam role account.

OOPSes are transported via rsync and (in a growing number of cases) AMQP.

Pruning is done on devpad via the datedir-repo bin/prune tool; no pruning is done (ATM) of db records. See crontab of lpqateam to find the pruner. There is not automation etc of its deployments as yet. Pruning is done against launchpad-project - things not in that project group will not have their oopses preserved.

oops-tools deployment

TL;DR: cd /srv/lp-oops.canonical.com/cgi-bin/lpoops && ./deploy.sh

To deploy a new version of LP's oops-tools instance:

ssh devpad
sudo su - lpqateam
cd /srv/lp-oops.canonical.com/cgi-bin/lpoops
pkill amqp2disk
bzr up

If there are changes to the models and/or you need to do data migration (a new migration has been created) then run:

bin/django migrate

Finally activate the new code:

• Ask a LOSA to graceful apache

• Run ./run-amqp2disk.sh to start the OOPS AMQP listeners again. (That script runs three consumers, one per LP instance, writing to /srv/oops-amqp/$instance - e.g. /srv/oops-amqp/production).

cronjobs

There are some cronjobs setup in the lpqateam crontab.

• update_db: script that loads oops reports from the file system into the oops-tools database.
• dir_finder: script that searches the file system for new oops directories and build the cache used by the update_db script to look up for oopses.
• report: script that saves a daily aggregation of OOPS report that happened in the previous day. It also emails the report to the launchpad mailing list.

Loading new prefixes

New prefixes are automatically accepted. Once an oops from one has been seen they need to be assigned to reports within oops-tools.

Deploying locally (e.g. devpad)

Follow src/oopstools/README.txt to get an instance up and running.

Use bin/amqp2disk --host XXXX --username guest --password guest --output . --queue oops-tools --bind-to oopses to setup and bind to a new exchange on your local rabbit for experimenting with Launchpad. See my notes for more info.

Admin

Creating new reports

To create a new report which will be sent to the LP mailing list daily go to: https://lp-oops.canonical.com/admin/oops/report. If you don't have access run bin/django createsuperuser.

There you can add a new report or change the existing ones.

Each report is composed by a name, the title of the report which will be used in the email sent to the list, a summary type, which know how to group the OOPSes and render the html and txt output and prefixes, which are the OOPS prefixes in the given report.