= Trying out template generation on dogfood/staging =

Right now we have no good test platform for this: `staging` has no build farm, and `dogfood` has no codehosting.  Expect this page to change a lot as we work and learn.  We already have pages about [[Translations/GenerateTemplatesOnLocalBuildFarm|doing this on a development system]] that may also help.

[[https://wiki.canonical.com/Launchpad/Soyuz/CommonTasks|Internal Soyuz documentation on dealing with dogfood.]]


== Approach ==

For now, we'll have to live with a split test setup:
 * Push changes to generate jobs on `staging`.
 * Copy the jobs over to the `dogfood` database.
 * Let the `dogfood` build farm generate the templates.

== Known Problems ==

=== intltool recognition ===

It looks like gettext has changed, and our check for intltool branches now also accepts some plain gettext branches.  This happens with {{{lp:gawk}}}, which pottery mistakenly recognizes as intltool but doesn't process successfully because there is no {{{GETTEXT_PACKAGE}}} definition.


== Checklist ==

=== Suitable branches ===

Make sure you have a branch that is pottery-compatible.  (We call the code that generates templates from code "pottery").  For testing, it should normally ''not'' contain any `.pot` files.  Otherwise we'd just end up importing those instead of ones we generated.

All recently generated templates on the production server will be on [[https://translations.launchpad.net/~vcs-imports/+imports|the import queue for vcs-imports]], so that's the best place to look for fully pottery-compatible branches.

According to `wgrant`, on many branches, editing `configure.ac` to set `GETTEXT_PACKAGE` does the trick. See [[Bug:568355|bug 568355]].

You can easily check if a branch will work by generating a template locally. Run the following command from the LP tree in the packages root directory:

{{{
scripts/rosetta/pottery-generate-intltool.py
}}}

It outputs the template files it creates. It uses the same code as the build slave.

Candidate branches:
 * {{{lp:gedit}}}: used to work with pottery (and no pots in the branch), but as of 2010-08-27 it doesn't seem to work.
 * {{{lp:gimp}}}, but it's large.
 * {{{lp:ubuntu/gawk}}} does not use intltool, but pottery seems to think that it does. See [[Bug:568372|bug 568372]].
 * {{{lp:libgnomeui}}}: OK, one template
 * {{{lp:gconf}}}: OK, one template, no pots in the branch.


Once you have a suitable branch, you can copy it to staging's codehosting using "`bzr push -d lp:<branch> lp://staging/<branch> --use-existing-dir`"


=== Builders ===

There must be at least one suitable amd64 builder active.  "Suitable" would be a virtualized builder for production runs, but here we set our jobs up for running on nonvirtualized ones so we can get at the logs.  (It'll also make the tail of the log visible on the builder page).


== Testing: Creating jobs ==

On `staging` we can test the generation of jobs, though we can't run them there.

Do all this on `staging`:

 * Set up a productseries.
 * Set up a pottery-compatible branch, e.g. by copying one off production.
 * Set the branch as the productseries' development branch.
 * Enable template imports for the productseries.
 * Push a change to the branch.

Once your branch change is scanned, a triplet of matching records should appear in the `staging` database: a {{{BranchJob}}}, a {{{Job}}}, and a {{{BuildQueue}}}.  They are matched on {{{BranchJob.job}}}, {{{Job.id}}}, and {{{BuildQueue.job}}}.  In SQL it should be at the top of...
{{{
SELECT *
FROM BranchJob
JOIN Job ON Job.id = BranchJob.job
JOIN BuildQueue ON BuildQueue.job = Job.id
WHERE
    BranchJob.job_type = 6 AND
    BuildQueue.job_type = 4
ORDER BY Job.id DESC;
}}}

It probably has `status` set to 0 for "not started yet."


== Testing: Running jobs ==

We can create job records directly in the `dogfood` database, which can't generate the jobs itself but which can dispatch them to its build farm.

Do all this on `dogfood`:

 * Set up a productseries.
 * ''Register'' a branch on `dogfood`, but ''do not push it'' (as `dogfood` has no codehosting).
 * Set the branch as the productseries' development branch.
 * Enable template imports for the productseries.

Now we create the job records.  Run:
{{{
make -C /srv/launchpad.net/codelines/current iharness
}}}

And in that, run:
{{{
>>> import transaction
>>> from lp.code.interfaces.branchlookup import IBranchLookup
>>> from lp.translations.interfaces.translationtemplatesbuild import ITranslationTemplatesBuildSource
>>> branch, _ = getUtility(IBranchLookup).getByLPPath('gconf')  # or some other branch path
>>> _ = getUtility(ITranslationTemplatesBuildSource).create(branch).queueBuild()
>>> transaction.commit()
}}}

Keep an eye on https://dogfood.paddev.net/builders where it should appear shortly (and then, after processing, disappear again).  Also, if everything works, templates should appear on your productseries' import queue!