launchpad development
Launchpad Translations API
This Launchpad enhancement proposal discusses extending the functionality of Launchpad Translations by developing an API which will expose the necessary interfaces to allow read and write access to translations-related data from Launchpad through launchpadlib.
On Launchpad: https://blueprints.launchpad.net/rosetta/+spec/api
As the Ubuntu Translations Coordinator
I'd like to propose the development of a Translations API
so that the Ubuntu Translations community and other customers can benefit from accessing the Launchpad Translations data in alternative ways. These will help them assessing the status of translations at the distro level, track progress and set translation goals which will improve the translation coverage of Ubuntu. In addition to that, other functionality provided by the API can assist the Ubuntu translations coordinators in managing the Ubuntu Translations import queue and automate the currently manual processes
Rationale
Due to other higher priority development work, the Translations component has been lacking a full API for quite some time now.
The need of an API arises now from the necessity of better reporting the status of Ubuntu translations. The main benefit accessing data through launchpadlib can provide to Ubuntu is on the statistics reporting side: currently the main access point to translating Ubuntu is at http://translations.launchpad.net/ubuntu. This view offers statistics to all translatable packages in the main and restricted repositories, regardless of distribution. While complete, it does not provide information on how well translated Ubuntu is as a distribution. This information can be very useful to Ubuntu translators in assessing what needs to be translated and on which packages they should concentrate their work. This can also be used to track progress, set up goals, and in short, as an aid to building community around translations. In the same way, this can be useful for commercial customers to assess translation coverage when evaluating to ship Ubuntu in a given language.
As the current focus of Launchpad development is on upstream integration, and as there is no planned UI work in that area, an API appears to be the most adequate way to allow accessing the Launchpad Translations data and present it in alternative ways to suit the stakeholders' needs.
This is only one of the uses the API would enable (reporting), but there are other functionalities (import queue management, translation) which would not only greatly reducing the amount of work in managing Ubuntu translations, but also open up the Translations component to many interesting uses.
Stakeholders
Ubuntu Translations Community: David Planella, Adi Roiban
Ubuntu Desktop Team: Arne Goetje
Canonical OEM team: Kyle Nitzsche
(David: as the drafter of this proposal, I'm regularly in touch with all of the stakeholders)
Constraints
The new behaviour must provide a way to access translations data from Launchpad, either in read (or depending on the value) or write mode through launchpadlib, without the need for an admin to query the database.
It must not provide a way to circumvent any security privileges built in the layers above (e.g. Launchpad UI)
Subfeatures
The API can be initially subdivided in several areas, which will be exposed separately below.
Reporting
This part of the API focuses on exposing read-only data for the purpose of reporting on the status of translations. Each subsection treats a separate aspect of reporting, with DATA lists as the proposals for the data to be exposed.
Languages
Retrieving the list of all languages available in Launchpad. For each language the following data would be required:
DATA:
- Language code
- Language name in English
- Language state (visible or not)
- Number of contributors
Plural forms [optional, for checking language support]
Text direction [optional]
The rest of the data at https://translations.launchpad.net/+languages/$LANG [optional]
It would be also desirable to have the ability of querying a single language as well.
PO Templates
Retrieving the list of all templates available for a series in Launchpad. For each template the following data would be required:
DATA:
- Template name in Launchpad
- Priority
- Length
- Date last updated
- Domain name
- Iscurrent (active or not)
- Source package associated
- Is it used in language packs
- The list of POFiles associated with this POTemplate
In which languages there is a translation available [optional]
It would also be nice to be able to query a template from a series, based on the Launchpad template name.
PO Files
Retrieving the list of all PO files for a template in a series.
DATA
- Language code
- Length
- Number of translated messages
- Number of unreviewed messages
- Number of changed messages
- When the translation was last updated
- Who the last translator was
Import queue
This part of the API focuses on exposing read and write data for the purpose of automating the management of the imports queue.
On a higher level, this is the required data for this proposal
- Ability to return all queue entries for a given series.
Similar to the already implemented getAllEntries call, but with the possibility of specifying a series and return only the entries for that one
For each import queue entry, the following data would be required:
DATA:
- File type
- Template path
- Source package
- Template name
- Translation domain
- Is included in language packs
- Ability to set the status of the entry (approved, blocked, needs review, etc.)
Note: most of these are already exposed
Translation
TODO: need to add more data or uses which would be interesting for translations.
- Uploading translations
- Downloading suggestions for a given PO File
- ...
Workflows
Danilo's thoughts on structuring the work for the reporting part:
Exposing API for single PO file statistics - a shared interface IRosettaStats will need to be exported through API, so all the other objects which are based on it will get the same API as well.
Devising a good API for a single PO template
- Other than the simple properties, the most interesting bit here is how to get statistics for a bunch PO files attached to a template. If you don't do anything but just export a single method to list out all the PO files, you'll be doing one API call for each of the languages for a PO template and that's going to be very slow. Eg. to get full status export for Ubuntu out of LP you'd need to do some ~400k API calls.
A much better solution to that is to provide a method which gives you all related PO file statistics: e.g. getPOFilesWithStatistics() which returns tuples of form (POFile, translated_count, changed_count, unreviewed_count, untranslated_count) directly. FWIW, I'd keep this a separate step as well.
IPOTemplateS(ubs)et API - as the final step, I'd do an export of IPOTemplateS(ubs)et APIs for all the different objects that have them. As another part of this step, you'd probably want to do the similar to above and provide a getPOFilesWithStatistics(language) method on them.
The idea behind this workflow is simply to use a gradual approach, which will be easier to get right and get landed, without ending up with a too large a branch to review or follow.
Success
Although the intention is to propose a full API, the two main success criteria are the implementation of API functions for reporting and the imports queue, so we will know that we are done when we can report on the status of Ubuntu translations at a distro level (i.e. on a given set of packages, for all Launchpad languages) by having sourced the data trough launchpadlib.
The success can be measured in how seamlessly this data can be obtained and in whether all requirements in terms of exposed data have been implemented.
