Message Sharing Migration

Part of the message sharing project.

Assumptions

• message-sharing-populate has run.
• Codebase is fully message-sharing-enabled.
• Schema elements that are to be removed are either gone or no longer used.

Code changes

To be landed before migration:

• (Pseudocode, any resemblance to Python is pure coincidence)

  class TranslationMessage:
      # ...
      def converge(self):
          """Make this message shared if possible, or merge it into an existing shared one."""
          if self.potemplate is None: return
  
          shared = TranslationMessage.get(potemplate=None, self.potmgset, self.language, self.variant, self.translations)
          current = TranslationMessage.get(potemplate=None, self.potmsgset, self.language, self.variant, is_current=True)
          imported = TranslationMessage.get(potemplate=None, self.potmsgset, self.language, self.variant, is_imported=True)
  
          if shared:
              # There's a shared message matching this one.  Try to merge.
              if self.is_imported and imported is None:
                  # Bequeath is_imported flag to shared equivalent.
                  shared.is_imported = True
              if self.is_current and current is None:
                  # Bequeath is_current flag to shared equivalent.
                  self.is_current = False
                  shared.is_current = True
              # This message is only worth keeping if it's a diverged current message.
              if not (self.is_current and not shared.is_current): self.destroy()
          else:
              # No matching shared message yet, so become one.
  
              # If we're about to converge: don't supplant an existing shared imported
              # message.  If we're staying diverged: don't permit imported messages to
              # be diverged.
              if imported is not None: self.is_imported = False
  
              # Converge, unless this is to be a diverged current message.
              if current is None or not self.is_current: self.potemplate = None

Note: When clearing is_current in validate_is_current, converge the TM that's having the flag cleared.

Migration steps

Migration consists of these phases:

• Initial Cleanup

• Merge POTMsgSets

• Merge TranslationMessages

• Restore POFileTranslator

Initial Cleanup

Before migrating, we should delete all non-current POTemplates. This eliminates some complications in migration.

Before we can delete a POTemplate, we must delete any rows that refer to it:

1. Clear TranslationMessage.pofile if the column still exists. We won't be using it anymore.

2. Delete all TranslationMessages that refer to the template in their potemplate fields.

3. Find all TranslationTemplateItems that refer to the template, and delete them.

4. Find any POTMsgSets that the deleted TranslationTemplateItems referred to, and that have no other TranslationTemplateItems referring to them.

5. Delete any TranslationMessages attached to those POTMsgSets.

6. Delete those POTMsgSets.

7. Delete all POFiles that refer to the template. Any references in POFileTranslator should have gone away when we deleted the TranslationMessages.

8. Delete the POTemplate.

Merge POTMsgSets

At the end of this phase, TranslationMessages will still be mostly diverged, but they'll be sharing POTMsgSets.

This phase requires at least a freeze on imports.

It's best to go through this separately per Ubuntu release, and maybe once for everything else. We must stop imports for whatever product or distroseries we're merging, which we can do individually for each Ubuntu series.

Commit transactions after every equivalence class of POTemplates. Keep track of which ones are done, so we can restart etc.

• Define equivalence class of POTemplate as belonging to POTemplates of the same name (not translation domain?) within either the same Product or the same DistroSeries.

• Define equivalence class of POTMsgSets as being attached (by TranslationTemplateItem) to POTemplates of the same equivalence class, and having the same (msgid_singular, COALESCE(msgid_plural, -1), COALESCE(context, )).

• Define the representative POTMsgSet in an equivalence class as the one in the series that has translation focus, if any; or failing that, the one with the highest id.

Pseudocode:

def get_key(potmsgset):
    return (potmsgset.msgid_singular, potmsgset.msgid_plural, potmsgset.context)


def merge_potmsgsets(potemplates):
    # Sort potemplates from "most representative" to "least representative."
    potemplates.sort(cmp=template_precedence)

    representatives = {}
    subordinates = {}

    # Figure out representative potmsgsets and their subordinates.
    for template in potemplates:
        for potmsgset in template.potmsgsets:
            key = get_key(potmsgset)
            if key not in representatives: representatives[key] = potmsgset
            representative = representatives[key]
            if representative in subordinates: 
                subordinates[representative].append(potmsgset)
            else:
                subordinates[representative] = []

    for representative, potmsgsets in subordinates.iteritems():
        for subordinate in potmsgsets:
            merge_translationtemplateitems(subordinate, representative)

            for message in subordinate.translation_messages:
                if message.potemplate is None:
                    # Guard against multiple imported messages.
                    if message.is_imported:
                        imported = representative.getImportedMessage(message.language, message.variant)
                        if imported is not None: message.is_imported = False
                    # Guard against multiple shared current messages.
                    if message.is_current:
                        current = representative.getCurrentMessage(message.language, message.variant, potemplate=None)
                        if current is not None: message.is_current = False
                message.potmsgset = representative
            subordinate.destroy()