Fast flexible search

Search should be fast and flexible.

As a Technical Architect
I want the infrastructure to support all the following stories
so that I can focus on other interesting things

As a Developer
I want Users to be presented with duplicate bugs reliably
so that only unique bugs are filed

As a User
I want to find the project launchpad easily
so that I can read about it.

As a translator
I want to find other translations to my language of the template
so that I can reuse them.

As a Launchpad hacker
I want to be able to find all of our bugs that also affect Bazaar but not the loom plugin
So that I can improve Launchpad / Bazaar integration without worrying about crappy plugins.

As an upstream project maintainer
I want to be able to quickly access previous searches that I've done
Because I am a complicated person

As a Launchpad hacker
I want to search for all of the bugs tagged "sshserver" that are not owned by anyone and not tagged "hard"
So that I can do some opportunisitic hacking

As an upstream project maintainer
I want to be able to search for an error string in the bugs & questions in my project and in all of its dependencies (both the original projects & their packages)
So that I can figure out what the hell is going on

PLEASE ADD MORE STORIES

This feature is not about tweaking our UI. It is about clarifying what we want to be able to do with search, so that an overhaul of the search core selects a thing with a good fit to our needs.

Rationale

Search is timing out, and fixing it is hard when we don't know what we actually want it to do.

We are doing it now because search is timing out and timing out gives a terrible experience to users.

This will eliminate most-or-all search related timeouts.

Stakeholders

As this is about the guts of the system, the stakeholders are the developers of launchpad that want to offer features to users.

Constraints and Requirements

Must

What MUST the new behaviour provide?

Nice to have

Must not

What MUST it not do?

Subfeatures

Other LaunchpadEnhancementProposals that form a part of this one.

Workflows

What are the workflows for this feature? Provide mockups for each workflow.

You do not have to get the mockups and workflows right at this point. In fact, it is better to have several alternatives, delaying deciding on the final set of workflows until the last responsible moment.

Success

We will know we are done when we have a set of stories we can distill into requirements for evaluating our current search implementation and other candidate implementations.

Thoughts?

We have two search engines - google and tsearch2; google can't do private objects, and neither are all that good at doing what we need for e.g. duplicate bug detection.

We also pay index costs in-transaction, which makes insertions of e.g. bug comments into the system slower. An out of transaction system (e.g. tsearch2 still, or lucene, or *** new shiny) can decrease page times for bug comments and the like.