========
LongPoll
========

.. contents::

Overview
========

Here is an overview of all the components involved in long polling::


                              +------------+  LP.cache.longpoll.{uri, key}
                              |     LP     |<-----------(1)-----------------+
                              +------------+                                |
                                    ^                                       v
 +-----------+                      |                           +--------------------------+
 |           |<---LongPollEvent-(3)-+                           |        Browser           |
 | RabbitMQ  |                                                  |--------------------------|
 |  Server   |                                                  |                          |
 |           |<---RabbitEvent---(4)-+                           | longpolljs  (JSEvent (6))|
 +-----------+                      |                           +--------------------------+                   
                                    v                                       ^
                              +------------+  /{uri}?uuid={key}&sequence=0  |
                              | txlongpoll |<------------(2)----------------+
                              +------------+           --(5)->
                                                       <-(7)--

The workflow for a typical usage of the LongPoll system of LP will go like this:

1) The LP web server serves a web page with ``LP.cache.longpoll.uri``
   and ``LP.cache.longpoll.key`` populated. If these are not
   populated, the LongPoll machinery will stop here.
2) The JavaScript long-poll library then uses these to initiate a long
   lasting XHR request to the Twisted-based ``txlongpoll`` server.
3) Some time later, something noteworthy happens in Launchpad, and the
   code that wants to tell the world obtains an object providing
   ``ILongPollEvent``, typically via adaption. It then uses
   ``long_poll_event.emit(**payload)`` to send a message out via
   RabbitMQ, keyed with ``long_poll_event.event_key``.
4) The event will then be published by RabbitMQ to the txlongpoll
   server.
5) The long lasting XHR request to the txlongpoll server returns (with
   the JSONified payload from the LongPollEvent).
6) This is handled by the javascript longpoll library which triggers a
   javascript event with the payload.
7) The javascript longpoll library now reconnects to be able to handle
   other events.

LongPoll's usage
================

- `Merge proposal's diff update`_: merge proposal's diffs are updated via XHR as soon as new version is available (no more page reloads!)

.. _`Merge proposal's diff update`: /MPDiffUpdate 


RabbitMQ
========

RabbitMQ_ is the messaging system used to connect events between
discrete components.  In this case it connects long running jobs, that
emit an event when finishing, to the "longpoll" server which informs
the browser.

.. _RabbitMQ: RabbitMQ

LongPoll server
===============

See `lp:txlongpoll`_ for a project that describes a so-called
long-polling server. It is Twisted-based and maps HTTP requests to
consumption of messages on Rabbit queues.  It is a completely separate
service to Launchpad.

.. _lp:txlongpoll: https://launchpad.net/txlongpoll

This means we can have Javascript in the browser doing XHR to the
LongPoll server and block on long-running jobs until the job emits a
message saying it's done. There is currently work in progress to make
merge proposal diffs appear as soon as the job completes so that the
end user doesn't have to refresh the MP page.

LongPoll request format
=======================

The LongPoll server awaits connections of the form
'/uuid={queue_name}&sequence={sequence_id}'. The uuid parameter will
be used to connect to a specific queue. The sequence parameter is a
sequential integer that ``txlongpoll`` requires and is used to better support load-balancing the async frontend.

----

Writing code on the server side
===============================

LongPollEvent class
-------------------

`LongPollEvent`_ is the abstract adapter base class that should be
used to create custom event adapters.

  **Why?** -- *Julian*

    All that's needed is an implementation of ``ILongPollEvent``;
    subclassing ``LongPollEvent`` is not *required*. However, it has a
    sensible implementation of ``emit`` that is either usable as-is,
    or worth reusing via calling-up from subclasses. -- *Gavin*

.. _LongPollEvent: http://bazaar.launchpad.net/~launchpad-pqm/launchpad/devel/view/head:/lib/lp/services/longpoll/adapters/event.py

Sub-classes need to define the `event_key` property and declare
something along the lines of::

 class LongPollAwesomeThingEvent(LongPollEvent):
     adapts(IAwesomeThing)
     implements(ILongPollEvent)

..

  **What is event_key for?** -- *Julian*

    See `ILongPollEvent`_ ;) It is the key with which events will be
    emitted. It should be predictable and stable. This is the key to
    which queues can be subscribed. An example could be
    ``longpoll.event.branchmergeproposal.1234``. -- *Gavin*

.. _ILongPollEvent: http://bazaar.launchpad.net/~launchpad-pqm/launchpad/devel/view/head:/lib/lp/services/longpoll/interfaces.py

Alternatively, use the `long_poll_event` class decorator::

 @long_poll_event(IAwesomeThing)
 class LongPollAwesomeThingEvent(LongPollEvent):
     ...

In both cases the adapter should be registered in a `configure.zcml`
somewhere sensible::

 <adapter factory=".adapters.LongPollAwesomeThingEvent" />

..

  **EXAMPLE NEEDED HERE** -- *Julian*

    See `LongPollStormEvent`_ -- *Gavin*

.. _LongPollStormEvent: http://bazaar.launchpad.net/~launchpad-pqm/launchpad/devel/view/head:/lib/lp/services/longpoll/adapters/storm.py

zope.event bridge for Storm life-cycle events
---------------------------------------------

As a convenience, subscription handlers for `life-cycle events`_
(IObjectCreatedEvent, IObjectDeletedEvent, IObjectModifiedEvent) of
Storm model objects already exist. They generate long-poll events
named "longpoll.event.{table_name}.{object_primary_key}" with the type
of event ("created", "deleted" or "modified") in the event payload.

.. _life-cycle events: http://bazaar.launchpad.net/~lazr-developers/lazr.lifecycle/trunk/view/head:/src/lazr/lifecycle/interfaces.py

For instance, if ``storm_object`` is an object from a table named
``FakeTable`` with a primary id of 1234, calling::

 notify(ObjectDeletedEvent(storm_object))

... will result in an event being fired with the following data::

 {"event_key": "longpoll.event.faketable.1234",
  "what": "deleted"})

Testing
-------

To test that your code triggers the appropriate long poll events, use
the class `LongPollEventRecord`_ and the context manager
`capture_longpoll_emissions`_::

 from lp.services.longpoll.testing import (
     capture_longpoll_emissions,
     LongPollEventRecord,
     )

 with capture_longpoll_emissions() as log:
     pass
     # Here something should trigger a long poll event.

 expected = LongPollEventRecord(
     "my-event-key", {
         "event_key": "my-event-key",
         "something": "somedata"})

 self.assertEqual([expected], log)

.. _`LongPollEventRecord`: http://bazaar.launchpad.net/~launchpad-pqm/launchpad/devel/view/head:/lib/lp/services/longpoll/testing.py

.. _`capture_longpoll_emissions`: http://bazaar.launchpad.net/~launchpad-pqm/launchpad/devel/view/head:/lib/lp/services/longpoll/testing.py

----

LongPoll subscription
=====================

``ILongPollSubscriber`` describes a subscriber to long-poll events. It
defines a ``subscribe_key`` property:

  "The key which the subscriber must know in order to be able to
  long-poll for subscribed events. Should be infeasible to guess, a
  UUID for example."

An example could be
``longpoll.subscribe.a966e051-8817-4897-840f-9de18404c8f0``.

It also defines a ``subscribe`` method that can be passed an object
that provides ``ILongPollEvent``. In the back-end this ends up
creating a queue named ``subscribe_key`` which is bound to event's
``event_key`` (not exclusively; it can be bound to multiple events).

As the time of this writing only a request (as in
``IApplicationRequest``) can be used as a subscriber using
LongPollApplicationRequestSubscriber_

  **I don't understand what a "request" and a "subscriber" is in this
  context** -- *Julian*

    I think I've clarified that above. -- *Gavin*

.. _LongPollApplicationRequestSubscriber: http://bazaar.launchpad.net/~launchpad-pqm/launchpad/devel/view/head:/lib/lp/services/longpoll/adapters/subscriber.py

This adapter ensures that the request's JSON cache is stuffed with the
necessary information for the JavaScript component of LongPoll can
operate.

A request can be subscribed to a events quite simply::

 ILongPollSubscriber(request).subscribe(FakeEvent())

However, there's an even easier way, using
``lp.app.longpoll.subscribe``::

  subscribe(FakeEvent())

Or, when ``some_object`` can be adapted to ``ILongPollEvent``::

  subscribe(some_object)

For example, if ``some_object`` is a Storm model object (i.e. inherits
from Storm, or indirectly via SQLBase) then this single call arranges
the following:

1. Declares a new subscribe queue with an unguessable name (see
   ``subscribe_key``).

2. Binds the subscribe queue to the ``event_key`` defined by the
   ``ILongPollEvent`` adapter of ``some_object``.

3. Puts the ``subscribe_key`` in the request's JSON cache as
   ``longpoll.key``, sets ``longpoll.uri`` so that the in-page
   JavaScript knows where to connect, and adds the ``subscribe_key``
   to the ``longpoll.subscriptions`` list.

That's the back-end plumbing, now it's up to the in-page JavaScript to
know what to do with the events that it'll be able to long-poll for...

Next: `/JavaScript`_