Services on API

Why are we doing this?

Improve how we provide services via the API to be consumed by LP pages.
Provide an alternative to the pattern whereby any methods to be invoked remotely (eg via the webservice API) currently need to be added to a domain object.

What are we doing?

Create new Service objects which are registered with the LP app and available through the API
Invoke exported methods on those services to get flattened data to be supplied to client side template engines or other JS tools.
Invoke exported methods on those services to initiate a service side application transaction, optionally returning client data.
Provide a mechanism to protect service methods using standard Launchpad security permissions.

What are the risks?

Added complexity/steps to providing data to a page
A change from current Launchpad practices, leading to fragmentation in the codebase
Increasing round trip time on page render, slowing launchpad down. This is not as much an issue if the view model data is written to the page's json request cache when the view is initialised.

How will we know if we were successful?

We have a cleaner, better way of exposing data to LP pages
We are able to easily use the API with our client side templates
We have a consistent mechanism and semantics used to invoke service APIs using XHR calls, launchpadlib, and server side business logic.
Our pages are faster, or at the very least no slower
At least one major unit of work and/or feature is delivered using this architecture

Process under experiment

Developer creates new Service for the API

Technical detail

This work provides a mechanism to easily implement a named service which exports methods to the api and which can provide json data to the client for rendering. Our standard lazr.restful library is used, but we are not attaching service methods to domain objects. This new implementation is more in line with a SOA approach which we are looking to adopt and allows the launchpadlib users and browser clients to access flattened data via the same API.

A key driver for this work is to move away from the approach where we have conflated domain objects with remoting abstractions. Say we have a new sharing/permissions model and want to (for example) create an an observer for a product. The current way would be:

IProduct
  @export_write_operation()
  def addObserver(person)

But distributions can also have observers added and so we would need to add the same method to IDistribution as well. And the consumers of these APIs would need to have this distinction coded in and the business logic would become brittle and the domain objects unnecessarily bloated.

A better way is to define a service which exposes behaviours via exported methods. Domain objects which encapsulate state are passed as parameters and the service business logic acts on those said objects. So the various pieces of business logic are better encapsulated, resulting in a system with better cohesion within modules and looser coupling between modules.

Defining and registering a service

Let's start with an example:

class IAccessPolicyService(IService):

    export_as_webservice_entry(publish_web_link=False, as_of='beta')

    @export_write_operation()
    @call_with(user=REQUEST_USER)
    @operation_parameters(
        pillar=Reference(IPillar, title=_('Pillar'), required=True),
        observer=Reference(IPerson, title=_('Observer'), required=True),
        access_policy_type=Choice(vocabulary=AccessPolicyType))
    @operation_for_version('devel')
    def addPillarObserver(pillar, observer, access_policy_type, user):
        """Add an observer with the access policy to a pillar."""

The addPillarObserver method returns json data which representing the result of the operation, sufficient to allow (for example) the view to be updated in the appropriate way. For the +sharing view, that would be to update the json request cache and add a new row to the observer table.

To expose the service, simply register it as a utility in zcml with a name:

<securedutility
    name="accesspolicy"
    class="lp.registry.services.accesspolicyservice.AccessPolicyService"
    provides="lp.app.interfaces.services.IService">
    <allow
interface="lp.registry.interfaces.accesspolicyservice.IAccessPolicyService"/>
</securedutility>

Services are traversed to via +services/<servicename>.

Service Invocation

There's 3 (consistent) ways to invoke the service apis.

1. Server side

service = getUtility(IService, 'accesspolicy')
service.addPillarObserver(....)

2. launchpadlib

from launchpadlib.launchpad import Launchpad
lp = Launchpad.login_with('testing', version='devel')
# Launchpadlib can't do relative url's
service = lp.load('%s/+services/accesspolicy' % self.launchpad._root_uri)
service.addPillarObserver(....)

Note: the above is what is required currently. launchpadlib itself will be modified to allow this:

lp = Launchpad.login_with('testing', version='devel')
lp.services.accesspolicy.addPillarObserver(....)

3. javascript client

var lp_client = new Y.lp.client.Launchpad();
lp_client.named_post('/+services/accesspolicy', 'addPillarObserver', y_config);

Permissions and security

Of course, a mechanism is required to limit access to service methods to only authorised users. For a given service which exports a number of methods, different permissions are more often than not required for each method. Because services operate on domain objects passed in as parameters, and permissions in Launchpad are defined for users against a domain object, the approach implemented provides a way to hook into these existing permission rules. We use a new method decorator lp.services.webapp.authorization.available_with_permission to declare the permission required to invoke a service method:

@available_with_permission('launchpad.Edit', 'pillar')
def addPillarObserver(self, pillar, observer, access_policy_type, user):
    """See `IAccessPolicyService`."""

The decorator is used to specify the permission which must be held by the requesting user and the named function argument from which to pull the object to be used in the permission check. So for a service call like this:

product = getUtility(IProductSet).getByName('firefox')
service = getUtility(IService, 'accesspolicy')
service.addPillarObserver(product, ....)

The above will raise Unauthorized if the user does not have launchpad.Edit permission on product 'firefox'.

View rendering/form submission

Many new Launchpad pages can avoid using any server side rendering. There will be a bit of TAL used for the page chrome but the business data is rendered in the browser using mustache (and soon handlebars). Data for the view model is obtained via a data retrieval method on the service. eg for the current example service, a method called getPillarObservers() could be used. This is invoked by the LaunchpadView instance defined as the page's view implementation and teh data is poked into the json request cache, from where it is accessed when the YUI view widget renders. The point here is that the exact same getPillarObservers() API could be used by a launchpadlib client to get json data for use in a script or tool or whatever.

class PillarSharingView(LaunchpadView):
    def initialize(self):
        super(PillarSharingView, self).initialize()
        cache = IJSONRequestCache(self.request)
        cache.objects['access_policies'] = self.access_policies
        cache.objects['sharing_permissions'] = self.sharing_permissions
        cache.objects['observer_data'] = self.observer_data

    def _getAccessPolicyService(self):
        return getUtility(IService, 'accesspolicy')

    @property
    def observer_data(self):
        service = self._getAccessPolicyService()
        return service.getPillarObservers(self.context)

When it comes time to write data from the view back to the server, either to initiate an action or save a form (or whatever), you can either:

1. use our standard LaunchpadFormView and associated infrastructure and delegate the submit processing to an instance of the service

2. invoke the service directly via an XHR call using a Y.lp.client.Launchpad() named_post

Final thoughts

The approach outlined herein is what we are using to prototype the +sharing view for the disclosure project. One key advantage of this approach over just adding methods to domain objects is that a service contract often will call for aggregated and/or flattened data to be returned to a caller (eg view or API user) and this requires gathering data from several places. So the service acts as a facade, allowing this to happen in a consistent, single sourced way for the different consumers of that data. Plus it encourages better separation of business logic between service functionality and encapsulation of state.

launchpad development

APIServices