ABCs of usable APIs

We spend a lot of time writing APIs at work: not only our external ones in the form of Atlas (the video and audio index) and Voila (our media product engine) but also the many internal ones which provide data to them and our other products. We therefore also spend a lot of time trying to make our APIs as pleasant to use as we can. At last October's MetaBeerTalks, I talked about how we do this. As the evening wasn't recorded, and the slides I used were less of a write-up and more a memory jog as I talked, I decided a blog post was in order to sum up what I said.

a: all the data

We aim to provide all data required for common use cases in a single API call. This makes life easier for clients of our APIs for a number of reasons:

1. Minimising HTTP requests: browsers typically have a limit on the number of concurrent requests to a single host.
2. Overhead per request: there's a cost to each HTTP request, and this can be especially noticable on devices which are resource constrained or are at the end of a poor connection. HTTP keep-alive certainly helps, but so too does keeping down the number of requests.
3. Consistent responses: clients don't need to worry about handling data changing between calls - there's only one, so it's guaranteed to be consistent.

Here's a snippet of an output from Voila for a well-known UK soap opera:

You'll see that a request for the EastEnders brand returns not only the brand-level data you'd expect, such as a description and an image, but also information on upcoming broadcasts, under the upcoming_items_detail key. This makes it easy to build a brand page with all the information you'd commonly put on it: upcoming episodes, links to episodes on-demand, all through a single call. You'll even get channel logos of various forms alongside each broadcast.

b: brevity

annotations

I value brevity: why say in 100 words what you can say in 10? The same is important in our APIs. While that may sound at odds with what I've just said about providing all the data in a single call, we use annotations to let clients decide what data they need, thus reducing both the amount of data transferred and processing cost. Let's look again at what Voila can tell us about EastEnders. Firstly, a request for the brand without any annotations (https://voila.metabroadcast.com/1.0/content?id=cf2¹):

Not really that much use. But if we add &annotations=description we get a little more data back:

And so on: upcoming_subitems_detail will return the episodes soon to be broadcast that you saw above, available_subitems_detail renders episodes available on-demand.

equivalence

A key feature of Atlas is equivalence: the process by which we match data across the many different sources we ingest data from. Here's a portion of an equivalence graph for the EastEnders brand, cf2 being Press Association's record for EastEnders and dms the BBC's:

Once all the hard work is done matching records together, we provide a consolidated view of the data. Instead of returning multiple records for EastEnders, a single, merged, record is output with on-demands sourced from the BBC, copy and images from PA. All of this is managed through API key configuration that lets users specify merge rules.

personalisation

It's not much use telling someone in Scotland about their favourite show being aired tonight on BBC One if it's not on BBC One Scotland until tomorrow. Having to implement a client-side filter based on someone's region means that a superset of data needs to be returned to a client then filtered using a swathe of reference data about platforms and regions. Instead of this hideousness, we do all the heavy lifting: the region parameter in Voila will ensure that users see only the broadcasts relevant to them. The less code there is between a front-end website and our APIs the better.

c: consistency

All endpoints across an API should behave in a predictable, consistent way. In Voila, a user's actions can just as easily be returned on the content endpoint as the schedule endpoint, so building a personalised schedule is a cynch. Here's EastEnders from the content endpoint (https://voila.metabroadcast.com/1.0/content?id=cf2&annotations=subscriptions,my\_actions):

And from the schedule endpoint (https://voila.metabroadcast.com/1.0/schedule/bbcone-london?from=2014-01-27T20:00:00Z&to=2014-01-27T20:30:00Z&annotations=subscriptions,my\_actions,broadcasts,description,extended_description)

Annotations will change a bit in the Atlas 4.0 API so their names reflect the keys they control; people will become content.people, for example.

There is a slew of standard features in all of our APIs: JSONP, pagination parameters, annotations, to name a few. We're building a set of shared Frisby tests to enforce consistency. We've long talked about adding a meta-API, and having recently had a wonderful experience writing an auto-generating Java client for Nitro, a BBC API, we're even keener to add it to Atlas 4.0 in the near future. This will bring much-needed consistency to a shiny new auto-generated Java Atlas client!

s: speed (and reliability)

We pity slow clients and want to do everything we can to help them out. Some profiling of our TV listings widget showed that a lot of time was being spent parsing JSONP. Adding CORS support let us use JSON instead, which yielded a 2.7x improvement in speed. CORS is now standard for all our APIs.

Server-side speed is also important, of course. The majority of Voila endpoints return personalised responses in under 50ms in normal circumstances. Combined with reliability—achieved through infrastructure that automatically scales based on demand, and the likes of circuit breakers to fail gracefully if dependent systems go under—there's no need to build caches between our APIs and end users.

save the date

If you didn't make it to the MetaBeerTalks where I waffled on about this, you missed my terrible slide transitions (made tolerable by having a bar at hand to numb the pain). All is not lost, though. How about coming along next month on Wednesday 12th February? We'll announce more details about who'll be talking and the venue closer to the time.

¹ Unfortunately I can't link directly to Voila as it'll only work if you're in possession of a valid API key.