Back in November I had the pleasure to attend the API Strategy Conference for the first time. It was two days of short (20 minute) sessions running in three different tracks with experts covering web service API design, practice, and other related topics. My goal as a newbie to API Strat was to get wider exposure to the API microversions work that we did with OpenStack, and to get out of that bubble to see what else was happening in the space.
Events Technologies Being Used on the Web
There were lots of talks that brought up the problem of getting real time events back to clients. Clients talking to servers is a readily solved problem with RESTful interfaces. But the other way is far from a solved item. The five leading contenders are webhooks (over http), HTTP long polling, Web Sockets, AMQP, and MQTT. Each technology push has their boosters and their place, but this is going to be a messy space for the next few years. OpenAPI’s version 3 specification includes webhooks, though with version 3 there is no simultaneously launched tooling. It will take a while before people build implementations around that. But that’s certainly a push towards webhooks. Nginx is adding MQTT proxy support sometime soon so that’s momentum for MQTT.
Webhooks vs. Serverless
Speaking of webhooks, the keynote from Glenn Block of Auth0 brought up an interesting point: Serverless effectively lives in the event technologies space as well.
Webhooks are all fine and good to make your platform easy for users to extend. A user tells your platform that when certain events happen, that platform should make an HTTP call out to a url that they specify. This HTTP call includes all the details of the event in question. Problem solved… or is it?
In order to reliably receive these events, the user needs to set up a webservice that is publicly accessible, and always available to receive these HTTP calls. That means making that service highly available and redundant. This service needs to be monitored, and have someone on call if there is an outage big enough to take it out. All of a sudden this “easy” extension of the platform just turned into a large new piece of maintenance. Many users will take a pass, and decide it’s not worth their trouble. This is exactly what Auth0 saw, too. While their platform could be extended in many ways that users wanted, there was too much overhead to get started.
Auth0 decided to try a different approach and built a serverless platform into their service to handle these webhooks. They found once they built out this serverless platform, they got much more uptake on their event API. Users could now get started without setting up extra infrastructure, which got many more of them to take advantage of it. Over time, they built in an “app store” so that users could share their scripts with each other, which drove even more usage of their API. They even found that power-user clients were building out some features that had long been on their ToDo list, but never quite made the cut to be implemented by their own development team. He made an excellent case that every online service should really be considering an embedded serverless environment.
Talking API Microversions at API Strat
My prime motivation to attend the API Strategy Conference (see my session presentation, here) to talk about API Microversions, an approach we used in OpenStack to manage the fact that deployments of OpenStack upgrade at very different cadences. The talk went pretty well and I hope everyone in the audience learned something helpful.
Twenty minutes was a huge challenge to explain something that took us all nearly six months to get our heads around. But I do think I managed to communicate the key challenge: When you build an open source system with a user-facing API, how do users control what they get? A lot of previously used “good enough” rules fall down in this scenario.
Darrel Miller had given a talk I attended “How to never make another breaking API change”. His first five minutes were really similar to mine, and then, because his presentation was about Azure, with a single controlled API instance, their solution veered in a different direction. It was solid reinforcement for that fact that we were on the right path here; and that the open source solution has a different additional constraint.
One of the toughest questions I got in my Q&A was one that I had been thinking about for a while. Does GraphQL make all these discussions obsolete? GraphQL was invented by Facebook to get away from the HTTP GET/POST model of passing data around, and lets you specify a pretty structured query about the data you need from the server. On paper, it solves a similar problem as microversions, because it if you are really careful with your GraphQL you can ask for the minimum data you need, and are unlikely to have a problem with additional data coming and going through the API. However in practice, I’m not convinced GraphQL would work. In OpenStack we saw that most API usage was not raw API calls, it was through an SDK provided by someone in the community. If you’re an SDK writer, it’s a lot harder to make assumptions about what parts of objects people want, so you’d tend to return everything. And that puts you right back with the same problem we have with REST in OpenStack.
API Documentation and “Skimming Hard”
There were many talks on better approaches for documentation, which resonated with me after our great OpenStack docs migration.
Taylor Barnett’s talk “Things I Wish People Told Me About Writing Docs” was one of my favorites. It included real user studies on what people actually read in documentation. It turns out that people don’t thoroughly read your API documentation, they skim hard. They will read your code snippets as long as they aren’t too long. But they won’t read the paragraph before it, so if there is something really critical and important to know about the code, make it a comment in the code snippet itself. There was also a great cautionary tale to stop using reassuring phases like “can be easily done”. People furiously hunting around your site trying to get code working are ramping up quickly and words like “easy” make them feel dumb and frustrated when they don’t get it working on the first try. Having a little more empathy for the state of mind of the user when they read your documentation goes a long way towards building a relationship with them. And the more confident they are in working with your APIs will make them more committed to your platform.
Making New Friends at API Strat
I also managed to have an incredible dinner the first night I was in town set up by my friend, Chris Aedo. Both the dinner and conversation were amazing, in which I learned about Wordnik, distributed data systems, and that you can lose a year of research because ferrets bred for specific traits might be too dumb to be trained but that’s another story.
Definitely a lovely conference, and one I hope to make it back to next year.
Catch up with my latest musings on my blog