☕ Tech Dive - APIs PART 2
Yesterday's email was focused on Request-Response APIs. Now, we're going to talk about Event-Driven APIs.
Hi Everyone!
Hope you’re all having a fantastic day!
Tech Dive - Event Driven APIs
In our last tech dive, we talked about what an API is, the Request-Response API Paradigm and REST/GraphQL.
Let’s say you’re a big fan of Billie Eilish. You’re such a massive fan, that you want to be alerted whenever she publishes a new music video on YouTube! Then, you can go and comment “first!” on the video.
So, you decide to write a script that will send you a text message whenever she publishes a new song on YouTube.
Your script will need to communicate with YouTube’s API to get data on her latest video. But how should you implement this?
Under the Request-Response paradigm, your script will have to constantly poll YouTube server’s with a HTTP GET request, where it gets Billie Eilish’s latest video. If the video was just uploaded, then your script will send you a text.
Remember, you have to comment “first!” within the first 30 seconds of the video being published if you want to have a shot at the first comment. Therefore, your script will have to be sending multiple requests every minute to Google’s servers.
Also, Billie uploads a new video every 2-3 weeks, so the vast majority of your requests will come back as failures.
Doing something like this is called Polling the API. WIth polling, a developer is constantly querying an API endpoint at a predetermined frequency to look for new data.
This is a huge waste of resources. What would be better is if Google’s server could alert your script when the event happens (Billie Eilish uploads a video) instead of you constantly polling Google’s servers.
This is where Event Driven APIs come in. Two ways of building an Event Driven API are with WebHooks and WebSockets.
BTW - YouTube’s API uses WebHooks to implement the functionality that we’re talking about. You can check it out here.
WebHooks
You can think of a WebHook as a “reverse” of Request-Response. With Request-Response, the server will have an API endpoint and the client will send an HTTP request.
With WebHooks, the client will have an API endpoint, and the server will send an HTTP request to it. The API provider implementing the WebHook will send a POST message to the configured URL when the event happens.
Here’s a great illustration
You can configure a WebHook by first setting up your own web server so you can recieve messages.
Then, give the API provider an endpoint and configure what kind of messages you want them to send you. Here’s an example with GitHub’s Webhook.
Now, there are some important considerations with WebHooks…
Security - As the developer of the endpoint, it is your responsibility to make sure you’ve received a legitimate WebHook. Any random person can send HTTP requests to your endpoint, so you have to verify that the requests are actually coming from the API provider. One way of doing this is with verification tokens.
Failures - You server will have occasional failures. Therefore, most WebHook providers will retry delivery in the event of a failure. Your system should be setup to handle this.
But, what if you don’t want to open up an HTTP endpoint? This is where WebSockets come in…
WebSockets
The WebSocket specification creates an API for the web that mimics “socket” functionality. In other words, it creates a persistent connection between the client and server so both parties can send data over at any time (the connection is full-duplex, so both parties send messages simultaneously if necessary).
Creating a WebSocket connection is quite simple. You’ll just run something like the following on your client…
var exampleSocket = new WebSocket(url);
To send data to the server ( or to the client if you’re the server), you just run
exampleSocket.send(“Wazup”)
Receiving messages can be done with the event handler onMessage
exampleSocket.onMessage = function(event){
console.log(“handle ” + event)
}
There are various other event handlers like onOpen so you can run a function once your connection opens or onError so you can handle errors.
WebSockets work get for fast, live streaming data and long-lived connections.
Issues with WebSockets
Need a Persistent Connection - With WebSockets both the client and the server need to keep the connection alive. So if your clients have a poor internet connection, they may keep dropping the connection and must go and reinstate it. This may add unnecessary load to your server.
Scalability - If 1,000,000 people want to use your app simultaneously, that means that you’ll have to manage 1,000,000 connections on your server (because WebSocket connections are persistent). This can get hairy, especially when the connections aren’t idle ( clients are sending a lot of messages to your server or vice versa). Additionally, horizontal scaling with WebSockets is a lot more tricky than it is for a REST API or WebHooks because you have to keep track of which server opened which connection. You can read more about the difficulties here.
Hope you enjoyed this tech dive!
Feel free to reply back with any feedback/comments.
Best,
Arpan