Throughout the short history of computing machines, programming has always been meant to streamline workflows and to automate repetitive actions in order to reduce errors and to increase productivity. This was true in the days of Babbage’s calculating machine and remains just as true in these days of web apps, mobile devices and the barest beginnings of artificial intelligence.
Webhooks are a relatively new method of achieving the goals of programming. As the name implies, they have to do with web pages and data. The “hook” part of the term refers to the programming method of hooking into the inner workings of a software system in order to capture important information as it is made available. Desktop applications, for instance, might capture keystrokes before they are sent to a word processor or other application.
Webhooks are a bit like APIs (application programming interfaces), but with an important difference. APIs are tools that programmers use to retrieve information from other software or websites. While APIs are request-driven, webhooks are driven by events. The reason why this distinction is important should become apparent shortly.
Let’s say you have a website that needs the latest foreign exchange rates. An API could be set up to poll the source of your data to see if anything has changed since the last request for update. This can end up becoming a bit of a resource hog, especially if most of your requests turn up zero changes. A webhook, on the other hand, simply has an open request for changes. When the data provider senses a change, it sends the updated information immediately. At all other times, it sends nothing.
So, does this mean that webhooks replace APIs? Only in certain types of situations. There are still plenty of opportunities to use APIs to perform specific, repetitive tasks. In fact, your webhooks can be used to call APIs to process the data that is received. The webhook can be what triggers the API’s request. There remains a need for both.
The beauty of webhooks is that it activates automatically when a specific, predefined event occurs. There is no need for a human to run some manual routine in order to gather data for processing batches on a recurring basis.
If a website visitor subscribes to your newsletter, they may automatically receive a welcome email. And a customer may automatically receive an email when their order is shipped. Both of these are perfect uses of webhooks to automate your business workflow.
How a Webhook Works
A webhook is, in programmer’s terms, an HTTP callback that is user-defined. It starts with a data provider — a website with information to offer. It could even be your own website. As an end-user of the desired data, you register a specific URL with the data provider and list the specific data to send and under which conditions to send it.
When the conditions are met, this triggers an event which prompts the provider to gather the desired data and to send it as an HTTP POST to the URL you provided. That URL is your “listener.” When it receives the data, it then does something with it.
A webhook can be likened to a doorbell. When someone presses the buzzer next to your door (an event), it triggers the sending of a signal to the “listener” — the bell itself, somewhere in your home, not far from the front door. When the bell receives the signal, it processes it by creating a sound, thus letting you know that someone is at the front door. An API in this analogy would continually need to poll the buzzer to see if someone pushed it; so, the API approach is not well suited for this kind of situation. If it were to poll every two minutes, your visitor could be waiting at your door for at least that long before you hear the doorbell buzzer.
Your URL leads to a page you create which includes code to process the HTTP POST. This could be in any one of a number of languages — for example, Rails, Django, PHP, Python, Java, Node, or .Net.
The data provider will give you one or more methods for receiving the data, most commonly JSON or XML.
In our foreign exchange rate example, earlier, you might merely post the current exchange rate for two key currencies, or you might have an on-page calculator which can accept a visitor’s input and convert that number of dollars into pesos, or some other currency. With the proper webhook, your online calculator can remain up-to-date and accurate.
Webhook Problems to Overcome
Not only do you need to program how your “listener” will handle the incoming data, you need to make certain your programming performs the proper level of security and data validation. For instance, if someone were to discover your “listener’s” URL, they could feed it false information. You need to ensure you use some method of data authentication.
You also need to know how your data provider handles error codes. If you receive data, but something goes wrong, what kind of error codes do you return to the provider? There’s always the danger that the information is incomplete or garbled. Another danger can occur when the information is successfully processed, but an error code is returned nonetheless. This could result in duplicate data being processed. If part of your data processing includes adding a new database record, you need to check for possible duplicates before completing the process.
Still another potential problem involves the sheer volume of data you might be setting up to receive. You want to make certain your system can handle the maximum possible volume of data no matter how fast it may change at the data source.
Three Modes of Webhook Operation
Your webhook could enable one of three types of operations:
· Push,
· Pipe, or
· Plugin.
The push is the simplest way to establish a webhook. It involves the data source simply “pushing” data to your page when a specific event is triggered. This ensures that you get the data hot out of the source the moment it is available. Using an API to poll the source would mean a delay most of the time of as much as the length of time between polls (the length of your polling cycle). On average, you’d receive your data within half of your polling cycle.
Pipes are merely webhooks that act as conduits for data, grabbing the data that it receives, sometimes doing something with that data and then passing it on to some other “listener.” For example, for your finance blog you might program a script to keep an eye on certain stocks and to email your subscribers when any of those stocks go above or below certain values.
Another example of a pipe webhook could involve your product catalog. Whenever you add a new item to your catalog, it triggers the creation of a Tweet to your Twitter followers.
Plugins allow others to extend the power of your own application. This works by allowing others to develop their own applications that utilize the data supplied by your webhooks to provide a new type of service.
As you can see, the possibilities are virtually limitless.
Summing it Up on Webhooks
Webhooks are a web based method of using events to trigger software to run. It has the power of APIs (application programming interfaces), but is event-based, rather than request-based.
Data is typically sent as an HTTP POST using JSON or XML and can be processed by programming code using your favorite web programming language.
Webhooks can be used to push data, to pipe information to another destination or to open up your own application to be extended by plugins created by third-party developers.
Whatever your business workflow, webhooks can take a great deal of the tedium from your team and automate that work for greater efficiency and accuracy.
Ready to take the next step, and launch your MVP? Drop us a line. Our team of ex-Googlers can help you every step of the way, from design, to engineering, and all the way through product-market fit.
