Webhooks and WebSub both fulfil the broad function of data transfer at scale. Both form key infrastructural components of today's internet. However the way they transfer data makes their use cases quite different. The article below sheds some light on when and how to use each.
What are Webhooks?
Webhooks are an important aspect of event-driven architectures. When someonething happens in one system, Webhooks fire an event to tell another system what's just happened. They're essentially an if this/then that. From deploying to GitHub to syncing contacts with Mailchimp, we use Webhooks multiple times a day to ensure what we do online has an impact elsewhere.
Webhooks work by providing a user-defined callback-URL, which needs to be registered at a service provider’s platform. This URL is called by the service provider using a
POST request to send information about an event to the vendor.
The vendor will either send the 200OK response (or a response with standards specified by the provider), to acknowledge the response was received. If the return response does not match the provider’s pre-defined rules then the provider will keep sending the information on the same (ie registered) callback-URL. This behavior can be customized and configured according to the service provider's needs.
Information on updates will be stored by the vendor to keep their databases in sync with the service provider's information.
Webhooks are generally used when there is one subscriber (company or a domain) that needs to be notified of events related to their business/domain and how these affect different providers, who both expose webhooks. (See below for more information on use cases).
A full deep-dive into the webhook protocol is available to read on the Ably Realtime Concepts page.
A recap on WebSub
WebSub functions through a network of Publishers, Subscribers, and Hubs. The Publisher is the content producer, the Subscriber the content consumer and the Hub the content distributor. WebSub makes it possible to share content over the internet to whoever has opted to receive this data.
WebSub has the advantage of inbuilt security mechanism for sharing, accepting and distributing content more securely between publishers and subscribers. This inbuilt security is known as 'verification of intent'. This security handshake happens at any stage of the process, often at the time of subscribing and unsubscribing.
Similarly, the subscriber subscribes the topic URL to the Hub, as advertised by the publisher. After this, whenever the publisher publishes content or updates existing content to the Hub, the Hub automatically pushes these contents to all the Subscriber(s) who are tuned in to this topic. So, instead of a server sending periodic polling requests, each Subscriber will receive updates - ie have the content pushed to them - in real time. The Subscriber can unsubscribe from this topic anytime - they just need to send the request to the Hub.
WebSub comes into its own where the same data from Publisher (eg an article from Medium author) is floated to millions of Subscribers using a Hub. So, the data is not really personal anymore. Anyone can hook to a Topic and get the data. There is minimal human input in the transfer of data. Automated fanning out of information is where WebSub shines.
Webhooks and WebSub functionality
The paragraphs and comparison tables below give an overview of webhooks' and WebSub's relative functional advantages/ disadvantages.
Webhooks advantages - a deeper dive
Webhooks function in near realtime to transfer information to vendors on events that have occurred provider-side. This realtime approach is known as event-driven notifications for Webhooks and is useful both for providing immediately actionable, up-to-date information and also for keeping two remote systems in sync.
A webhook loosely couples two applications to establish a communication channel between them. These two systems or applications can exchange information on particular events, acknowledging the respective event without having to ‘know’ the other party’s server architecture, functionality or configuration. The communication happens asynchronously and is event-driven. This saves time and hassle when integrating systems.
Downtime Intolerance (delivery guarantees)
With webhooks, the event information will never get lost, even if the vendor party is experiencing downtime. This is because the Provider will keep sending event data periodically to the callback-URL until the vendor responds back with the 200OK response with the Provider’s predefined response body. Usually, the Provider follows up the retry several times if they don’t get a success response from the vendor, before marking this data as abandoned. This functionality differs provider by provider, so it is worth reading their individual strategies to make sure you get what you need.
Easy integration and consumption
WebHooks is easy to integrate into any application or system. It’s just a matter of registering the callback-URL. All other control will be in the consumer’s hands once they consume the data from the webhook call.
Webhooks don’t have widely accepted security standards nor any in-built security system. But if we can build our own security layer on top of the webhook then we can keep ownership of the security controls and enhance the security according to the existing system architecture’s security standard. Here security of each webhook connection would be governed by webhooks’ provider and consumer. Both will follow a predefined set of security standards before or while communicating via webhook.
Webhooks - disadvantages
Webhook callback-URL is called from the remote application with a data payload. As it is executed from an external environment it is not easy to debug the request client-side. To overcome this problem, a provider should expose the sandbox environment to developers. This allows for testing and debugging using trial and error.
As long as the webhook publicly exposes the callback-URL, there’s a risk it will be misused by attackers. The attacker can send malicious event data to the webhook and generate high traffic to bring down a vendor’s servers. To avoid this kind of situation, strong security is required. Such attacks can attack both provider and vendor side, thus anti-DDoS measures should be put in place by both the parties.
At the time of downtime, the vendor could miss notifications from the provider if the provider is not retrying (or limited retry) the WebHook to deliver the information to the respected vendor. Here webhooks are absolutely controlled by the third-party service provider. No control at the vendor’s hand.
The webhook payload should be consumed properly. The return response must be the 200OK and follow the provider’s pre-defined standards. Otherwise, the provider will keep calling the webhook multiple times and the database will get clogged up with duplicate entries vendor side. As discussed above, the debugging process is difficult for webhooks and this use case must be taken care as a top priority.
A consumer receives the topic contents and updates in realtime, avoiding the need for periodic polling. A Publisher posts content to the Hub, which further optimizes distribution to subscribers.
WebSub comes with an in-built security feature and follows the community adopted security standards at all stages of the data transfer process. At the time of subscription, unsubscription and publish content, the Hub checks the locally computed HMAC keys to ensure the security of the connection.
In WebSub pattern, the Subscriber doesn’t know anything about the architecture, configurations or structure of the Hub’s or Publisher’s server in order to get contents. This allows for an enhanced level of security abstractions between all the WebSub network entities plus minimal admin faff both ends.
WebSub - disadvantages
Debugging & Integration
Just as with webhooks, debugging can be complex in WebSub as it’s functions in a loosely-coupled architecture and the content format is not predictive. Integration also requires several steps - finding the topics, subscription request, and security check request, which is a lengthy process..
The Publisher and Subscriber are both dependent on the Hub. The publisher sends the content to Hub and shares with the Subscriber. If the Hub goes down then the whole WebSub pattern cycle is interrupted.
Which is the better choice (and when)?
The perfect answer is it depends - on the use case. The table and explanatory paragraphs below will help you decide.
When to use webhooks
Webhooks are a good choice for the event-driven notification system. When the consumer needs to be notified on every event occurrence with their account at the providers' platform, the webhook implementation is ideal. WebHooks help a consumer keep in sync with the provider’s status as the provider sends data on each event back to the consumer via callback-URL.
The Webhook is a better choice when SLA of the delivery is considerable. Webhooks (by their nature) may have a stricter SLA requirements, while WebSub may have looser constraints on timely delivery.
Examples for when webhooks win out are payment gateways, email providers, SMS & calling service provider, analytics platforms and so on. When any payment happens at the payment gateway, the vendor system must receive notification so they can confirm the order (ie perform a follow-on function). Here payment gateways return payment information using webhooks to inform the vendor. Stripe, SendGrid, Twilio, Slack are good examples of webhooks in action.
For example, on every charge or refund, Stripe uses webhooks to report these results to the vendors/consumer synchronously to keep them updated with stripe’s events.
SendGrid provides webhooks to get notified on every event occurrence for email delivery and its engagement. SendGrid’s webhook will return the success delivery event data. The various events are delivery, opened, click on a link, spam.
Twilio, just like SendGrid sends event data back to vendor application on Voice Call and SMS events like call received, call hangup, SMS delivered, read, etc.
HubSpot, Freshdesk, SalesForce and other CRMs provide the webhooks to return the event information like account creation, permission assign, ticket rise/deletion/response and other CRM centric events.
A fully functional system can architect with multiple individual microservices, each microservice is responsible to perform a specific job. Like email service, Auth service, notification service. Here webhooks can be a communication bridge between all the microservices. They can share the event information for the communication and controlling purpose and keep the system synchronized across all the servers.
When to use WebSub
Websub is a good choice for use-cases such as broadcasting public information/data from a single publisher (or small number of publishers) to millions of subscribers via hub. In a broader way, many subscribers subscribe to their interested topics and get notified when any publisher publish those topics to hub. In such a case the provider and the consumer don’t need to know each other. They communicate solely relevant information, and contact is dealt with by the intermediary Hubs.
For platforms who frequently generate or update new content, by whom we mean news aggregators, podcast networks, publishers, streaming services, and so on, WebSub can be considered the perfect solution. It will distribute content changes in real time, as updates are tailor-made for the audience.
WebSub is a better choice when SLA of the delivery is not a concern. WebSub (by their nature) may have a looser SLA requirements, vs the Webhook, which may have a strict constraints on timely delivery.
Real-life examples of WebSub implementations in action include:
Publishers: Medium, Wordpress, Blogger, CNN, etc
Subscribers are Flipboard, Feedly, NewsBlur etc.
Hubs are pubsubhubbub.appspot.com and pubsubhubbub.superfeedr.com
Publishers, Hubs, and Subscribers
All these blogs and article publishers publish their content to some community hosted Hubs like pubsubhubbub.appspot.com and pubsubhubbub.superfeedr.com which are maintained and operated by Google and Superfeedr.
And subscriber examples are Flipboard, Feedly, NewsBlur which subscribes to the Hubs to get the latest updates from the Publishers.
Conclusion - more a choice than a battle
We hope the above information has helped/ will help give an idea of the nuances between WebSub and webhooks and the two different functions. The decision of what to use depends on what you want the data to do. WebSub is generally more useful if you want to publish updates or receive updates on a changing situation. As mentioned above, webhooks are part of the architecture that enable updates intended to having a follow-on affect, i.e. trigger other functions. Open-source, home-built solutions work well for both, with a number of hosted solutions available as you start to require functionality at scale.
Read more about the practicalities of engineering event-driven systems on the Ably Engineering blog. Read about the theoretical underpinnings of realtime communication on the Ably Realtime Concepts pages.
Ably Realtime provides cloud infrastructure and APIs to help developers simplify complex realtime engineering. We make it easy to power and scale realtime features in apps, or distribute data streams to third-party developers as realtime APIs. Create a free Ably account to get started.