Business Case: Hookdeck vs Amazon EventBridge

It is often joked that the answer to every architecture or infrastructure decision is "it depends." Compromise is the name of the game and each day, software and system architects find themselves making trade-offs between a list of desired conditions based on pricing, performance, time investment, human resources, and more. In this post, we will look at one of those architectural scenarios involving the processing of webhooks, compare two approaches to solving the problem using AWS EventBridge and Hookdeck, and come to a decision on the preferred approach.

Read our comparison on how to get started to receive a webhook on AWS Eventbridge vs Hookdeck.

Criterias of Desired Setup for Webhook Processing

So, we are using a SaaS application (Shopify, Stripe, etc.) that sends webhooks to our API. We want to asynchronously process the webhook requests to ensure that we don't drop a single webhook when there is a network error, or when there are traffic spikes. Spikes in traffic can cause our API server to shutdown, and could negatively impact business and user experience.

Our desired setup should be able to do the following:

1. Ingest webhook requests
2. Route webhook requests to our API
3. Throttle delivery of webhooks when traffic spikes
4. Retry failed webhook requests (automatically or manually)
5. Provide visibility into the status of webhook requests

Aside from these functional requirements, we also desire a solution that is:

1. Cost-effective
2. Requires minimal time to set up with little or no maintenance required
3. Secure

We are going to evaluate each approach to processing webhooks using the criteria above.

AWS EventBridge Architecture, Requirements, Pros and Cons

The first solution we will explore is AWS EventBridge. EventBridge is an Infrastructure as a Service system for event-driven architectures. It uses an event bus to ingest events from event sources, and routes them to the appropriate event consumers.

Architecture

Let's take each of the functional requirements and explore how each one can be achieved using EventBridge.

Ingest webhook requests

• Webhook requests are HTTP requests, and EventBridge only receives events, so an AWS API Gateway must be set up in order to receive webhook requests.
• The webhook request must then be converted into an event that EventBridge can receive. This can be achieved using an AWS Lambda function connected to the API gateway.

Route webhook requests to our API

• EventBridge receives and sends out events via its event bus; thus, an event consumer is needed to receive outbound events. This can be achieved by creating another Lambda function as the target for the outbound events.
• An event bus Rule is required to ensure that only the events from the webhooks we want to process get routed to the target Lambda function.
• The Rule setup above needs to be done for different categories of webhooks to be processed.
• The Lambda function, upon receiving the event, calls our API with the payload contained in the event.

Throttle delivery of webhooks when traffic spikes

• Rules must be added to the event bus to deliver events at a certain rate.

Retry failed webhook requests (automatically or manually)

• EventBridge Replays need to be created to replay archived events, and specify a timeframe for the events to be replayed.

Provide visibility into the status of webhook requests

• Role permissions must be set up to ensure that each service used (API Gateway, Lambda, etc.) can publish logs to AWS CloudWatch.

With the above breakdown, we will have an architecture that looks like the one below:

Requirements

The architecture above is quite complex, and to implement requires at least the following:

1. Proficiency in cloud architecture design
2. Solid understanding of AWS (IAM roles and policies, familiarity with each service used)
3. Proficiency in one of the languages supported by AWS Lambda
4. Good understanding of event-driven architecture

Pros and cons

Now that we have a good understanding of the architecture and requirements for processing webhooks with EventBridge, let's take a look at some of the advantages and disadvantages to this approach.

Pros

1. Highly configurable
2. Can work with any SaaS provider by tweaking the architecture to their requirements
3. AWS documentation and community is large so you have a higher chance of finding answers to problems
4. Roles and permissions can be configured for different users/clients
5. Webhook events can be routed to other AWS services

Cons

1. Architecture can become complex, introducing many points of failure
2. Requires proficiency in a cloud provider (AWS)
3. User experience is tedious (so many screens and forms required to set it up)
4. CloudWatch logs can be too raw to make sense of when dealing with webhooks
5. Various moving parts require maintenance
6. Tracing the webhook request through the various components while troubleshooting is not a straight-forward process
7. No centralized dashboard to view the activities taking place in the setup

Hookdeck Architecture, Requirements, Pros and Cons

The second solution we will explore is Hookdeck. Hookdeck is an Infrastructure as a Service system for processing webhooks. Hookdeck provides a message queue that asynchronously processes webhooks by ingesting webhook requests from your SaaS applications, and distributing them to your callback endpoint based on the load your API can handle.

Architecture

Let's take a look at how Hookdeck can be used to achieve the functional requirements referenced above.

Ingest webhook requests

• Hookdeck is built for webhooks, and therefore naturally ingests webhook HTTP requests.
• The SaaS provider needs to be added as a connection on Hookdeck, and a unique endpoint for the connection generated by Hookdeck is required to register with the SaaS provider as the destination endpoint.

Route webhook requests to our API

• Hookdeck requires we supply the destination endpoint on our API when creating the connection.
• Hookdeck routes all webhook requests from the SaaS platform to this endpoint.

Throttle delivery of webhooks when traffic spikes

• A rate limit per Destination can be set to throttle the delivery of webhooks at the specific pace your system can handle.

Retry failed webhook requests (automatically or manually)

• Hookdeck Rulesets can be used to control how webhook requests are retried for each connection; this includes how many times they are retried and at what intervals.
• Manual and Bulk retries can be done on the dashboard or via the API.

Provide visibility into the status of webhook requests

• Hookdeck has a centralized dashboard for monitoring and troubleshooting webhook requests.

The architecture of the breakdown above is shown below:

Requirements

The major requirement for using Hookdeck to process webhooks is knowledge of how the platform works. You need a basic understanding of how to:

1. Set up a Connection
2. Define Rulesets for retries
3. Filter and browse events on the dashboard

All of this information can be found in Hookdeck's documentation.

Pros and cons

Hookdeck is obviously a simpler option than EventBridge. However, let's look at the advantages and disadvantages to this approach.

Pros

1. Quick and easy to set up (can set up a connection in 2-3 minutes)
2. Requires minimal to no architectural knowledge
3. Has an intuitive dashboard designed to monitor, trace and troubleshoot webhook requests
4. User experience is appealing, as connections can be set up in 2 to 3 simple steps
5. Does not require different components to make it work, thus has minimal points of failure
6. No coding knowledge required
7. Can fanout webhook requests to more than one destination
8. Can setup routing based on attributes, queries, and paths

Cons

1. Currently does not support all SaaS providers
2. Reliability not as tested as that of AWS services.
3. Article archives cover a lot of use cases, but documentation still building on peculiar use cases for sites like Stackoverflow or GitHub issues
4. Does not allow you to configure roles and permissions for different users (for now, it's on the roadmap)

Comparing EventBridge and Hookdeck

We have looked at these two approaches from the perspective of functional requirements, and as much as both help solve the problem, we have seen the areas where one approach has an edge over the other.

However, functional requirements are not the only reasons for choosing a solution. We also need to take a look at other non-functional requirements like cost, time investment, security, and so on. The table below compares EventBridge and Hookdeck using these additional criteria, which also play a key role when considering which system to use.

Conclusion

As it was stated at the beginning of this article, most architectural decisions will eventually come down to "it depends". If you're looking for a quick and reliable solution to process webhooks asynchronously, with minimal architectural knowledge, low cost and very clear visibility into your webhooks, then Hookdeck is the way to go. However, if you like having control over every aspect of your system and have the time, budget, and savvy engineers knowledgeable about system architecture, then you can go the way of EventBridge and have all the power of AWS behind you.