Hookdeck vs Amazon EventBridge: Which way to go for your webhooks?

A webhook is a method of responding to a change in a system using a custom callback in your web applications. Webhook requests are triggered from SaaS applications like Shopify, Stripe, etc., to an API endpoint in your application, and if these requests increase rapidly, you will need a way to asynchronously process requests to ensure that your server is not overloaded.

In this post, we will be taking a look at Amazon EventBridge and Hookdeck, two services that offer systems for asynchronous processing, to find out which is the best option for processing webhook requests.

What Is Amazon EventBridge and How Does It Work?

Amazon EventBridge is an Infrastructure as a Service system for event-driven architectures. It uses an Event bus to ingest events from event sources, and routes them to the appropriate event consumers. The event sources and event consumers include AWS services, SaaS applications and HTTP endpoints. Apart from the default event bus in every instance, EventBridge also allows you to create a custom event bus for peculiar needs of your system. EventBridge also offers filtering, monitoring and auditing of events.

What Is Hookdeck and How Does It Work?

Hookdeck is an Infrastructure as a Service system for processing webhooks. Hookdeck provides a message queue that asynchronously processes webhooks by ingesting webhook requests from your SaaS applications, and distributing them to your callback endpoint based on the load your API can handle. With Hookdeck, you also get a logging system for your requests, request filtering, retries, and a dashboard to search and troubleshoot your webhooks.

Processing Webhooks: Amazon EventBridge and Hookdeck

To get a first-hand experience with these two, I decided to set up webhooks from my Shopify store on both AWS EventBridge and Hookdeck.

EventBridge uses events, so you cannot make HTTP webhook requests to it. Thus, I will be using the architecture below to get EventBridge to work with webhooks (check out this article for more details about this architecture).

I will be sharing my experience using the following criteria:

1. Setup requirements
2. Number of steps to complete the process
3. UI/UX
4. Time to complete the setup

1) Requirements

AWS EventBridge

In order to implement the above architecture and route my Shopify webhooks through EventBridge to my backend API, the following components are required:

1. An AWS Lambda function to produce and publish events into EventBridge's default event bus
2. An AWS API Gateway to enable HTTP requests targeting my (event producer) Lambda function
3. An AWS Lambda function to consume events from EventBridge's default event bus and then call my backend API
4. Knowledge of one of the supported languages for writing Lambda functions (I used Nodejs, but PHP, Java, Ruby, C# and Go are also supported)
5. An HTTPS endpoint to my backend API

Hookdeck

To achieve the same webhook routing using Hookdeck requires the following component(s):

1. An HTTPS endpoint to my backend API

Setup Process

AWS EventBridge

Brace yourself because this is a huge one. Honestly, I have cut this list down to remove all the extra steps I took to troubleshoot the entire setup, yet it still feels like a lot of steps. Here are the main steps I took to get webhooks working on EventBridge:

Set up IAM Role for services in this architecture

1. Create the IAM role
2. Add policies/permissions for
3. Publishing events to EventBus
4. Creating log groups on Cloudwatch
5. Publishing logs to Cloudwatch
6. Add the IAM role to your AWS user

Create (event producer) Lamdba function

1. Create Lambda function using the Author from scratch option
2. Write the Lambda function code to:
3. Receive the webhook request body
4. Build an event object defining the DetailType , EventBus and Source parameters
5. Add the webhook request body as the Detail parameter for the event object
6. Use the AWS SDK to publish the event object to EventBridge
7. Deploy the function

Create an API Gateway

1. Create a new REST API Gateway
2. Add a new resource
3. Add a POST method to the resource
4. Link the POST method to the Lambda function created above

Create (event consumer) Lambda function

1. Create the same function as the event producer and add code to do the following
2. Extract the event from the EventBridge default bus to access the webhook request body
3. Make a POST request to your backend API with the webhook request body
4. Deploy the function

Setup EventBridge

1. Create a new Rule for the EventBridge default bus
2. Add an event matching pattern that corresponds to your DetailType and Source from the event producer Lambda function. This will ensure that only events that match the DetailType and Source will be pushed into the event bus
3. Add your event consumer Lambda function as the target for the Rule created

Test your setup

Phew! Now that's some setup.

Hookdeck

Now on to Hookdeck. Here is a list of steps I took to successfully get my Shopify webhooks to be processed through Hookdeck's message queues:

1. Create a new connection
2. Name your connection (for me this was Shopify Store Hooks)
3. Enter destination label (for me this was My production API)
4. Enter destination URL (your backend https endpoint)
5. Deploy Connection (click the Create Connection)
6. Replace your endpoint
7. Replace the endpoint in Shopify with the one generated by Hookdeck after the connection has been created. ‍ Done! That's all there is to it.

UI/UX

The next criteria I used to judge my experience was user interface and user experience, which often depend on each other. So let's look at what to expect from these two services as far as the number of screens and forms there are to go through when setting up webhooks. Please note that these are minimum estimates based on my demo, and you might have a different number depending on how you accomplish the task.

AWS EventBridge

1. Creating a Lambda function: 2 screens, 1 form, 3 compulsory fields (double this for both Lambda functions)
2. Creating the API gateway: 5 screens, 4 forms, 6+ compulsory fields (depends on options)
3. Setting up EventBridge: 2 screens, 1 form, 5 compulsory fields

In total, we have 11 screens, 7 forms, and 17+ compulsory fields. I intentionally left out the number of screens it takes to set up IAM just to have a fair comparison. You might also end up moving through more screens and forms while troubleshooting.

Hookdeck

1. Creating a connection: 1 screen, 1 form, 3 compulsory fields 2. Replacing your endpoint in Shopify: 1 screen, 1 form, 1 compulsory field

In total, we have 2 screens, 2 forms, and 4 compulsory fields. This is obviously a less laborious user experience.

Setup Time

This is one metric that will not be exactly the same for everybody, but I will try and breakdown my experience to give some insight into how much time you should expect to spend setting up webhooks on each platform.

AWS EventBridge

As a first-time user, I had to take time to get some knowledge on how to set it up, even though I perfectly understand what it does. I am not including the time spent on Youtube videos and documentation pages, however I believe this is also an important consideration for anyone using EventBridge for the first time.

I began setting up the architecture at 7pm and finished configuring all the moving parts by 10pm (3 hours later). Looking at the steps above, you might think that doesn't correlate but there was quite a learning curve to making sure everything is correctly set up, which adds to the overall time you spend on it.

Then I moved to the next step, which was testing the entire setup to ensure that everything worked fine. There, I ended up spending another 2 hours troubleshooting the whole process. If you're familiar with AWS, you might guess that IAM policies/permissions are the reason for such an exhausting process.

I was humbled by IAM and was super exhausted after figuring it out eventually. This might not be your issue while setting up, but be conscious that you might have to create some room for more time troubleshooting your setup

Hookdeck

Filling one single form that mainly required my https endpoint was quite a straightforward task on Hookdeck. It took me around 10 minutes to go through the whole process, from Hookdeck to updating my Shopify store, and an extra 5 minutes testing the setup from Shopify to my backend API.

Conclusion

So there you have it, my experience working with webhooks on both AWS EventBridge and Hookdeck. I will leave the final judgement to you however obvious it may seem, but my take away is that if you want the following then Hookdeck is the way to go.

1. An all-in-one tool that is quick to set up and does not require advanced systems design knowledge or ties to a certain cloud provider
2. To save time setting up webhooks
3. A simple onboarding process for other members in your team that want to setup webhooks for asynchronous processing
4. A CLI to receive webhooks locally for the purpose of debugging and troubleshooting
5. A collaborative platform where your webhook processing and information can be accessible and manageable by members of your team
6. An all-around pleasing user experience working with webhooks