architecture | May 3, 2021
Hookdeck vs Amazon EventBridge: Which way to go for your webhooks?
Fikayo Adepoju Oreoluwa
A webhook is a method of responding to a change in a system using a custom callback in your web applications. Webhook requests are triggered from SaaS applications like Shopify, Stripe, etc., to an API endpoint in your application, and if these requests increase rapidly, you will need a way to asynchronously process requests to ensure that your server is not overloaded.
In this post, we will be taking a look at Amazon EventBridge and Hookdeck, two services that offer systems for asynchronous processing, to find out which is the best option for processing webhook requests.
What Is Amazon EventBridge and How Does It Work?
Amazon EventBridge is an Infrastructure as a Service system for event-driven architectures. It uses an Event bus to ingest events from event sources, and routes them to the appropriate event consumers. The event sources and event consumers include AWS services, SaaS applications and HTTP endpoints. Apart from the default event bus in every instance, EventBridge also allows you to create a custom event bus for peculiar needs of your system. EventBridge also offers filtering, monitoring and auditing of events.
What Is Hookdeck and How Does It Work?
Hookdeck is an Infrastructure as a Service system for processing webhooks. Hookdeck provides a message queue that asynchronously processes webhooks by ingesting webhook requests from your SaaS applications, and distributing them to your callback endpoint based on the load your API can handle. With Hookdeck, you also get a logging system for your requests, request filtering, retries, and a dashboard to search and troubleshoot your webhooks.
Processing Webhooks: Amazon EventBridge and Hookdeck
To get a first-hand experience with these two, I decided to set up webhooks from my Shopify store on both AWS EventBridge and Hookdeck.
EventBridge uses events, so you cannot make HTTP webhook requests to it. Thus, I will be using the architecture below to get EventBridge to work with webhooks (check out this article for more details about this architecture).
I will be sharing my experience using the following criteria:
- Setup requirements
- Number of steps to complete the process
- Time to complete the setup
In order to implement the above architecture and route my Shopify webhooks through EventBridge to my backend API, the following components are required:
- An AWS Lambda function to produce and publish events into EventBridge's default event bus
- An AWS API Gateway to enable HTTP requests targeting my (event producer) Lambda function
AWS Lambdafunction to consume events from EventBridge's default event bus and then call my backend API
- Knowledge of one of the supported languages for writing Lambda functions (I used
Goare also supported)
HTTPS endpointto my backend API
To achieve the same webhook routing using Hookdeck requires the following component(s):
HTTPS endpointto my backend API
Brace yourself because this is a huge one. Honestly, I have cut this list down to remove all the extra steps I took to troubleshoot the entire setup, yet it still feels like a lot of steps. Here are the main steps I took to get webhooks working on EventBridge:
Set up IAM Role for services in this architecture
- Create the IAM role
- Add policies/permissions for
- Publishing events to
- Creating log groups on Cloudwatch
- Publishing logs to
- Add the
IAMrole to your AWS user
Create (event producer) Lamdba function
Lambdafunction using the Author from scratch option
- Write the
Lambdafunction code to:
- Receive the webhook request body
- Build an event object defining the
- Add the webhook request body as the
Detailparameter for the event object
- Use the AWS SDK to publish the event object to EventBridge
- Deploy the function
Create an API Gateway
- Create a new
- Add a new resource
- Add a
POSTmethod to the resource
- Link the
POSTmethod to the
Lambdafunction created above
Create (event consumer) Lambda function
- Create the same function as the event producer and add code to do the following
- Extract the event from the EventBridge default bus to access the webhook request body
- Make a
POSTrequest to your backend API with the webhook request body
- Deploy the function
- Create a new
Rulefor the EventBridge
- Add an event matching pattern that corresponds to your
Sourcefrom the event producer Lambda function. This will ensure that only events that match the
Sourcewill be pushed into the event bus
- Add your event consumer Lambda function as the target for the
Test your setup
Phew! Now that's some setup.
Now on to Hookdeck. Here is a list of steps I took to successfully get my Shopify webhooks to be processed through Hookdeck's message queues:
- Create a new connection
- Name your connection (for me this was
Shopify Store Hooks)
- Enter destination label (for me this was
My production API)
- Enter destination URL (your backend
- Deploy Connection (click the
- Replace your endpoint
- Replace the
endpointin Shopify with the one generated by Hookdeck after the connection has been created. Done! That's all there is to it.
The next criteria I used to judge my experience was user interface and user experience, which often depend on each other. So let's look at what to expect from these two services as far as the number of screens and forms there are to go through when setting up webhooks. Please note that these are minimum estimates based on my demo, and you might have a different number depending on how you accomplish the task.
- Creating a Lambda function: 2 screens, 1 form, 3 compulsory fields (double this for both Lambda functions)
- Creating the API gateway: 5 screens, 4 forms, 6+ compulsory fields (depends on options)
- Setting up EventBridge: 2 screens, 1 form, 5 compulsory fields
In total, we have 11 screens, 7 forms, and 17+ compulsory fields. I intentionally left out the number of screens it takes to set up IAM just to have a fair comparison. You might also end up moving through more screens and forms while troubleshooting.
1. Creating a connection: 1 screen, 1 form, 3 compulsory fields 2. Replacing your endpoint in Shopify: 1 screen, 1 form, 1 compulsory field
In total, we have 2 screens, 2 forms, and 4 compulsory fields. This is obviously a less laborious user experience.
This is one metric that will not be exactly the same for everybody, but I will try and breakdown my experience to give some insight into how much time you should expect to spend setting up webhooks on each platform.
As a first-time user, I had to take time to get some knowledge on how to set it up, even though I perfectly understand what it does. I am not including the time spent on Youtube videos and documentation pages, however I believe this is also an important consideration for anyone using EventBridge for the first time.
I began setting up the architecture at 7pm and finished configuring all the moving parts by 10pm (3 hours later). Looking at the steps above, you might think that doesn't correlate but there was quite a learning curve to making sure everything is correctly set up, which adds to the overall time you spend on it.
Then I moved to the next step, which was testing the entire setup to ensure that everything worked fine. There, I ended up spending another 2 hours troubleshooting the whole process. If you're familiar with AWS, you might guess that IAM policies/permissions are the reason for such an exhausting process.
I was humbled by IAM and was super exhausted after figuring it out eventually. This might not be your issue while setting up, but be conscious that you might have to create some room for more time troubleshooting your setup
Filling one single form that mainly required my
https endpoint was quite a straightforward task on Hookdeck. It took me around 10 minutes to go through the whole process, from Hookdeck to updating my Shopify store, and an extra 5 minutes testing the setup from Shopify to my backend API.
So there you have it, my experience working with webhooks on both AWS EventBridge and Hookdeck. I will leave the final judgement to you however obvious it may seem, but my take away is that if you want the following then Hookdeck is the way to go.
- An all-in-one tool that is quick to set up and does not require advanced systems design knowledge or ties to a certain cloud provider
- To save time setting up webhooks
- A simple onboarding process for other members in your team that want to setup webhooks for asynchronous processing
- A CLI to receive webhooks locally for the purpose of debugging and troubleshooting
- A collaborative platform where your webhook processing and information can be accessible and manageable by members of your team
- An all-around pleasing user experience working with webhooks
Try Hookdeck Today
Instantly & reliably manageLearn More
Business Case: Hookdeck vs Amazon EventBridge
Read our business case to compare Hookdeck and Amazon Eventbridge for webhook processing based on: architecture, complexity, costs, security, scalability, and reliability.
Why Implement Asynchronous Processing of Webhooks
Learn about implementing asynchronous processing to handle volume spikes in a timely and efficient way.
Best Practices When Deploying Webhooks in Production
In this article, learn what are the practices for setting up , scaling and error recovery to deploy webhooks in production.