Complete Guide to Troubleshooting and Debugging Webhooks

If software development existed without bugs, developers would be the happiest people in the world. There would be no banging of hands on the keyboard in frustration, or waking up in the middle of the night just because you suddenly get a brain buzz telling you "how to fix that bug." Unfortunately, this luxury is not the case as part of software development is iteratively removing bugs from written code until there are (almost) none.

When developing a piece of software, a huge percentage of time is spent testing and debugging the code to remove bugs, and developing with webhooks is no exception. In this article, we will look into troubleshooting webhooks. We go over what brings about bugs in the process of developing with webhooks, how to tackle these bugs, and tools that can help the debugging process.

Why are my webhooks not working?

So, why are my webhooks not working?

This statement is very similar to the other popular question, Why is my code not working?

Developers ask this a lot because there is a 90% chance your code will not work the first time, and it's the same with webhooks. This obstacle will almost definitely occur and is not inherently a big deal, however it is very important to know where to look when your webhooks are not working as expected.

Receiving webhooks

The first thing you want to be sure of is that you are receiving your webhooks. Because webhooks are regular HTTP requests originating from a source application to your API, they can fail silently if there is no facility to trace the request.

You can make use of Hookdeck’s Online Test tool to check that you’re receiving your webhooks at the expected endpoint on your server.

One thing you want to do before working with webhooks is read the webhook provider's documentation properly (not the entire thing, just the section on webhooks) and look at sample code on their website to master how to consume the webhook.

Some other steps you can take:

Receiving Webhooks Validation	Reason
Do a health check on your webhook source	Sometimes (rarely), your webhook provider's system is down and not sending webhooks. A good number of webhook providers have a system status page that indicates when the system is up and running or down.
Check if the webhook is pointing to the right destination	Sometimes you may have an error on the webhook URL provided, and your webhooks are sent to the wrong endpoint. You want to check that you have registered the correct webhook URL.
Make sure your destination server is running	You'll be surprised by how many times you think you have your API running when it's not. Also, if the webhook request encounters an error upon hitting your endpoint (code throws an exception), your system can go down or fail silently, making it seem as if the webhook did not arrive.

Webhook requirements

Another common reason why your webhooks could not be working is that you're not fulfilling the total requirements for consuming them. I once had an experience where I was receiving the webhooks but the request body was empty. It turned out that my server had to parse the raw request stream. This is one of the reasons why you want to ensure that you properly go through the developer's documentation. How to go about navigating webhook documentation is explained in depth in this article.

However, some general requirements for webhooks are:

Webhook Requirement	Explanation
Subscription event	A webhook request is triggered by an event happening in the source system thus before you can have webhooks, you have to be subscribed to an event.
Webhook URL	Event triggers are sent as an HTTP request to a given endpoint; ensure that your destination endpoint has been supplied correctly.
Secure HTTP (HTTPS)	If your destination server does not have SSL installed, you need to install one or use a proxy server for SSL termination.

What to debug

Another aspect of confusion in the debugging process is knowing what to debug. Just like any debugging process, debugging webhooks starts with tracing the error source, reading any error messages, and interpreting the messages in order to reach and fix the root cause. Let me share a few tips from my experience debugging webhooks.

Debugging Step	Verification
Was the webhook received?	As explained earlier, the first thing you want to be sure of is that the request reached your endpoint. You can take the simple approach of inspecting the request object on your controllers by printing it out, or use software like Zipkin for request tracing.
Is it originating from the expected source?	You don't want to be getting webhook requests from the wrong source, so checks should be put in place for this.
Does it contain the information expected?	Most webhook requests come with information in the payload that is critical to the work you want to do on your server. Make sure that this information is available in the webhooks you are receiving.
Are you validating the information you need?	I once had a scenario where a boolean value was sent in string format. My code was trying to use this variable as a boolean, and that created bugs. So, you want to make sure you're validating the data types of the data you're receiving and transforming them where necessary.
Is the webhook for the expected event?	This is another human error that can easily creep into your setup. For example, you thought you submitted an endpoint for an account updated event meanwhile, due to one reason or another, you submitted it for an account created event. This simple error can lead to hours of hopeless troubleshooting. Ensure you do the proper checks; some providers are even kind enough to add the event type in a header or the request body.
Did your API respond appropriately?	This is specific to how each application responds to the webhook it's subscribed for. You want to make sure that you're processing the webhook appropriately and returning the right status code.
Is the processing idempotent?	Idempotency is a big deal with webhooks as you want to avoid duplicate processing when webhooks fire more than once.
Is authentication required?	Sometimes your webhooks will bounce because you're not fulfilling their authentication requirements. You want to make sure you're taking care of this appropriately. Information on how to authenticate webhooks can be found on the provider's website.

One of the ways to get answers to the above questions quickly is to use the online webhook test tool by Hookdeck to test your webhook endpoint in development/production.

Why debugging webhooks can be hard

Let's face it, debugging can be quite hard. Debugging webhooks is even harder as it involves both the network (webhook HTTP requests) and application (your code) layers. Let's go over some of the reasons why debugging webhooks can get frustrating quickly.

Reproducing error in local environment

When an error occurs, the first step is to recreate it to inspect it in an isolated environment. Doing this with webhooks is quite hard, as you have to simulate both the conditions from the webhook provider (e.g. Stripe, Shopify, etc), and the conditions on your endpoint under which the webhook was received. For example, when trying to debug my Shopify webhooks in production, I need to create a new Shopify store for development and recreate the scenario that caused the error in production. Imagine trying to replicate this same scenario in a trial account for a store that has production data. Inconveniently, Shopify's free trial expires after 14 days, so you have to either pay to maintain a development store or create a new one all over again. Keep in mind that your debugging setup also has to be done in your local environment to keep it isolated, which makes it even harder to recreate the error.

Working on localhost

When recreating your error locally, you can only spin up local server environments on localhost. These servers are not publicly accessible and thus your webhook provider cannot reach them. This introduces an extra setup in the debugging process to bypass firewalls and NATs in order to make the local server publicly available. Keep in mind that you have to provide access to your server over HTTPS.

To get past all this and be able to receive and test your webhooks on localhost, you can use Hookdeck’s webhook test tool.

Idempotency when sending mock data

Mock data is mostly used for debugging, and one attribute of mock data is that it will always produce the same result. Imagine your test creates a new unique user in the database; this means that anytime you run the test, you need to clear the user from the database before you run another test. This can get frustrating if not automated. Sometimes developers have to comment out the logic that makes the processing idempotent, just to be able to run multiple tests. This is bad practice as bugs may be introduced in the process or you might forget to uncomment the code, thereby compromising the integrity of your application.

What you need to debug webhooks

Hopefully this article hasn't scared you into thinking that debugging webhooks is an impossible task. Don't panic, there are tools for achieving all that is mentioned above but before we get into that, let's look at a couple of things you will need to start debugging webhooks.

Webhook Debugging Requirement	Explanation
Webhook URL	You need to be able to generate a publicly available webhook URL for your application endpoint over HTTPS.
Webhook test	You need to set up your server locally and have it running with the appropriate code to debug the process.
Inspection tools	Your debugging setup needs to have tools to inspect your webhook request. This could be done by using the debugging tools in your IDE (set line breaks for request inspection) or by using specialized tools with dashboards for visualizing the request object.

Webhook debugging tools

Now that you have a good understanding of the debugging process for webhooks, let's take a look at tools that have been built to achieve this.

Hookdeck Webhook Test Tool

The Hookdeck Webhook Test tool is an online test tool for validating the behaviour of your webhooks. The tool allows you to use both local and remote webhook URLs and test them with webhooks from your actual webhook provider or a mocked provider. This way, you can get past all the inconveniences of running tests on a webhook while in development.

Hookdeck CLI

Hookdeck CLI combines the ability to generate a publicly accessible and persistant webhook URL to your endpoint with an inspection dashboard for reading the details of your webhook requests.

It can be installed and set up on all operating systems, and it's free.

Ngrok

Ngrok is an open-source tool that exposes local servers hidden behind NATs and firewalls to the public internet over secure tunnels. Ngrok makes it easy to quickly generate a secure URL to be used as your webhook URL. It also includes a simple dashboard for monitoring requests hitting your endpoint.

You can learn more about Ngrok and how to set it up on their website. You can also check out our guide to debugging using Ngrok.

RequestBin

RequestBin is a free service by Pipedream that enables you to capture, parse and inspect HTTP requests. With RequestBin, you can spin up a publicly accessible URL at the click of a button and point your webhook requests directly to it.

When a request hits your endpoint, RequestBin gives you clarity into the request object for inspection. RequestBin also offers private endpoints and file uploads. You can learn more about RequestBin on their official website.

Conclusion

Debugging can be a very tedious process but knowing what to debug and having the right tools makes it a lot easier. This goal of this guide is to make your webhooks debugging process less of a pain — however, this one article cannot cover everything about debugging webhooks, so stick around for follow-up articles in this series on the subject.

Happy coding!