Guide to Building or Buying Your Webhook Infrastructure

So far in our webhook management solution series, we have taken a look at architecting a design blueprint. We also took a deep dive into each component that makes up our proposed webhook solution, and spent time discussing how our webhook infrastructure can be monitored to improve performance, detect faults and avoid failures.

This article aims to answer a very important question for businesses working with webhooks:

Should you build this webhook solution or buy a ready-made one?

We will discuss the two options by looking at the scenarios where it is favorable for a team to build and where the team is better off buying the webhook solution. I’ll be making business and technical arguments to support each approach.

Excited? Let’s get started.

Solution recap

Before we get started, let’s take another look at the core requirements for our webhook solution. These are the requirements that were decided on in the design article and that birthed the architectural blueprint.

Technical and business factors to consider for building or buying

Before we begin, let’s get some terminology defined on both the technical and business sides of things. The terms below are the factors we will use to support our decisions on either building or buying the webhook solution.

Technical factors

Business factors

Now that we have a good grasp on the terms that will be used to support the arguments, let’s discuss the best scenarios for building or buying a webhook solution.

Building the webhook infrastructure

Building your webhook solution is the process of using in-house or contracted experts to develop a solution from scratch that satisfies all the requirements stated in our design blueprint.

In what scenario should you decide to go with this option? What is the persona of a team that should favor building their webhook infrastructure over buying an existing one?

• You have a very unique and specific set of features that your webhook solution needs to support
• You plan to sell your software as a product (not as a service)
• You are ready to give the planning, development, and testing processes as much time as they require
• You have access to professionals that can take on the project

If your current situation agrees with at least two of the points above, then you should strongly consider building a webhook solution from scratch.

If you have determined from the above points that building the webhook solution is the way to go for you and your team, below are some of the benefits of this approach on both the technical and business sides:

• Configuration: If you require a substantial amount of customization to the workflow of your webhooks, for example validation checks, peculiar data transformation, or conditional throttling, there is a huge upside to developing your webhook infrastructures as it allows you room for more specific configuration/customization.
• Control: Building your webhook infrastructure gives you total control over the features. You also have control over when to implement software updates or add new features, and the type of technology to use for a certain component (for example RabbitMQ over Kafka for message queues) based on ease of use and/or the preference of your team.
• Maintainability: Experts who want to enforce their own maintenance culture can take advantage of the openness of building from scratch. For example, building allows you to maintain specific versions of the software within the architecture and upgrade when you believe it is the right step for your team. You can also design your scalability strategy to meet preferred parameters. Because you have good knowledge of the capacity and throughput of the components within your architecture, you can better estimate the number of resources required to scale it and maintain its performance against growing a webhook load.
• Interoperability: No webhook solution exists in isolation — it complements other systems already in a company’s ecosystem of applications. Building your webhook solution allows you to design it to properly integrate with existing software applications in your suite.
• Feature set: When you own the technology and control all technological choices, you can design the system to be extendable in custom ways. This helps you to easily bring in new features no matter how unique or peculiar they are. Also, you do not depend on an external team to develop new features or roll out upgrades.

Options for building your webhook infrastructure

There are different approaches you can take when building out your webhook solution. Based on our solution blueprint, we will discuss three ways you can implement your own customized webhook solution.

1) The Open Source Approach

This involves using purely open source technologies in your stack. With your components identified, you can use open source software like RabbitMQ or Apache Kafka for messaging, Nginx for load balancing, Prometheus for metrics monitoring, and Elastisearch for logging.

Using a stack made up of open source technologies offers the broadest range of configuration and customization options. You are also free to choose the type of deployment environment for your webhook solution.

To see a demonstration of how to build a webhook solution using open source technologies, check out our article on webhook solution implementation options.

2) The Cloud Service Provider Approach

This approach makes use of service offerings by full-featured cloud service providers like Amazon Web Services (AWS), Google Cloud Platform (GCP), and Microsoft Azure. These providers offer services that can be used to implement any of the components within the webhook solution blueprint.

One major advantage to this approach is the speed of deployment and little to no code requirement. Most of the service offerings also have scalability features built-in.

To see a demonstration of how to build a webhook solution using AWS services, check out our article on webhook solution implementation options.

3) The Hybrid Approach

This style takes a more pragmatic approach to implementing the webhook solution by combining the best of the two previous approaches. Because the components are decoupled, you can choose to use open source technologies for some of the components and cloud services for others.

For example, you can deploy an API gateway or load balancers with Nginx while using AWS SQS for your message queueing. This way, you can use open source technologies for the components where you have proficiency and want total control, while you purchase cloud services for other components to take advantage of built-in scalability, quality, and ease of use.

Buying the webhook infrastructure

Buying a webhook solution involves purchasing or subscribing to a cloud service or installed software solution to handle your webhook workflows. Unlike the cloud service providers in one of the build options above, a webhook management cloud service is designed specifically to handle webhooks.

Just as we did in the “build” section, let’s look at the points that define a team that should buy an existing webhook solution.

• The webhook solution is an enabler for your business and not the main service/product
• You need a cost-efficient tool because you’re working with a limited budget
• You need to deploy fast
• You’re not equipped with the technical knowledge and professionals to take on and/or maintain the solution’s architecture and scalability requirements

If you can see yourself and your team in at least two of these scenarios, then it is highly recommended that you buy an existing solution to handle your webhook ingestion and distribution.

Below are some of the business and technical factors that support buying a webhook solution.

• Configuration: Purchased solutions come with templated configurations that are ready-made for different use cases, for example rate-limiting settings, retry settings, and alert settings. These configurations can also be customized based on your preferences. Because these configuration templates have been thoroughly tested, there’s very little room for errors. Also, making configuration changes is a lot less complicated compared to custom-built solutions.
• Engineer onboarding: Onboarding new engineers takes less time compared to a solution that was built from scratch. The purchased solution’s documentation contains all the required information and is the same for every deployment. Also, engineers with prior experience using the solution can get started right away.
• Time to market: There is less hassle to have it up and running because there’s no need to spend so much time building it. This helps you to deploy fast and start serving your webhooks to their destination(s) effectively while you spend the time saved on more customer-focused needs of your business.
• Engineering cost: Buying a webhook solution saves you from all the financial headaches of building a custom solution. There are also no unpredictable issues on the development side that can lead to variations in project costs. Even when there is a substantial upfront cost in the case of one-off solution purchases, it is always a good investment in the long run. Products with subscription-based models also make the cost more approachable.
• Maintenance: All maintenance activities (architectural complexities, bug fixes, updates, upgrades, etc.) are solely the responsibility of the solution provider. Providers can also offer support channels for you to communicate your issues and have them resolved as soon as possible.
• Quality: A webhook solution provided by a credible brand has gone through quality assurance testing. This ensures that all the features are working optimally. Most vendors also offer free trials for you to test out the system to see that it meets your operational needs.

Hookdeck: A webhook management option for buying your webhook infrastructure

Hookdeck is a cloud-based webhook solution for reliably managing your webhooks that sits in between your webhook providers and your API, making sure that all requests are properly handled without dropping any. Hookdeck also provides total visibility into all transactions and events between your webhook requests and your API. Its features are tailored to give you full control over the workflow of your webhooks while removing the burden of building a reliable, fault-tolerant, and scalable system from your shoulders.

To further understand why Hookdeck should be your go-to webhook solution, let’s take a look at what Hookdeck offers according to the factors we have discussed so far.

Configuration and Control

Hookdeck is not an HTTP service you’re trying to adapt for webhooks; instead, Hookdeck was made for webhooks. Therefore it comes bundled with features that give you total control over how your webhooks are transmitted from source to destination.

Some of the features include:

• Retrying a failed webhook
• Controlling the rate at which webhooks are delivered to their destinations
• Routing of webhooks to multiple destinations and based on conditions
• Transforming webhook payload before it arrives at its destination
• Pausing the flow of webhooks from source to destination

The webhook solution also includes configuration options to control how these features behave, for example you can configure your webhook retries to either be done automatically at set intervals or manually.

Hookdeck also comes with an intuitive dashboard that gives you full visibility into the activities of your webhooks. You can track timestamps, view payload and headers, detect errors, read error messages, and also query historical webhook events.

Cost and Time to Market

A connection between a webhook source and its destination can be set up on Hookdeck in under 30 seconds (yes, seconds). Once you have this connection set up, you can start serving webhooks right away with all the controls and reliability that Hookdeck offers. This means that if webhooks are critical to the operations of your business, it literally takes seconds to have them served efficiently.

Hookdeck has a free tier that gives you up to 100 thousand requests per month with a three day retention period for your webhook data. For as low as $39/month, you can serve up to a million webhooks and persist your webhook data for up to a week.

Maintainability and Security

With Hookdeck, you don’t have to worry about software patches, server upgrades, feature updates, and more. The Hookdeck team abstracts all that from you for your peace of mind.

We also stay on top of security and have released a security checklist that ranks as one of the top security articles for webhooks on Google. This is an indication of how seriously we take the security of your webhooks so that you can focus on other important aspects of your business.

Interoperability

Hookdeck is designed to be platform and technology agnostic. This allows you to connect any provider to any destination. Whether you’re looking to connect your webhooks to a custom API or a SaaS application like Discord, Hookdeck makes the process seamless.

Check out our Platform guides for a full list of articles to help you get started quickly with services like Shopify, Stripe, GitHub, and CircleCI.

Onboarding and Continuity

Hookdeck has a gentle learning curve and enables you to set up a connection in just a matter of seconds. This makes the knowledge easily transferrable. Even if an engineer that has no idea about Hookdeck joins your team, we have provided enough documentation to make them a Hookdeck expert by the end of their first day.

The documentation is also broken down into different sections as listed below to help engineers quickly find any information they are looking for:

• Getting Started to get up and running quickly
• Product Documentation for in-depth knowledge about product features and how to use them
• Platform Guides for practical examples and information on how to use Hookdeck for different platforms like Stripe and Shopify
• Webhook Guides to learn more about webhooks and how to handle them optimally
• API Documentation for engineers that want to use Hookdeck’s programmatic interface

Conclusion

In this article, we looked at our webhook solution requirements and discussed the scenarios that favor building the solution from scratch or buying an existing one. We also took a look at the benefits of each approach and their options.

To conclude, you can use this as a rule when making your decision: if the webhook solution is the actual product you’re selling or the core feature and you have very specific feature requirements, the development time, and the professionals to take on the project, then building the solution is justified.

However, if webhooks are simply an enabler for your business, you’re on a budget and you need a cost-effective tool that also helps you meet up with your schedule by deploying fast, or you’re not equipped with the technical knowledge to take on such complex architectural projects, then it is highly recommended that you buy an existing solution like Hookdeck.

Happy coding!