Phil Leggetter
What are Event Destinations?
Our applications are more distributed than ever before, relying on a multitude of services and APIs to deliver the functionality we need. We use Shopify for e-commerce, Stripe for payments, Twilio for communications (or customer engagement), GitHub to automate our SDLC, and the list goes on. This distributed nature has led to the rise of event-driven applications (though we probably don't quite think in those terms yet), where services communicate with each other by sending and receiving events.
Since Jeff Lindsay conceived webhooks in 2007, they have grown to be the go-to mechanism for event producers to deliver developer platform events to event consuming applications. However, as the industry has evolved, the limitations of webhooks have become more apparent, particularly at scale, leading to:
- Event consumers having to introduce additional infrastructure and logic to their applications to reliably ingest, verify, queue, and process events delivered via webhooks.
- Event producers adding platform features to manage and provide observability for the inevitable failure scenarios
Companies like Stripe and Shopify have identified this burden and, while still supporting webhooks, recommend that events are delivered directly to GCP Pub/Sub or Amazon EventBridge instead. Hookdeck solves the common webhook ingestion issues for developers by sitting between the webhook provider and the developer's webhook endpoints, avoiding additional infrastructure.
Event Destinations represent an opportunity to think beyond webhooks as the only event delivery option, providing increased choice, a better developer experience, and improve the event delivery landscape for event producers and consumers alike.
Why Move Beyond Webhooks?
“At Twilio, we ultimately spent thousands of developer hours building queuing, security, retry logic, and a reverse load balancer to reliably deliver webhooks at scale.”
Patrick Malatack
Former VP Product, Twilio
Webhooks come with limitations that can impact reliability, scalability, and ease of use:
Lack of Standards
- No Uniform Protocols: There is no universally accepted standard for retries, timeouts, payload formatting, or security, leading to inconsistencies across implementations.
- Fragmentation: Each provider implements webhooks differently, requiring custom solutions for each integration.
- Failed Standardization Attempts: There have been various attempts at webhook standards to provide open source tools and guidelines to send webhooks easily, securely, and reliably. However, fragmentation is already in place across hundreds of providers and continues to be a growing problem.
Developer Burden
- High Cognitive Load: Developers must manually implement complex retry, security, and error-handling logic.
- Tool Fragmentation: Lack of centralized webhook management across multiple integrations creates additional complexity.
Security Challenges
- Opt-In Security: Webhook security typically requires manual setup, such as signing payloads or validating requests. This is prone to errors or misconfiguration.
- Exposed Endpoints: If not properly secured, public-facing HTTP endpoints are vulnerable to attacks such as DDoS, replay attacks, and spoofing.
Debugging Difficulties
- Lack of Visibility: Debugging webhook delivery failures or misconfigurations can be time-consuming due to limited logging and monitoring tools.
- Transient Issues: Temporary network issues or latency spikes are hard to diagnose and reproduce.
Operational Complexity
- Custom Infrastructure: Consumers must set up and maintain HTTP endpoints, handle retries, and debug issues—tasks that add significant overhead.
- Error Handling: Managing webhook retries, timeouts, and backoffs requires robust error-handling logic, which can become complicated.
Reliability Issues
- High Failure Rates: Webhooks depend on the availability and reliability of the consumer's endpoint. Temporary outages or slow responses can cause delivery failures. This adds a significant burden to the event producer.
- Delivery Uncertainty: Without robust retry mechanisms or clear visibility, it's hard for platform providers to know if the event was delivered, processed, and, in retry scenarios, if idempotency was handled correctly.
Scalability Constraints
- High-Throughput Limitations: Webhooks are inefficient for large volumes of events due to their lack of batching and reliance on individual HTTP requests.
- Performance Bottlenecks: Without efficient protocols like Protobuf or persistent connections, high frequency and high volume webhook event delivery adds significant overhead to both webhook producer and consumer.
Cost Inefficiency
- Resource Intensive: High failure rates increase retry attempts, leading to increased resource usage and operational costs.
- Scaling Costs: As the number of events and consumers grows, the infrastructure costs to support webhooks can escalate significantly.
What are Event Destinations?
Event Destinations are endpoints or systems to which event producers can send events and give the developer the choice to use the tools they are familiar with directly. For example:
- Google Cloud Pub/Sub
- AWS SQS
- Hookdeck Event Gateway
- Amazon EventBridge
- Kafka
- RabbitMQ
- And, of course, traditional HTTP webhooks
Event Destinations expand the capabilities of event producers, allowing them to offer more tailored and efficient integrations and with better deliverability. Event consumers benefit directly from the producer changes in addition to an overall improvement in the developer experience and enhanced observability.
Key Features of Event Destinations
To define Event Destinations, it's essential to establish core principles:
Multiple Event Destination Support: Event Destinations should support at least two event destination types.
Scalability and Reliability: Built-in support for retries and error handling ensures robust event delivery.
Observability and Debugging: Tools for monitoring, tracing, and troubleshooting event flows provide transparency and ease of use.
Security: Support for authentication, encryption, and access controls ensures secure delivery of events.
Existing Event Destinations Adopters
The concept of Event Destinations already exists within some major platforms:
Stripe: Stripe provides Event Destinations to deliver notifications and data updates directly to cloud services or HTTP endpoints as webhooks. Stripe currently supports the Amazon EventBridge cloud service. See the Stripe Event Destinations documentation.
Shopify: Shopify recommends delivery methods beyond traditional HTTP endpoints, such as GCP Pub/Sub and Amazon EventBridge. While currently branded under "webhooks," the underlying architecture aligns with the broader concept of Event Destinations. See the Shopify webhooks documentation.
A Call to Action for Event Producers
The transition from webhooks to Event Destinations represents a paradigm shift in how events are produced, delivered, and consumed. To lead this evolution, event producers should:
- Recognize the limitations of traditional webhooks and embrace diverse delivery mechanisms
- Invest in infrastructure that supports multiple Event Destination types
- Educate their developer communities on the benefits and capabilities of Event Destinations
By adopting Event Destinations, platforms can begin to see scalability benefits as customers migrate from using webhooks. Customers additionally benefit from greater flexibility and reliability, empowering developers to build more robust event-driven systems.
Conclusion
The concept of Event Destinations addresses the growing demand for flexibility, scalability, and improved DX in API platforms for both event producers and consumers. By expanding beyond webhooks alone, platforms can offer a diverse range of delivery mechanisms tailored to the unique needs of their users. Stripe and Shopify have already laid the groundwork, and now it's time for the broader industry to recognize and adopt Event Destinations. It's not just about delivering events—it's about delivering them the right way, every time.