How to Handle Out-of-Order and Duplicate Chargebee Webhook Events

If you're integrating with Chargebee for subscription billing, you've likely encountered this scenario: your webhook handler processes a subscription update, adjusts a customer's access level, and updates your database, but then everything gets overwritten moments later when an older webhook arrives out of sequence. Or worse, the same webhook arrives twice and you've accidentally double-credited a customer's account.

Out-of-order and duplicate webhooks are common challenges with webhook integrations and Chargebee is no different. Handling them incorrectly can lead to data inconsistencies, incorrect subscription states, or customers receiving the wrong service tier.

In this guide, we'll explain why Chargebee sends webhooks out of order or in duplicate, practical techniques to handle them, and how Hookdeck can simplify these challenges for your application. If you're just starting out with Chargebee webhooks, see our getting started with Chargebee webhooks guide first.

Why Chargebee sends duplicate webhooks

Chargebee, like most webhook providers, operates on an "at-least-once" delivery model rather than "exactly-once" delivery. This means your application might receive the same event multiple times. Understanding why duplicates occur is the first step to handling them effectively.

Timeout and retry behaviour

Chargebee expects your endpoint to respond within a specific timeframe. If your server doesn't acknowledge the webhook in time (whether due to slow processing, network latency, or server load), Chargebee assumes the delivery failed and schedules a retry.

The timeout values depend on your Chargebee site type:

When a webhook fails, Chargebee retries with an aggressive schedule:

1. 2 minutes after the failure
2. 6 minutes after the previous retry
3. 30 minutes after the previous retry
4. 1 hour after the previous retry
5. 5 hours after the previous retry
6. 1 day after the previous retry
7. 2 days after the previous retry

During this multi-day retry window, network issues or temporary server problems can trigger multiple delivery attempts for the same event.

Infrastructure and network conditions

In distributed systems, webhooks may occasionally be delivered more than once even without explicit retries. This isn't a bug but an intentional design choice that prioritises reliable delivery over preventing duplicates. Network hiccups, load balancer behaviours, and infrastructure failovers can all contribute to duplicate deliveries.

Why Chargebee sends webhooks out of order

Out-of-order delivery is a separate challenge from duplicates, though they often occur together.

Asynchronous processing

Chargebee explicitly warns in their documentation that webhooks are asynchronous and not recommended for time-critical applications. Multiple events for the same resource (like a subscription or customer) may be processed and dispatched at slightly different times, and network conditions can cause them to arrive in a different order than they occurred.

Retry-induced reordering

Consider this scenario: a customer's subscription is updated twice in quick succession. The first webhook fails due to a temporary network issue, while the second succeeds immediately. When the first webhook is retried minutes later, it arrives after the second but it's now carrying stale data that could overwrite more recent changes.

Techniques for handling duplicate webhooks

Since duplicate webhooks are inevitable, your application needs to handle them gracefully. The goal is idempotent processing: ensuring that processing the same event multiple times produces the same result as processing it once.

Track processed events by ID

Every Chargebee event includes a unique id field that remains consistent across retry attempts. This is the most reliable way to detect duplicates.

const processedEvents = new Map(); // In production, use Redis or a database

app.post("/webhooks/chargebee", async (req, res) => {
  const event = req.body;
  const eventId = event.id;

  // Check if we've already processed this event
  if (processedEvents.has(eventId)) {
    console.log(`Duplicate event ${eventId}, skipping`);
    return res.status(200).send("OK");
  }

  // Mark as processed before handling (prevents race conditions)
  processedEvents.set(eventId, Date.now());

  // Respond immediately to avoid timeout retries
  res.status(200).send("OK");

  // Process the webhook asynchronously
  try {
    await handleChargebeeEvent(event);
  } catch (error) {
    console.error(`Failed to process event ${eventId}:`, error);
    // Consider implementing a retry mechanism here
  }
});

Respond first, process later

One of the most effective ways to prevent duplicate deliveries is to respond to Chargebee immediately, then process the webhook asynchronously. This approach has two benefits:

1. It prevents timeout-triggered retries by responding within Chargebee's timeout window
2. It decouples your response time from your processing time

Given Chargebee's relatively short timeouts (20-60 seconds for live sites), this pattern is essential for any non-trivial processing.

Use persistent storage for event IDs

For production applications, store processed event IDs in a persistent datastore rather than in-memory. Redis is a good choice because of its speed and built-in TTL support:

const Redis = require("ioredis");
const redis = new Redis();

async function isEventProcessed(eventId) {
  const result = await redis.setnx(`chargebee:event:${eventId}`, "1");
  if (result === 1) {
    // First time seeing this event
    // Set TTL to cover Chargebee's retry window (about 3 days)
    await redis.expire(`chargebee:event:${eventId}`, 259200);
    return false;
  }
  return true;
}

app.post("/webhooks/chargebee", async (req, res) => {
  const event = req.body;

  if (await isEventProcessed(event.id)) {
    return res.status(200).send("OK");
  }

  res.status(200).send("OK");
  await handleChargebeeEvent(event);
});

The TTL should match or exceed Chargebee's retry window (approximately 3 days) to ensure you catch all possible duplicates.

Techniques for handling out-of-order webhooks

Detecting and handling out-of-order events requires a different approach than deduplication.

Use the resource_version field

Chargebee includes a resource_version field in webhook payloads that acts as a version number for each resource. This value increases with each change to a resource, allowing you to detect stale updates.

async function handleSubscriptionUpdate(event) {
  const subscription = event.content.subscription;
  const subscriptionId = subscription.id;
  const resourceVersion = subscription.resource_version;

  // Get the current stored version
  const currentVersion = await db.subscriptions.getResourceVersion(subscriptionId);

  // Only process if this is a newer version
  if (currentVersion && resourceVersion <= currentVersion) {
    console.log(`Stale event for subscription ${subscriptionId}, skipping`);
    return;
  }

  // Update with the new data and version
  await db.subscriptions.upsert({
    where: { chargebeeId: subscriptionId },
    create: {
      chargebeeId: subscriptionId,
      resourceVersion: resourceVersion,
      ...mapSubscriptionData(subscription),
    },
    update: {
      resourceVersion: resourceVersion,
      ...mapSubscriptionData(subscription),
    },
  });
}

The resource_version is available on most Chargebee resources including subscriptions, customers, invoices, and payment sources.

Combine deduplication with ordering

For robust webhook handling, combine both techniques:

app.post("/webhooks/chargebee", async (req, res) => {
  const event = req.body;

  // Step 1: Check for duplicate events
  if (await isEventProcessed(event.id)) {
    return res.status(200).send("OK");
  }

  // Respond immediately
  res.status(200).send("OK");

  // Step 2: Handle the event with version checking
  try {
    const eventType = event.event_type;
    const content = event.content;

    if (eventType.startsWith("subscription_")) {
      await handleSubscriptionEvent(content.subscription, eventType);
    } else if (eventType.startsWith("customer_")) {
      await handleCustomerEvent(content.customer, eventType);
    }
    // ... handle other event types
  } catch (error) {
    console.error(`Failed to process event ${event.id}:`, error);
  }
});

async function handleSubscriptionEvent(subscription, eventType) {
  const isNewer = await checkAndUpdateVersion(
    "subscriptions",
    subscription.id,
    subscription.resource_version
  );

  if (!isNewer) {
    console.log(`Ignoring stale ${eventType} for ${subscription.id}`);
    return;
  }

  // Process the subscription update
  await syncSubscription(subscription);
}

Make your processing idempotent

Beyond detecting duplicates and stale events at the webhook level, design your business logic to be idempotent. This provides defence in depth.

How Hookdeck simplifies webhook handling

While building deduplication and ordering logic into your application is possible, it adds complexity and requires maintaining additional infrastructure. Hookdeck provides an event gateway that handles these challenges before events reach your application.

What is Hookdeck?

Hookdeck is webhook infrastructure that sits between webhook sources (like Chargebee) and your application. It provides reliability features including queuing, retries, rate limiting, and deduplication.

Setting up deduplication with Hookdeck

With Hookdeck, you point Chargebee's webhooks to a Hookdeck URL, configure your deduplication rules, and let Hookdeck forward deduplicated events to your application.

Create a connection

In the Hookdeck dashboard, create a connection with Chargebee as the source. You'll receive a unique URL to configure in Chargebee's webhook settings.

Configure deduplication rules

Add a deduplication rule that uses the event ID as the key:

{
  "type": "deduplicate",
  "window": 300000,
  "include_fields": ["body.id"]
}

This configuration drops any webhook with the same event ID within a 5-minute window (300,000 milliseconds).

Update your Chargebee webhook URL

In Chargebee's Settings → Configure Chargebee → API Keys and Webhooks, replace your application's webhook URL with the Hookdeck source URL.

Deduplication strategies for Chargebee

Hookdeck offers flexible deduplication strategies for different scenarios:

Event ID deduplication: Use when Chargebee retries the same webhook multiple times:

{
  "type": "deduplicate",
  "window": 300000,
  "include_fields": ["body.id"]
}

Content-based deduplication: Use when you want to ignore events where only timestamps changed:

{
  "type": "deduplicate",
  "window": 300000,
  "exclude_fields": [
    "body.occurred_at"
  ]
}

By excluding volatile fields, you deduplicate based on the actual event content rather than metadata.

Additional Hookdeck benefits

Beyond deduplication, Hookdeck provides features that complement your Chargebee integration:

• Automatic retries: If your endpoint fails, Hookdeck retries with configurable backoff, independent of Chargebee's retry schedule
• Rate limiting: Protect your application from webhook spikes during bulk operations
• Transformations: Modify payloads before delivery using JavaScript—useful for normalising Chargebee's event structure
• Filters: Route specific event types to different endpoints
• Observability: Monitor webhook delivery, latency, and failures in real-time
• Local development: Route webhooks to localhost during development without exposing your machine

Conclusion

Out-of-order and duplicate Chargebee webhooks are an inevitable part of building reliable subscription integrations. By understanding why these issues occur and implementing proper handling you can build robust webhook handlers that maintain data consistency regardless of delivery anomalies.

Additional resources

• Chargebee Webhook Documentation
• Chargebee Events API
• Hookdeck Documentation
• Hookdeck Deduplication