Mailgun webhooks in production: duplicates, ordering, retries and timeouts
This guide covers the problems Mailgun webhooks cause in production and how to handle each one. Most of it is about event webhooks, which report what happens to email you send. Inbound Routes, which post email you receive, have their own section. What Mailgun sends and how it's signed is on the Mailgun webhooks overview, every event type is on Mailgun webhook events, and signature verification code is on Verify Mailgun webhooks.
At a glance
| Problem | What to do | With Event Gateway |
|---|---|---|
| Timeouts | Return exactly 200 fast and queue the work; Mailgun publishes no limit | Accepts the POST at once, then queues it for your service |
| Duplicates | Deduplicate on event-data.id, keeping keys for about a day | Can drop repeats by field before they reach your handler |
| Ordering | Compare timestamp and never move a message back a state | Logs every event and attempt, so you can see arrival order when debugging |
| Retries | Stay available, and reconcile delivered from the Logs API | Accepts POSTs while your service is down and replays them |
| Choosing events | Subscribe to what you process; match failed and read severity | Filters and routes events by their contents |
| Testing | Use a temporary bin, then real email through a tunnel | Forwards to localhost and replays past events |
| Other limits | Plan for regions, URL limits, truncation and redaction | Gives Mailgun one stable URL and fans out to many destinations |
| Inbound Routes | Handle form fields or headers, and retries of their own | Accepts the POST while your service is down |
Mailgun webhook timeouts and slow handlers
Return a 200 as fast as you can, then process the event in a background job. Mailgun's docs don't publish a response timeout or say how they treat a slow response, so don't design around a number.
Why it happens: Mailgun acts on the status code you return. A 200 is a success and a 406 is a rejection, and neither is retried. Mailgun retries any other code, so a handler that returns a 201 or a 204 gets the same event again even though it worked. A handler that does slow work before responding can also fail for reasons unrelated to the event, which leads to more retries.
Fix it in your handler:
- Verify the signature and return a 200 straight away. Return nothing but a 200 for success.
- Push the event onto a queue (SQS, RabbitMQ or Redis) and process it in a worker. See why to process webhooks asynchronously.
- Return a 406 only when you never want that POST again, such as a failed signature check. Mailgun's Go SDK example does this. Other codes, 4xx included, bring the event back.
With Event Gateway: Hookdeck accepts each Mailgun POST immediately, then queues it and delivers it to your service at a rate it can handle, with its own retries. A slow handler doesn't fail a Mailgun delivery.
Duplicate Mailgun webhooks
A Mailgun retry can repeat an event you already processed, for instance when your handler did the work and then returned something other than a 200. Deduplicate on event-data.id.
Why it happens: Mailgun retries whenever it doesn't see a 200 or a 406. Its docs don't say whether a retry carries the same signature.token, so don't use the token as your only idempotency key. Mailgun does avoid one duplicate for you: the same URL configured for the same event at both domain and account level gets that event once. Different URLs each get their own copy, and a parent account's webhook and a subaccount's webhook both fire.
Fix it in your handler:
- Track processed
event-data.idvalues, using a unique constraint or a fast store with an expiry. See how to implement idempotency. - Expire keys after about a day. Mailgun says the ID is unique within a day, so a key kept much longer could match a different event, and retries finish within about 8 hours.
- Don't use
message.headers.message-idas the key. It identifies the message, which has many events.
With Event Gateway: Hookdeck's deduplication can use specific fields as its key, so you can drop repeats on event-data.id before they reach your endpoint.
Out-of-order Mailgun webhooks
Mailgun's docs make no promise about delivery order. Write handlers that accept events in any order, and don't assume an opened arrives after its delivered.
Why it happens: A failed POST is retried minutes or hours later while newer events keep arriving. delivered isn't retried, so it can go missing altogether while later events for that message arrive. Mailgun also says its events are generated by physical hosts and follow different routes to its event storage, so they don't always appear in the order they occurred. For more on the general problem, see Webhook Ordering: Why It's Hard and How to Handle It.
Fix it in your handler:
- Group events by
message.headers.message-idand comparetimestamp, which says when the event was generated, in Unix epoch seconds with a fractional part in Mailgun's examples. - Rank your states so a late
acceptedcan't overwritedelivered, instead of storing whichever event came last. - Don't treat a temporary failure as final.
failedwithseverityset totemporarymeans Mailgun will try again, so adeliveredcan follow it.delivery-status.attempt-nosays which attempt it was.
With Event Gateway: Hookdeck logs every event and delivery attempt, so when state looks wrong you can see the order events actually arrived in and replay them.
Mailgun webhook retries and lost events
Mailgun retries a failed POST for about 8 hours, then stops, and it never retries the delivery notification. Keep the endpoint available and reconcile what you miss from the Logs API.
Why it happens: Webhook Retries lists intervals of 5, 10 and 15 minutes, then 1, 2 and 4 hours, and Mailgun's docs describe no way to change them. The page on receiving messages by HTTP lists a different set for the same behaviour (10, 15 and 30 minutes, then 1, 2 and 4 hours), so don't build logic on the exact timing. Both exclude the delivery notification, so a delivered event that fails during an outage isn't sent again. Mailgun's docs describe no way to resend a webhook yourself, no dead letter queue and no delivery log for webhooks.
Fix it in your handler:
- Run webhook endpoints on high-availability infrastructure, and alert on your own error rate and on gaps in expected traffic.
- Store each raw POST as it arrives, so you know what you've processed.
- After an outage, reconcile against the Logs API. Mailgun retains event data for 30 days, though access varies by plan, and its older Events API is deprecated. Query a range that ends well before now, because new events can still be on their way to storage. Mailgun's polling guidance uses half an hour as its example.
- See dead letter queues for the failures on your side.
With Event Gateway: Hookdeck captures every Mailgun POST while your services are down, so Mailgun's deliveries succeed, including the delivered notifications it wouldn't send twice. Failed events can be retried automatically, manually or in bulk once you recover.
Choosing which Mailgun events to listen to
Subscribe only to the event types you process, then read event-data.event. The names you subscribe with and the names you receive aren't quite the same.
Why it matters: Webhooks are configured per event type, at domain or account level. There are 8 types: accepted, delivered, temporary_fail, permanent_fail, opened, clicked, unsubscribed and complained. The full list, with sample payloads, is on Mailgun webhook events. rejected, stored and the mailing list events appear in Mailgun's logs but can't be subscribed to as webhooks.
Fix it in your handler:
- Read
event-data.event, not a top-levelevent. Mailgun's Webhook Payloads page shows only the object insideevent-data. - Match
failed.temporary_failandpermanent_failboth arrive asfailed, so readevent-data.severityto tell them apart. - Remove addresses on
permanent_fail, nottemporary_fail. Readdelivery-status.bounce-typeandreasonif you need the detail. - Turn on what the events depend on.
openedandclickedneed tracking on and the CNAME record pointing tomailgun.org,unsubscribedneeds unsubscribe tracking, andcomplaineddepends on feedback loops.
With Event Gateway: Hookdeck filters match on request contents, so you can route on event-data.event and event-data.severity, and one source can feed several connections.
Testing and debugging Mailgun webhooks locally
Point a webhook at a temporary bin to see what arrives, then send real email through a tunnel, because Mailgun's published examples don't show the whole body.
Why it matters: Mailgun needs a URL it can reach, so it can't post to localhost. Its Webhook Payloads examples also leave out the signature and event-data wrapper, so fixtures copied straight from that page don't match a real delivery.
How to test it:
- Create a temporary URL at
bin.mailgun.net, which Mailgun recommends for this, and set it as the webhook URL to read what Mailgun sends. - Send real email and follow a message from
acceptedtodeliveredtoopenedandclicked. Then try an address that bounces. Allow up to 10 minutes for an account-level webhook change to apply. - Take your endpoint offline for a while to watch the retries, and return a 406 to confirm they stop.
- Keep fixtures of the whole body for your tests. Mailgun webhook events has sample payloads to start from.
With Event Gateway: The Hookdeck CLI receives webhooks on your machine without a public server, and you can replay past events with one click.
Other Mailgun webhook limits
Plan for certificates, regions, URL limits and fields that vary before they reach production.
HTTPS certificates. An HTTPS endpoint needs a certificate signed by a trusted CA. A self-signed certificate won't work.
Regions. US and EU are isolated, so create webhooks in each region you send from.
URL limits. Each event type can have up to 3 URLs.
Custom variables. v: variables you set when sending appear under user-variables, and anything over 4 KB is truncated in events and webhooks. Store large data elsewhere and send a reference.
Fields that vary. Mailgun says its examples aren't exhaustive and warns against hardcoding fields, because new ones can appear. Some also change shape: user-variables is an empty array in its examples, and storage is an object on most examples but an empty array on unsubscribed.
Personal data. Optional PII redaction replaces values such as recipient with a placeholder like [REDACTED], at account or domain level. message-id stays intact, so correlate on that if you turn it on. Redaction is inherited by subaccounts.
Legacy webhooks. Mailgun's older legacy webhooks post form fields with event at the top level, so nothing above about event-data applies to them. Mailgun made their API read only on April 15, 2023 and says they all need migrating, so move any you still have to the JSON webhooks.
Alerts. Mailgun's sending alerts can also go to a webhook. That's a separate feature with its own signing key and X-Sign header, and it isn't what these pages describe.
With Event Gateway: Hookdeck gives Mailgun one stable URL, so you don't use up its 3 URLs, and sends each event to as many destinations as you need.
Mailgun Inbound Routes
Inbound Routes are a separate surface. A route's forward() or store(notify=...) action posts email received at your domain, so there's no event-data or event type, and the sections above on events don't apply. Return a 200 quickly, and check which format your URL gets.
How it differs: For forward(), the content type depends on your URL and the message. A URL ending in json gets application/json, one ending in mime gets application/x-www-form-urlencoded with a raw body-mime field, and any other URL gets application/x-www-form-urlencoded, or multipart/form-data when the email has attachments. store(notify=...) posts form-encoded, or multipart with attachments. A POST carries fields such as sender, recipient, subject, body-plain and stripped-text.
Set it up safely:
- Verify every POST with the same Webhook Signing Key.
timestamp,tokenandsignatureare top-level form fields here, not asignatureobject. - Handle JSON routes differently. A URL ending in
jsongetsX-Mailgun-SignatureandX-Mailgun-Timestampheaders instead, and the whole request body takes the place oftokenin the HMAC. - Return a 200 for success and a 406 to reject a message. The same rules apply as for event webhooks, and Mailgun retries any other code for about 8 hours.
With Event Gateway: Hookdeck accepts multipart requests and accepts each POST while your service is down, so an outage doesn't use up Mailgun's retries. See building AI email triage with Mailgun and Claude for a worked example.
Don't lose delivered events to an outage.
Mailgun never retries its delivery notification. Hookdeck Event Gateway accepts every Mailgun POST, verifies its signature, queues it and retries to your service on your schedule, with a log you can replay from.