Slack webhooks in production: duplicates, ordering, retries and timeouts
This guide covers the problems Slack webhooks cause in production and how to handle each one. What Slack sends and how it's signed is on the Slack webhooks overview, every event type is on Slack webhook events, and signature verification code is on Verify Slack webhooks.
Slack uses the word webhook for features that run in opposite directions. The Events API sends requests to your Request URL, and most of this guide is about it. Incoming webhooks are requests you send to a channel's URL, and their limits are in Rate limits and incoming webhooks. To send another service's events into Slack, see how to send external webhooks to Slack.
At a glance
| Problem | What to do | With Event Gateway |
|---|---|---|
| Timeouts | Return a 2xx within 3 seconds; process in a background job | Accepts the delivery at once, then queues it for your service |
| Duplicates | Record processed event_id values; make handlers idempotent | Drops repeats before they reach your handler |
| Ordering | Don't rely on arrival order; fetch current state when order matters | Logs every event and attempt, so you can see arrival order when debugging |
| Retries | Stay under the 95% failure limit; turn on Delayed Events; reconcile after an outage | Accepts deliveries while your service is down and retries them |
| Choosing events | Subscribe to what you process; switch on event.type, not the subscription name | Filters and routes events by type |
| Testing | Use Socket Mode or a tunnel; test the challenge and a bad signature | Sample payloads in Inspect, and forwarding and replay with the CLI |
| Rate limits and incoming webhooks | Alert on app_rate_limited; respect Retry-After; keep webhook URLs secret | Queues what you send to Slack and delivers it at a rate you set |
Slack webhook timeouts and slow handlers
Respond with an HTTP 2xx within 3 seconds, then do the work in a background job.
Why it happens: Slack's Events API treats a response that takes longer than three seconds as a failed attempt, and nothing in its docs lets you extend that. The retry then arrives with x-slack-retry-reason: http_, and it counts against your failure limit. Handlers that write to a database or call other APIs before responding are the usual cause, and so are serverless cold starts, which use up part of the 3 seconds before your code runs.
Fix it in your handler:
- Verify the signature and return a 200 straight away.
- Push the event onto a queue and process it in a worker. See why to process webhooks asynchronously.
- Keep the request path free of calls to other services, including the Slack Web API.
Socket Mode doesn't remove this work. You still acknowledge each event so Slack knows whether to retry, and Slack recommends HTTP for production.
With Event Gateway: Hookdeck accepts each delivery within Slack's limit, then delivers events to your service with its own timeouts and retries. Your handler can take as long as it needs.
Duplicate Slack webhook events
Assume an event can arrive more than once, and deduplicate on event_id.
Why it happens: Slack retries any attempt it counts as failed, so a handler that finishes its work and then responds late sees the same event again. Slack describes event_id as a unique identifier for the event, globally unique across workspaces. Its docs don't state outright that a retry repeats the value, though a retry is a repeat of that event.
Fix it in your handler:
- Record the
event_idof each event you process, in a table with a unique constraint or a store with a TTL. - Make every handler idempotent: the same result however many times it runs.
- Don't drop a request because it has an
x-slack-retry-numheader. That header says Slack is retrying, not that you finished the first attempt.
async function processSlackEvent(payload) {
const { event_id, event } = payload;
// SET NX returns null when the key already exists
const first = await redis.set(`processed:${event_id}`, '1', 'EX', 86400, 'NX');
if (!first) return;
try {
await handleEvent(event);
} catch (err) {
// Let a later attempt process it
await redis.del(`processed:${event_id}`);
throw err;
}
}
With Event Gateway: Hookdeck deduplicates events before they reach your endpoint, so repeats are filtered at the infrastructure level rather than in each handler.
Out-of-order Slack webhook events
Slack makes no general promise about delivery order, so don't rely on it.
Why it happens: Each event is its own HTTP request, and retries come up to 5 minutes later, or up to 24 hours later with Delayed Events, so a later event can arrive first. Slack calls the Events API a best-effort system whose delivery can be delayed during incidents. The one ordering statement in its docs is on app_uninstalled: you can receive tokens_revoked after it. For more on the general problem, see Webhook Ordering: Why It's Hard and How to Handle It.
Fix it in your handler:
- Compare
event.event_tsbefore you apply an update, so an older event can't overwrite newer state.event_timeat the top level is when Slack dispatched the event. - When order matters, treat the event as a signal and fetch current state from the Web API, such as
conversations.infofor a channel. - Handle
app_uninstalledandtokens_revokedin either order.
With Event Gateway: Hookdeck logs every event and delivery attempt, so when state looks wrong you can see the order events actually arrived in and replay them.
Slack webhook retries and disabled subscriptions
Slack retries three times over a few minutes, and if most deliveries keep failing it turns your event subscriptions off. Keep the endpoint up, turn on Delayed Events, and reconcile after any outage.
Why it happens: The first retry is nearly immediate, the second comes after 1 minute and the third after 5 minutes. Each carries x-slack-retry-num and an x-slack-retry-reason of http_timeout, too_many_redirects, connection_failed, ssl_error, http_error or unknown_error. Slack's docs describe no way to resend an event or browse past deliveries. Delivery can also stop altogether:
- The failure limit. If timeouts, SSL failures, more than 2 redirects or non-2xx responses make up more than 95% of delivery attempts within 60 minutes, Slack temporarily disables your event subscriptions and emails the app's creator and owner. Retries count towards this. Apps receiving fewer than 1,000 events an hour aren't automatically disabled.
- The two-hour cutoff. By default Slack won't attempt an event more than two hours late, which matters during Slack's own incidents as well as yours.
Fix it in your handler:
- Turn on Delayed Events under Event Subscriptions. Slack then follows the three retries with hourly retries for 24 hours, and attempts late events regardless of the two-hour cutoff.
- Make sure someone reads the emails sent to your app's owner, and re-enable deliveries in your app's settings once the endpoint is fixed.
- Return
x-slack-no-retry: 1with a non-200 response when you don't want an event sent again. Slack's docs say only that it stops re-delivery of that event, so don't assume the response leaves your failure count alone. - After an outage, reconcile what the Web API can tell you.
conversations.historyreturns messages you may have missed.
With Event Gateway: Hookdeck acknowledges Slack's deliveries while your service is down, so your outage doesn't count against Slack's failure limit, and it retries to your service on your schedule, well past Slack's few minutes. You can inspect failed attempts and replay them once the endpoint is fixed.
Choosing which Slack events to listen to
Subscribe only to the events you process, and route on event.type. The name you subscribe to and the value you receive don't always match.
Why it matters: Your app has one Request URL for every event and every workspace, so all of it arrives at one handler. Several names on the event reference also behave differently from how they read:
- Subscription names.
message.channels,message.groups,message.imandmessage.mpimare names you subscribe to. Each delivery hasevent.type: message, andevent.channel_typesays where it was posted. - Message subtypes. Edits, deletions, joins and similar changes arrive as
event.type: messagewith anevent.subtypesuch asmessage_changed. Not all subtypes are deliverable in the Events API. - No inner event.
url_verificationandapp_rate_limitedhave noeventobject, so their name is in the outertype. - RTM-only events. The reference also lists events for the legacy RTM API, such as
hello,user_typingandpresence_change, which don't reach a Request URL. Each page's Compatible APIs says which. The list on Slack webhook events leaves those out.
Fix it in your handler:
- Check the outer
typefirst, then switch onevent.type. - Use
app_mentionif your app only needs messages that mention it, instead of subscribing to every message in a channel. Direct messages don't trigger it, so addmessage.imfor those. - Check for
bot_idwhen you handlemessageevents, so your app can tell a bot's message from a person's. - Remember that scopes control delivery. An event you subscribe to doesn't arrive until the installing user has granted its OAuth scope, so adding a new scope means sending existing users through OAuth again.
- Slack sends one event when several installed users can see it, with one user in
authorizations. Callapps.event.authorizations.listif you need all of them.
With Event Gateway: Hookdeck filters and routes events by type or any other property, so one Slack Request URL can feed several services.
Testing and debugging Slack webhooks locally
Use Socket Mode or a tunnel, and test the URL challenge and a bad signature as well as real events.
Why it happens: Slack needs a public HTTPS URL and won't save it until your endpoint answers the challenge, so your handler has to be running at that moment. A failed challenge means no events at all, with nothing to inspect. Request URLs are case-sensitive, and an endpoint that's slow to wake up can use the Retry button in your app's settings.
Fix it in your handler:
- For early development, use Socket Mode, which Slack recommends for working locally because it needs no public URL. Test over HTTP before you deploy, since signature checks and the challenge only happen there.
- Answer
url_verificationwith HTTP 200 and thechallengevalue, as plain text, form-encoded or JSON. - Send your endpoint a request with a bad signature and one with an old timestamp, and check that both are rejected.
- Generate real events in a test workspace: post a message, add a reaction, mention your app. Slack's docs describe no way to send a test event.
- Try the failure paths too: a handler that throws, and one that takes longer than 3 seconds.
With Event Gateway: Hookdeck Inspect shows sample Slack payloads with no account or setup. The Hookdeck CLI forwards events to your machine and replays past ones, so you can test a fix without triggering the event again in Slack.
Rate limits and incoming webhooks
The Events API cap. Event deliveries max out at 30,000 per workspace per app per 60 minutes. Above that Slack sends app_rate_limited every minute you're limited, with minute_rate_limited saying when it began. The limit applies to what Slack sends, so a busy workspace can reach it however fast your endpoint is. Slack's docs don't say whether withheld events are sent later, so alert on this event, reconcile from the Web API, and drop subscriptions you don't need.
The rest of this section is about incoming webhooks, where you're the sender. They have no signature, retries or delivery log on Slack's side, so handle failures in your own code.
Rate limits. Apps may post no more than one message per second per channel, with short bursts allowed. Over the limit, Slack returns a 429 with a Retry-After header giving the seconds to wait. Queue messages on your side and respect that header.
Failed sends. Success is a 200 with the body ok. Errors come back as strings alongside status codes such as 400, 403 and 404. Don't retry invalid_payload, action_prohibited, channel_not_found or channel_is_archived without fixing the cause. no_service means the webhook has been disabled, removed or is invalid.
The URL is the credential. It contains a secret, and anyone who has it can post to your channel. Keep it out of source control and logs, since Slack actively searches out and revokes leaked secrets, which also stops your messages.
One channel per URL. You can't override the channel, username or icon in the payload, so posting to ten channels takes ten URLs. Call chat.postMessage with a bot token if you need to choose the channel at runtime or delete messages later.
Threads. An incoming webhook can post into a thread with thread_ts, but it doesn't return the ts of what you sent. You'd get that from a message event, conversations.history or assistant.search.context.
Message limits. A message can have at most 100 attachments, and more returns too_many_attachments. Include text alongside blocks, since it's the fallback, and a payload with no text can return no_text.
With Event Gateway: Send to Hookdeck and make your Slack URL its destination. You set a maximum delivery rate, Hookdeck queues what's above it and retries failures, a transformation can turn another service's event into a Slack message, and your sources never see the Slack URL. Hookdeck can't raise the Events API cap, because Slack applies that before anything is delivered.
Keep an outage from disabling your Slack event subscriptions.
Hookdeck Event Gateway answers Slack's URL verification, accepts every delivery within Slack's 3 seconds, verifies the signature, queues events and retries to your service on your schedule.