# Slack webhooks in production: duplicates, ordering, retries and timeouts

Agent skill: `npx skills add hookdeck/webhook-skills --skill slack-webhooks` ([details](https://hookdeck.com/webhooks/slack/skill))

This guide covers the problems Slack webhooks cause in production and how to handle each one. What Slack sends and how it's signed is on the [Slack webhooks](/webhooks/slack) overview, every event type is on [Slack webhook events](/webhooks/slack/events), and signature verification code is on [Verify Slack webhooks](/webhooks/slack/verify).

Slack uses the word webhook for features that run in opposite directions. The **Events API** sends requests to your Request URL, and most of this guide is about it. **Incoming webhooks** are requests you send to a channel's URL, and their limits are in [Rate limits and incoming webhooks](#other-limits). To send another service's events into Slack, see [how to send external webhooks to Slack](/webhooks/platforms/how-to-receive-and-replay-external-webhooks-in-slack-with-hookdeck).

## At a glance

| Problem | What to do | With Event Gateway |
| --- | --- | --- |
| [Timeouts](#timeouts) | Return a 2xx within 3 seconds; process in a background job | Accepts the delivery at once, then queues it for your service |
| [Duplicates](#duplicates) | Record processed `event_id` values; make handlers idempotent | Drops repeats before they reach your handler |
| [Ordering](#ordering) | Don't rely on arrival order; fetch current state when order matters | Logs every event and attempt, so you can see arrival order when debugging |
| [Retries](#retries) | Stay under the 95% failure limit; turn on Delayed Events; reconcile after an outage | Accepts deliveries while your service is down and retries them |
| [Choosing events](#choosing-events) | Subscribe to what you process; switch on `event.type`, not the subscription name | Filters and routes events by type |
| [Testing](#testing) | Use Socket Mode or a tunnel; test the challenge and a bad signature | Sample payloads in Inspect, and forwarding and replay with the CLI |
| [Rate limits and incoming webhooks](#other-limits) | Alert on `app_rate_limited`; respect `Retry-After`; keep webhook URLs secret | Queues what you send to Slack and delivers it at a rate you set |

## Slack webhook timeouts and slow handlers {#timeouts}

Respond with an HTTP 2xx within 3 seconds, then do the work in a background job.

**Why it happens:** Slack's Events API [treats a response that takes longer than three seconds as a failed attempt](https://docs.slack.dev/apis/events-api#responding), and nothing in its docs lets you extend that. The retry then arrives with `x-slack-retry-reason: http_timeout`, and it counts against your [failure limit](#retries). Handlers that write to a database or call other APIs before responding are the usual cause, and so are serverless cold starts, which use up part of the 3 seconds before your code runs.

**Fix it in your handler:**

1. Verify the signature and return a 200 straight away.
2. Push the event onto a queue and process it in a worker. See [why to process webhooks asynchronously](/webhooks/guides/why-implement-asynchronous-processing-webhooks).
3. Keep the request path free of calls to other services, including the Slack Web API.

Socket Mode doesn't remove this work. [You still acknowledge each event](https://docs.slack.dev/apis/events-api/using-socket-mode#acknowledge) so Slack knows whether to retry, and [Slack recommends HTTP for production](https://docs.slack.dev/apis/events-api/comparing-http-socket-mode#which-one).

**With Event Gateway:** Hookdeck accepts each delivery within Slack's limit, then delivers events to your service with its own timeouts and retries. Your handler can take as long as it needs.

## Duplicate Slack webhook events {#duplicates}

Assume an event can arrive more than once, and deduplicate on `event_id`.

**Why it happens:** Slack retries any attempt it counts as failed, so a handler that finishes its work and then responds late sees the same event again. Slack describes `event_id` as a unique identifier for the event, globally unique across workspaces. Its docs don't state outright that a retry repeats the value, though a retry is a repeat of that event.

**Fix it in your handler:**

1. Record the `event_id` of each event you process, in a table with a unique constraint or a store with a TTL.
2. Make every handler [idempotent](/webhooks/guides/implement-webhook-idempotency): the same result however many times it runs.
3. Don't drop a request because it has an `x-slack-retry-num` header. That header says Slack is retrying, not that you finished the first attempt.

```javascript
async function processSlackEvent(payload) {
  const { event_id, event } = payload;

  // SET NX returns null when the key already exists
  const first = await redis.set(`processed:${event_id}`, '1', 'EX', 86400, 'NX');
  if (!first) return;

  try {
    await handleEvent(event);
  } catch (err) {
    // Let a later attempt process it
    await redis.del(`processed:${event_id}`);
    throw err;
  }
}

```

**With Event Gateway:** Hookdeck [deduplicates](/docs/deduplication) events before they reach your endpoint, so repeats are filtered at the infrastructure level rather than in each handler.

## Out-of-order Slack webhook events {#ordering}

Slack makes no general promise about delivery order, so don't rely on it.

**Why it happens:** Each event is its own HTTP request, and retries come up to 5 minutes later, or up to 24 hours later with Delayed Events, so a later event can arrive first. Slack calls the Events API a best-effort system whose delivery can be delayed during incidents. The one ordering statement in its docs is [on `app_uninstalled`](https://docs.slack.dev/reference/events/app_uninstalled): you can receive `tokens_revoked` after it. For more on the general problem, see [Webhook Ordering: Why It's Hard and How to Handle It](/webhooks/guides/webhook-ordering-why-its-hard-and-how-to-handle-it).

**Fix it in your handler:**

1. Compare `event.event_ts` before you apply an update, so an older event can't overwrite newer state. `event_time` at the top level is when Slack dispatched the event.
2. When order matters, treat the event as a signal and fetch current state from the Web API, such as `conversations.info` for a channel.
3. Handle `app_uninstalled` and `tokens_revoked` in either order.

**With Event Gateway:** Hookdeck logs every event and delivery attempt, so when state looks wrong you can see the order events actually arrived in and replay them.

## Slack webhook retries and disabled subscriptions {#retries}

Slack [retries three times](https://docs.slack.dev/apis/events-api#retries) over a few minutes, and if most deliveries keep failing it turns your event subscriptions off. Keep the endpoint up, turn on Delayed Events, and reconcile after any outage.

**Why it happens:** The first retry is nearly immediate, the second comes after 1 minute and the third after 5 minutes. Each carries `x-slack-retry-num` and an `x-slack-retry-reason` of `http_timeout`, `too_many_redirects`, `connection_failed`, `ssl_error`, `http_error` or `unknown_error`. Slack's docs describe no way to resend an event or browse past deliveries. Delivery can also stop altogether:

* **The failure limit.** If timeouts, SSL failures, more than 2 redirects or non-2xx responses make up [more than 95% of delivery attempts within 60 minutes](https://docs.slack.dev/apis/events-api#failure-limits), Slack temporarily disables your event subscriptions and emails the app's creator and owner. Retries count towards this. Apps receiving fewer than 1,000 events an hour aren't automatically disabled.
* **The two-hour cutoff.** By default Slack [won't attempt an event more than two hours late](https://docs.slack.dev/apis/events-api#delayed-events-retry), which matters during Slack's own incidents as well as yours.

**Fix it in your handler:**

1. Turn on **Delayed Events** under Event Subscriptions. Slack then follows the three retries with hourly retries for 24 hours, and attempts late events regardless of the two-hour cutoff.
2. Make sure someone reads the emails sent to your app's owner, and [re-enable deliveries in your app's settings](https://docs.slack.dev/apis/events-api#resume-event-deliveries) once the endpoint is fixed.
3. Return `x-slack-no-retry: 1` with a non-200 response when you don't want an event sent again. Slack's docs say only that it stops re-delivery of that event, so don't assume the response leaves your failure count alone.
4. After an outage, reconcile what the Web API can tell you. `conversations.history` returns messages you may have missed.

**With Event Gateway:** Hookdeck acknowledges Slack's deliveries while your service is down, so your outage doesn't count against Slack's failure limit, and it [retries](/docs/retries) to your service on your schedule, well past Slack's few minutes. You can inspect failed attempts and replay them once the endpoint is fixed.

## Choosing which Slack events to listen to {#choosing-events}

Subscribe only to the events you process, and route on `event.type`. The name you subscribe to and the value you receive don't always match.

**Why it matters:** Your app has one Request URL for every event and every workspace, so all of it arrives at one handler. Several names on the [event reference](https://docs.slack.dev/reference/events) also behave differently from how they read:

* **Subscription names.** `message.channels`, `message.groups`, `message.im` and `message.mpim` are names you subscribe to. Each delivery has `event.type: message`, and [`event.channel_type`](https://docs.slack.dev/reference/events/message.channels) says where it was posted.
* **Message subtypes.** Edits, deletions, joins and similar changes arrive as `event.type: message` with an `event.subtype` such as `message_changed`. [Not all subtypes are deliverable in the Events API](https://docs.slack.dev/reference/events/message#subtypes).
* **No inner event.** `url_verification` and `app_rate_limited` have no `event` object, so their name is in the outer `type`.
* **RTM-only events.** The reference also lists events for the legacy RTM API, such as `hello`, `user_typing` and `presence_change`, which don't reach a Request URL. Each page's Compatible APIs says which. The list on [Slack webhook events](/webhooks/slack/events) leaves those out.

**Fix it in your handler:**

1. Check the outer `type` first, then switch on `event.type`.
2. Use `app_mention` if your app only needs messages that mention it, instead of subscribing to every message in a channel. [Direct messages don't trigger it](https://docs.slack.dev/reference/events/app_mention), so add `message.im` for those.
3. Check for `bot_id` when you handle `message` events, so your app can tell a bot's message from a person's.
4. Remember that scopes control delivery. An event you subscribe to doesn't arrive until the installing user has [granted its OAuth scope](https://docs.slack.dev/apis/events-api#modify-events), so adding a new scope means sending existing users through OAuth again.
5. Slack sends one event when several installed users can see it, with one user in `authorizations`. Call `apps.event.authorizations.list` if you need all of them.

**With Event Gateway:** Hookdeck [filters](/docs/filters) and routes events by type or any other property, so one Slack Request URL can feed several services.

## Testing and debugging Slack webhooks locally {#testing}

Use Socket Mode or a tunnel, and test the URL challenge and a bad signature as well as real events.

**Why it happens:** Slack needs a public HTTPS URL and [won't save it until your endpoint answers the challenge](https://docs.slack.dev/apis/events-api/using-http-request-urls#verification), so your handler has to be running at that moment. A failed challenge means no events at all, with nothing to inspect. Request URLs are case-sensitive, and an endpoint that's slow to wake up can use the **Retry** button in your app's settings.

**Fix it in your handler:**

1. For early development, use [Socket Mode](https://docs.slack.dev/apis/events-api/comparing-http-socket-mode#which-one), which Slack recommends for working locally because it needs no public URL. Test over HTTP before you deploy, since signature checks and the challenge only happen there.
2. Answer `url_verification` with HTTP 200 and the `challenge` value, as plain text, form-encoded or JSON.
3. Send your endpoint a request with a bad signature and one with an old timestamp, and check that both are rejected.
4. Generate real events in a test workspace: post a message, add a reaction, mention your app. Slack's docs describe no way to send a test event.
5. Try the failure paths too: a handler that throws, and one that takes longer than 3 seconds.

**With Event Gateway:** [Hookdeck Inspect](https://inspect.hookdeck.com/webhooks/slack) shows sample Slack payloads with no account or setup. The [Hookdeck CLI](/docs/cli) forwards events to your machine and [replays past ones](/webhooks/platforms/how-to-test-and-replay-slack-webhooks-locally-with-hookdeck), so you can test a fix without triggering the event again in Slack.

## Rate limits and incoming webhooks {#other-limits}

**The Events API cap.** Event deliveries [max out at 30,000 per workspace per app per 60 minutes](https://docs.slack.dev/apis/events-api#rate-limiting). Above that Slack sends `app_rate_limited` every minute you're limited, with `minute_rate_limited` saying when it began. The limit applies to what Slack sends, so a busy workspace can reach it however fast your endpoint is. Slack's docs don't say whether withheld events are sent later, so alert on this event, reconcile from the Web API, and drop subscriptions you don't need.

The rest of this section is about **incoming webhooks**, where you're the sender. They have no signature, retries or delivery log on Slack's side, so handle failures in your own code.

**Rate limits.** Apps [may post no more than one message per second per channel](https://docs.slack.dev/apis/web-api/rate-limits#posting-messages), with short bursts allowed. Over the limit, Slack returns a `429` with a `Retry-After` header giving the seconds to wait. Queue messages on your side and respect that header.

**Failed sends.** Success is a `200` with the body `ok`. [Errors come back as strings](https://docs.slack.dev/messaging/sending-messages-using-incoming-webhooks#handling_errors) alongside status codes such as `400`, `403` and `404`. Don't retry `invalid_payload`, `action_prohibited`, `channel_not_found` or `channel_is_archived` without fixing the cause. `no_service` means the webhook has been disabled, removed or is invalid.

**The URL is the credential.** It contains a secret, and anyone who has it can post to your channel. Keep it out of source control and logs, since [Slack actively searches out and revokes leaked secrets](https://docs.slack.dev/messaging/sending-messages-using-incoming-webhooks#create_a_webhook), which also stops your messages.

**One channel per URL.** You [can't override the channel, username or icon](https://docs.slack.dev/messaging/sending-messages-using-incoming-webhooks#advanced_message_formatting) in the payload, so posting to ten channels takes ten URLs. Call `chat.postMessage` with a bot token if you need to choose the channel at runtime or delete messages later.

**Threads.** An incoming webhook can post into a thread with `thread_ts`, but [it doesn't return the `ts` of what you sent](https://docs.slack.dev/messaging/sending-messages-using-incoming-webhooks#threads). You'd get that from a message event, `conversations.history` or `assistant.search.context`.

**Message limits.** A message can have at most 100 attachments, and more returns `too_many_attachments`. Include `text` alongside `blocks`, since it's the fallback, and a payload with no `text` can return `no_text`.

**With Event Gateway:** Send to Hookdeck and make your Slack URL its destination. You [set a maximum delivery rate](/docs/destinations#set-a-max-delivery-rate), Hookdeck queues what's above it and [retries](/docs/retries) failures, a [transformation](/docs/transformations) can turn another service's event into a Slack message, and your sources never see the Slack URL. Hookdeck can't raise the Events API cap, because Slack applies that before anything is delivered.