Retries

A retry is a new delivery attempt for an event. Retries can be triggered manually on any event, scheduled or automatically on failed events.

Events are limited to 50 automatic retries, but can be manually retried as many times as you like.

Manually retry events

Manually retrying events allows you to re-attempt delivery to a destination.

Manual retries are useful when troubleshooting, testing, or after resolving an issue on the destination. If a manual retry is successful, any further automatic retries on that event will be canceled – otherwise, future retries will continue as planned.

Manual retries can be triggered individually or in bulk.

Retry a single event

{
  "id": "evt_njCeMZHzNhJWdkkZ7PLCfEw5",
  "team_id": "tm_nlcetVe8k1lMAY0KR0OxNuHr",
  "webhook_id": "web_UfM5ZY2wxLD6sLhDaKMcx5wp",
  "source_id": "src_9YmY0SHklrv6zBQz78cKGwSW",
  "destination_id": "des_fYYCay655IHwbOMhIaUXtNq2",
  "attempts": 4,
  "response_status": 422,
  "last_attempt_at": "2021-08-02T13:52:16.439Z",
  "next_attempt_at": null,
  "successful_at": "2021-08-01T21:56:35.181Z",
  "updated_at": "2021-08-02T14:12:56.321Z",
  "created_at": "2021-08-01T21:44:57.513Z",
  "status": "QUEUED",
  "event_data_id": "evtreq_HlJhb39nsjIP0FDsU3SY0fo7",
  "cli_id": null,
  "data": {
    "headers": {
      "content-length": "20",
      "x-original-url": "/e/src_9YmY0SHklrv6zBQz78cKGwSW",
      "cookie": "",
      "x-client-version": "FirebaseCLI/undefined",
      "content-type": "application/json",
      "user-agent": "FirebaseCLI/undefined"
    },
    "body": {
      "test": true
    },
    "query": "",
    "parsed_query": {},
    "path": "/"
  }
}

Hookdeck will now attempt to deliver the event to its destination. If it succeeds, any scheduled automatic retries for that event will be canceled.

Retry many events

Bulk retrying events may impact your delivery latency, as retries are queued alongside new events.

{
  "query": {
    "source_id": [
      "src_oDhfDrVAZV6OIPdBzcD97of2"
    ]
  }
}

{
  "id": "blkr_6dxoWIomd8DQdH",
  "team_id": "tm_UkFg3CHYN50M",
  "query": {
    "rejection_cause": [
      "SOURCE_DISABLED"
    ]
  },
  "updated_at": "2022-05-13T09:06:03.983Z",
  "created_at": "2022-05-13T09:05:55.394Z",
  "completed_at": null,
  "cancelled_at": "2022-05-13T09:06:03.982Z",
  "estimated_batch": 4,
  "processed_batch": 2,
  "estimated_count": 28,
  "completed_count": 8,
  "failed_count": 0,
  "number": 15,
  "in_progress": false,
  "progress": 0.29
}

Hookdeck will now attempt to deliver the events. For each event whose delivery succeeds, any scheduled automatic retries will be canceled.

Automatically retry events

With automatic retries, Hookdeck will continue to attempt delivery for failed events.

Each automatic retry is counted as a new delivery attempt. Once an attempt succeeds, further automatic retries on that event are canceled.

Events are limited to 50 automatic retries, but can still be manually retried after that limit is hit.

Configuring automatic retries

Configure a connection's automatic retries by setting its retry rule. There you can define the number of automatic retries Hookdeck will attempt after an event fails, the time interval between each attempt, the retry strategy and the response HTTP status codes to apply the retry rule to.

The retry strategy can be either linear or exponential. With a linear strategy, retries occur at regular intervals; with an exponential strategy, each retry is delayed twice as long as the previous (1 hour, 2 hours, 4 hours, etc.).

The response status codes can be used to specify which status codes should trigger a retry. For example, if you set the retry rule to apply to status codes 500-599, Hookdeck will retry events that receive any of those status codes. The status codes supports inclusion, exclusion, ranges and greater than/less than comparisons (>=, <=, >, <).

For example, to retry all 5xx errors, you can use 500-599 or >=500 and to retry all 5xx errors except 501 you can use 500-599, !501.

In the case of a conflicting HTTP status code, Hookdeck will take the last matching statement. For example, 500, !500 will not retry 500s.

Identifying retries

When a HTTP request is made to your destination, Hookdeck will include a x-hookdeck-will-retry-after header in the request headers. This header indicates the number of seconds that the next retry will be scheduled in once your server responds, assuming that a retry is applicable. If there's no x-hookdeck-will-retry-after then the request is the last automatic retry.

Scheduling a automatic retry

Your destination can respond with a Retry-After header to indicate how long Hookdeck should wait before retrying the event. This can be useful to implement custom retry logic or scheduling of retries based on specific errors. The Retry-After header must be an integer value representing the number of seconds to wait before retrying the request, a valid date string as per the HTTP spec or a ISO string.

Returning a value of -1 will cancel any further automatic retries.

If your destination returns a Retry-After header, the value will take precedence over any retry rule. You don't need to set a retry rule in order to use the Retry-After header.

Using the Retry-After header, a retry can be scheduled up to 7 days in the future and at most 50 times.

Viewing the next attempt

To determine when a specific event will be retried, locate the event and note its Next Attempt At property.

Cancel scheduled retries

Canceling an automatic retry will prevent all future scheduled retries from occurring on an event.

{
  "id": "evt_njCeMZHzNhJWdkkZ7PLCfEw5",
  "team_id": "tm_nlcetVe8k1lMAY0KR0OxNuHr",
  "webhook_id": "web_UfM5ZY2wxLD6sLhDaKMcx5wp",
  "source_id": "src_9YmY0SHklrv6zBQz78cKGwSW",
  "destination_id": "des_fYYCay655IHwbOMhIaUXtNq2",
  "attempts": 5,
  "response_status": 422,
  "last_attempt_at": "2021-08-02T13:52:16.439Z",
  "next_attempt_at": null,
  "successful_at": "2021-08-01T21:56:35.181Z",
  "updated_at": "2021-08-02T14:12:56.466Z",
  "created_at": "2021-08-01T21:44:57.513Z",
  "status": "QUEUED",
  "event_data_id": "evtreq_HlJhb39nsjIP0FDsU3SY0fo7",
  "cli_id": null,
  "data": {
    "headers": {
      "content-length": "20",
      "x-original-url": "/e/src_9YmY0SHklrv6zBQz78cKGwSW",
      "cookie": "",
      "x-client-version": "FirebaseCLI/undefined",
      "content-type": "application/json",
      "user-agent": "FirebaseCLI/undefined"
    },
    "body": {
      "test": true
    },
    "query": "",
    "parsed_query": {},
    "path": "/"
  }
}