Gareth Wilson Gareth Wilson

Guide to Apify Webhooks: Features and Best Practices

Published


Apify webhooks tell your systems when something happens to an Actor run or build on Apify, the web scraping and automation platform. A scraper finishes and you want to pull its dataset, a run fails and someone needs to know, a build succeeds and a deployment should move on. Webhooks let your services react to those changes as they happen instead of polling the Apify API for run status.

This guide covers how Apify webhooks are configured, the run and build events they report and the shape of the default payload, how to authenticate requests that Apify does not sign, the retry schedule, and the best practices for running an Apify receiver in production.

What are Apify webhooks?

An Apify webhook pairs one or more system events with an action. The only action Apify offers is to send an HTTP POST with a JSON body to a URL you choose. You define webhooks on the Integrations tab of an Actor or Actor task in Apify Console, or attach single-use ad-hoc webhooks to a specific run when you start it through the API.

The body comes from a payload template. The default template produces an object with userId, createdAt, eventType, eventData and resource, but you can rewrite the template to send any JSON you like, built from the same variables. A second template, the Headers template, adds your own HTTP headers to the request.

Apify does not sign webhook requests. Its security advice is to put a secret token in the webhook URL, or to send one in a header with the Headers template. Both are static secrets: nothing is computed over the body. A header is the better choice, because it keeps the secret out of URLs and access logs, and it is what Hookdeck's Apify source checks.

Apify webhook features

FeatureDetails
ConfigurationIntegrations tab of an Actor or task in Apify Console, or ad-hoc webhooks passed when starting a run through the API
ActionHTTP POST with a JSON body. No other action is available
PayloadBuilt from a payload template. The default is {userId, createdAt, eventType, eventData, resource}
Custom headersHeaders template: a JSON object whose values must be strings
System headersHost, Content-Type, X-Apify-Webhook, X-Apify-Webhook-Dispatch-Id and X-Apify-Request-Origin, always set and not overridable
AuthenticationNone built in. A secret in the URL or in a custom header
SignatureNone
Timeout2 minutes
RetriesNon-2xx responses retried with exponential back-off: about 1 minute, 2 minutes, 4 minutes, up to about 32 hours on the eleventh retry
DuplicatesPossible "in rare cases"
Source IPsTen published static IP addresses
TestingA Test button in the webhook settings

Common events

Apify documents 11 event types, six for Actor runs and five for Actor builds:

EventFires when
ACTOR.RUN.CREATEDA new Actor run has been created
ACTOR.RUN.SUCCEEDEDA run finished with status SUCCEEDED
ACTOR.RUN.FAILEDA run finished with status FAILED
ACTOR.RUN.ABORTEDA run finished with status ABORTED
ACTOR.RUN.TIMED_OUTA run finished with status TIMED-OUT
ACTOR.RUN.RESURRECTEDA run has been resurrected
ACTOR.BUILD.CREATEDA new Actor build has been created
ACTOR.BUILD.SUCCEEDEDA build finished with status SUCCEEDED
ACTOR.BUILD.FAILEDA build finished with status FAILED
ACTOR.BUILD.ABORTEDA build finished with status ABORTED
ACTOR.BUILD.TIMED_OUTA build finished with status TIMED-OUT

Note the mismatch in the timeout events: the event name uses an underscore (TIMED_OUT) while the run or build status uses a hyphen (TIMED-OUT). A webhook defined on a task fires only for runs started for that task.

Here is Apify's default payload for a successful run, with the resource object shortened:

{
  "userId": "abf6vtB2nvQZ4nJzo",
  "createdAt": "2019-01-09T15:59:56.408Z",
  "eventType": "ACTOR.RUN.SUCCEEDED",
  "eventData": {
    "actorId": "fW4MyDhgwtMLrB987",
    "actorRunId": "uPBN9qaKd2iLs5naZ"
  },
  "resource": {
    "id": "uPBN9qaKd2iLs5naZ",
    "actId": "fW4MyDhgwtMLrB987",
    "userId": "abf6vtB2nvQZ4nJzo",
    "startedAt": "2019-01-09T15:59:40.750Z",
    "finishedAt": "2019-01-09T15:59:56.408Z",
    "status": "SUCCEEDED"
  }
}

For run events, eventData carries actorId, actorRunId and, when a task started the run, actorTaskId. For build events it carries actorId and actorBuildId. resource is the object the Get Actor run or Get Actor build endpoint would return at the moment the webhook fired. Apify notes that eventData and resource hold redundant data for backward compatibility.

The run event is a notification, not the results. To fetch what a scraper produced, take actorRunId and call https://api.apify.com/v2/actor-runs/[ACTOR_RUN_ID]/dataset/items with your API token.

Inspect Apify webhooks as they arrive. Point Apify at a Hookdeck Inspect URL to inspect and replay real deliveries, with no account or setup required.

Setting up Apify webhooks

Generate a long random secret first (openssl rand -hex 32 works) and store it in your app's environment, for example as APIFY_WEBHOOK_SECRET.

  1. In Apify Console, open the Actor or task and go to the Integrations tab.
  2. Add a webhook and select the event types to send, for example ACTOR.RUN.SUCCEEDED and ACTOR.RUN.FAILED.
  3. Set the request URL to your HTTPS endpoint. Don't put the secret in the URL.
  4. In the Headers template, add your secret header:
{"X-Webhook-Secret": "your-long-random-secret"}
  1. Save, then use the Test button to send a request to your endpoint.

The header name is up to you, but avoid the system headers listed above, because Apify rewrites them on every request. If you type a prefix into the value, such as {"Authorization": "Bearer abc123"}, your receiver has to expect the whole string Bearer abc123.

If you leave the payload template at its default, your handler can rely on eventType, eventData and resource. If you change it, your handler has to match the shape you built, and eventType is only present if you include {{eventType}}.

For local development, use the Hookdeck CLI: hookdeck listen 3000 apify --path /webhooks/apify gives you a public HTTPS URL that forwards to your local server, plus a web UI for inspecting and replaying deliveries, with no account required. Use the generated URL as the webhook's request URL.

With Hookdeck. Create an Apify source in Hookdeck and enter the Header Name (for example X-Webhook-Secret) and the Header Value exactly as they appear in your Headers template, including any Bearer prefix. Then use the source URL as the webhook's request URL. Apify's first suggestion, a token in the URL, is not something the source can check, so use the Headers template.

Securing Apify webhooks

With no signature to verify, authenticating an Apify request means comparing your secret header against the value you stored. Look the header up case-insensitively, compare the whole value with a timing-safe check, and check lengths first because crypto.timingSafeEqual throws on buffers of different lengths. Don't treat X-Apify-Webhook or X-Apify-Webhook-Dispatch-Id as proof of origin: they are system values, not secrets.

const crypto = require('crypto');
const express = require('express');

const app = express();

function safeEqual(a, b) {
  const x = Buffer.from(String(a));
  const y = Buffer.from(String(b));
  return x.length === y.length && crypto.timingSafeEqual(x, y);
}

function verifyApifyRequest(headers) {
  const secret = process.env.APIFY_WEBHOOK_SECRET;
  if (!secret) return false; // fail closed when unconfigured
  const name = (process.env.APIFY_WEBHOOK_HEADER || 'X-Webhook-Secret').toLowerCase();
  const received = headers[name]; // Node lowercases header names
  return typeof received === 'string' && safeEqual(received, secret);
}

app.post('/webhooks/apify', express.raw({ type: '*/*' }), (req, res) => {
  if (!verifyApifyRequest(req.headers)) {
    return res.sendStatus(401);
  }

  let event;
  try {
    event = JSON.parse(req.body.toString('utf8'));
  } catch {
    return res.sendStatus(400);
  }

  // Acknowledge quickly; Apify times out after 2 minutes
  res.sendStatus(200);

  switch (event.eventType) {
    case 'ACTOR.RUN.SUCCEEDED':
      // Queue a job to fetch /actor-runs/{actorRunId}/dataset/items
      break;
    case 'ACTOR.RUN.FAILED':
    case 'ACTOR.RUN.TIMED_OUT':
      // Alert, or start the run again
      break;
  }
});

The same check in Python:

import hmac
import os


def verify_apify_request(headers) -> bool:
    secret = os.environ.get("APIFY_WEBHOOK_SECRET")
    if not secret:
        return False
    name = os.environ.get("APIFY_WEBHOOK_HEADER", "X-Webhook-Secret")
    received = headers.get(name)  # Starlette and Flask headers are case-insensitive
    if not received:
        return False
    return hmac.compare_digest(received.encode("utf-8"), secret.encode("utf-8"))

A static header proves the sender knows the secret. It does not prove the body is unchanged, and it gives no replay protection, because there is no timestamp and nothing is signed. Serve the endpoint over HTTPS only, keep the secret out of source control and logs, and rotate it if it may have leaked. Apify also publishes ten static egress IP addresses that you can allow at a firewall as a second layer, though the header check remains the authentication.

Make Apify webhooks production-ready. Hookdeck Event Gateway checks your secret header on every request, durably queues each run and build event, and lets you replay anything your service failed to process.

Apify webhook limitations and pain points

No signature, and the default advice is a URL token

The Problem: Apify requests carry no signature. The documented protection is a secret in the URL, which ends up in proxy logs, browser history and anywhere the URL is copied.

Why It Happens: Apify's webhook action is a plain HTTP request built from templates. Authentication is whatever you add to the URL or the headers yourself.

Workarounds:

  • Use the Headers template for the secret and keep the URL free of credentials.
  • Generate a long random value and rotate it if it may have been exposed.
  • Allow Apify's published IP addresses at your firewall as an extra layer.

How Hookdeck Can Help: Hookdeck's Apify source compares the header you name against the exact value you enter. A request without a matching header still gets a 200 response, but Hookdeck records it as failed verification and creates no event, so it never reaches your destination. Every request is logged with its headers and body, which makes a mistyped template easy to spot.

A two-minute timeout on slow handlers

The Problem: A handler that downloads the run's dataset before answering can run past Apify's timeout, and the delivery is then treated as failed and retried.

Why It Happens: Apify waits 2 minutes for a response. Apify's own advice is to respond immediately and use a queue for time-consuming work.

Workarounds:

  • Return a 2xx as soon as the header checks out, then fetch results in a background job.
  • Treat retries as normal and make the handler idempotent.

How Hookdeck Can Help: Hookdeck acknowledges Apify straight away and stores the event, then delivers it to your service with automatic retries on a schedule you control, so slow processing on your side no longer counts as a failed webhook.

Retries stretch over more than a day, then stop

The Problem: After a failure, Apify retries with exponential back-off, from about a minute up to about 32 hours. After 11 retries it gives up, and the event is gone unless you go back to the API.

Why It Happens: Apify's retry schedule is fixed. Long gaps between later retries also mean a fix on your side can take hours to be picked up.

Workarounds:

  • Alert on failed deliveries rather than waiting for the retries to run out.
  • Reconcile by listing recent runs through the Apify API after an outage.

How Hookdeck Can Help: Hookdeck accepts each delivery as it arrives, so Apify's retry budget is not spent on your downtime. Issues alert you when deliveries to your service fail, and you can replay any stored event once the fix is out.

Duplicate deliveries

The Problem: The same webhook can arrive more than once.

Why It Happens: Apify says that "in rare cases, the webhook might be invoked more than once", and retries after a timeout can repeat a request your service already processed.

Workarounds:

  • Deduplicate on eventType plus eventData.actorRunId or eventData.actorBuildId. Add resource.finishedAt to the key if you subscribe to ACTOR.RUN.RESURRECTED, because a resurrected run can finish a second time.
  • Log X-Apify-Webhook-Dispatch-Id with each request for tracing.

How Hookdeck Can Help: Hookdeck deduplication can drop repeated deliveries based on fields you choose, such as eventType and resource.id, before they reach your handler.

Custom payload templates change the contract

The Problem: A handler written for the default payload breaks when someone edits the template, and eventType may be missing altogether.

Why It Happens: The payload template is fully editable. Apify only guarantees the variables, not the shape you build from them.

Workarounds:

  • Keep the default template, or keep {{eventType}} and {{resource}} in any custom one.
  • Validate the body and return 400 for shapes you don't expect, then log them.

How Hookdeck Can Help: Transformations can normalise different template shapes into one structure before delivery, and filters can route events by eventType to different destinations.

Best practices

Authenticate with a header, never the URL

Put the secret in the Headers template, compare the whole header value with a timing-safe check, and fail closed if the secret isn't configured. Use a different secret for each Actor or environment so you can rotate one without touching the others.

Acknowledge fast and fetch results later

The webhook says a run finished. Return 200 once the header is checked, then fetch the dataset, key-value store or logs in a background job. See why to process webhooks asynchronously.

Make handlers idempotent

Record the run or build ID with the event type (and finishedAt for resurrected runs) before acting, and skip repeats. See our guide to webhook idempotency.

Handle every terminal state

A run can end as SUCCEEDED, FAILED, ABORTED or TIMED-OUT. Subscribing only to success hides the runs that need attention. Subscribe to the failure events too and route them to alerting.

Prefer the API for results over the payload

resource is a snapshot from when the event fired. Use the IDs it carries to fetch current data from the Apify API before doing anything with lasting effect.

Conclusion

Apify sends a JSON POST for Actor run and build events, built from a template you can edit, with no signature on any request. Authenticate with a secret header from the Headers template rather than a URL token, compare it in constant time over HTTPS, answer within the 2 minute timeout, and deduplicate on the run or build ID because retries and rare duplicates both happen.

Hookdeck Event Gateway checks the secret header, queues every event durably and lets you replay anything that failed, so a slow or offline receiver doesn't use up Apify's retries.

Get started with Hookdeck for free and handle Apify webhooks reliably in minutes.


Gareth Wilson

Gareth Wilson

Product Marketing

Multi-time founding marketer, Gareth is PMM at Hookdeck and author of the newsletter, Community Inc.