# Guide to Firecrawl Webhooks: Features and Best Practices

Firecrawl webhooks tell your application how a scraping job is going: a crawl starts, each page is scraped, the batch finishes, an extraction fails, an agent takes an action, or a monitor check finds changed pages. If you use Firecrawl, the web scraping and crawling API, webhooks let you process results as they arrive instead of polling the job status endpoint.

This guide covers how Firecrawl webhooks are attached to jobs and signed, the lifecycle events and their payloads, and the practices that keep a Firecrawl receiver reliable when a crawl sends hundreds of page events.

## What are Firecrawl webhooks?

Firecrawl has no standing webhook subscription. Instead, you add a `webhook` object to each job request: a crawl, batch scrape, extract, agent job or monitor. It names the URL, optional custom headers, optional metadata to echo back, and the events you want. Firecrawl then sends JSON `POST` requests to that URL as the job moves through its lifecycle.

Every request is signed with one secret per Firecrawl account. The `X-Firecrawl-Signature` header holds `sha256=` followed by a hex HMAC-SHA256 of the raw body.

## Firecrawl webhook features

| Feature | Details |
| --- | --- |
| Configuration | A `webhook` object on each job request (`url`, `headers`, `metadata`, `events`) |
| Supported jobs | Crawl, batch scrape, extract, agent, monitor |
| Event filter | Short names such as `started`, `page`, `completed`, `failed`. Default: all |
| Signature header | `X-Firecrawl-Signature: sha256=<hex>` |
| Scheme | HMAC-SHA256, hex |
| Signed content | The raw request body only |
| Secret | One per account, in the **Advanced** tab of account settings, regenerable there |
| Event type | `type` in the body, such as `crawl.page` |
| Response window | `2xx` within 10 seconds |
| Retries | 3: 1 minute, 5 minutes and 15 minutes after the failure |
| After the last retry | The webhook is marked as failed |

Firecrawl does not send a timestamp and publishes no source IPs.

## Common events

Firecrawl documents 16 event types.

| Job | Events |
| --- | --- |
| Crawl | `crawl.started`, `crawl.page`, `crawl.completed` |
| Batch scrape | `batch_scrape.started`, `batch_scrape.page`, `batch_scrape.completed` |
| Extract | `extract.started`, `extract.completed`, `extract.failed` |
| Agent | `agent.started`, `agent.action`, `agent.completed`, `agent.failed`, `agent.cancelled` |
| Monitor | `monitor.page`, `monitor.check.completed` |

The `events` filter on a request uses the short name, such as `page` or `completed`, scoped to the job type. The delivered `type` is always the full name, such as `crawl.page`.

Every event shares the same structure. Here is Firecrawl's `extract.failed` example:

```json
{
  "success": false,
  "type": "extract.failed",
  "id": "550e8400-e29b-41d4-a716-446655440000",
  "webhookId": "f1e2d3c4-0000-0000-0000-000000000010",
  "data": [],
  "error": "Failed to extract data: timeout exceeded",
  "metadata": {}
}

```

| Field | Meaning |
| --- | --- |
| `success` | Whether the operation succeeded |
| `type` | The event type |
| `id` | The **job** id, the same for every event in the job |
| `webhookId` | A unique id for this webhook delivery. Sent with crawl, batch scrape, extract and monitor events, not agent events |
| `data` | Event-specific data: scraped pages, agent actions, monitor results |
| `metadata` | The `metadata` you set on the request |
| `error` | The error message when `success` is `false` |

`crawl.page` and `batch_scrape.page` carry the scraped content for each page in `data`, so they are the largest and most frequent events. `agent.action` reports each tool the agent runs, with `creditsUsed`. `monitor.page` arrives as each monitored page finishes, before the whole check is reconciled, and `monitor.check.completed` then summarises the check with counts of same, changed, new, removed and errored pages.

## Setting up Firecrawl webhooks

1. Open the **Advanced** tab of your Firecrawl account settings and copy the webhook secret. Store it server-side, for example as `FIRECRAWL_WEBHOOK_SECRET`.
2. Add a `webhook` object to the job request:

```bash
curl -X POST https://api.firecrawl.dev/v2/crawl \
  -H 'Content-Type: application/json' \
  -H "Authorization: Bearer $FIRECRAWL_API_KEY" \
  -d '{
    "url": "https://docs.firecrawl.dev",
    "limit": 100,
    "webhook": {
      "url": "https://example.com/webhooks/firecrawl",
      "metadata": { "crawl_purpose": "docs-index" },
      "events": ["started", "page", "completed"]
    }
  }'

```

The same object works on `/v2/batch/scrape`, extract, agent and monitor requests. Use `metadata` to carry your own ids, such as a tenant or task id, so the receiver can tell which request an event belongs to without a lookup.

## Securing Firecrawl webhooks

Split `X-Firecrawl-Signature` on the first `=`, check that the algorithm is `sha256`, compute an HMAC-SHA256 of the raw body with your account secret, hex-encode it, and compare in constant time.

```javascript
const crypto = require('crypto');
const express = require('express');

function verifyFirecrawlSignature(rawBody, signatureHeader, secret) {
  if (!signatureHeader || !secret) return false;
  const i = signatureHeader.indexOf('=');
  if (i < 0 || signatureHeader.slice(0, i) !== 'sha256') return false;

  const expected = Buffer.from(
    crypto.createHmac('sha256', secret).update(rawBody).digest('hex'),
    'utf8'
  );
  const received = Buffer.from(signatureHeader.slice(i + 1).trim().toLowerCase(), 'utf8');
  return received.length === expected.length && crypto.timingSafeEqual(received, expected);
}

const app = express();

app.post('/webhooks/firecrawl', express.raw({ type: '*/*', limit: '10mb' }), (req, res) => {
  const secret = process.env.FIRECRAWL_WEBHOOK_SECRET;
  if (!secret) return res.sendStatus(500); // fail closed

  if (!verifyFirecrawlSignature(req.body, req.get('x-firecrawl-signature'), secret)) {
    return res.sendStatus(401);
  }

  const event = JSON.parse(req.body.toString('utf8'));
  // Route on event.type; store page data and process it asynchronously.
  res.sendStatus(200);
});

```

The same check in Python:

```python
import hashlib
import hmac

def verify_firecrawl_signature(raw_body: bytes, signature_header, secret: str) -> bool:
    if not signature_header or not secret:
        return False
    algorithm, _, received = signature_header.partition("=")
    if algorithm != "sha256" or not received:
        return False
    expected = hmac.new(secret.encode("utf-8"), raw_body, hashlib.sha256).hexdigest()
    return hmac.compare_digest(
        received.strip().lower().encode("utf-8"), expected.encode("utf-8")
    )

# In FastAPI: raw_body = await request.body()
# verify_firecrawl_signature(raw_body, request.headers.get("x-firecrawl-signature"),
#                            os.environ["FIRECRAWL_WEBHOOK_SECRET"])

```

Page events can be large. Raise your body parser's size limit, as with the `limit` option above, or large pages are rejected before your code runs.

> **Make Firecrawl webhooks production-ready.** [Hookdeck Event Gateway](/event-gateway) verifies the `X-Firecrawl-Signature` header, answers Firecrawl with an immediate `200 OK`, and durably queues every page and job event before your service sees it.

## Firecrawl webhook limitations and pain points

### Page events arrive in bursts

**The Problem:** A large crawl floods your endpoint, slow handlers time out, and pages are retried or lost.

**Why It Happens:** A crawl sends one `crawl.page` event per page, each with the scraped content, as each page is scraped. Firecrawl waits only 10 seconds per request.

**Workarounds:**

* Store each page and return `200` immediately, then process from a queue.
* Subscribe only to `started` and `completed` if you plan to fetch results from the API at the end.

**How Hookdeck Can Help:** Hookdeck absorbs the burst, acknowledges each request immediately, and delivers to your service at a rate you set, so a 1,000-page crawl doesn't overwhelm a small worker.

### Three retries in about 20 minutes

**The Problem:** A brief outage loses events for good.

**Why It Happens:** Firecrawl retries a failed delivery three times, at 1, 5 and 15 minutes after each failure, and then marks it failed.

**Workarounds:**

* Keep the receiver highly available.
* After an incident, fetch job results from the Firecrawl API rather than relying on the missing events.

**How Hookdeck Can Help:** Hookdeck stores every event it receives and retries delivery to your service for as long as you configure, with replay for anything that still fails.

### One secret for the whole account

**The Problem:** Every environment and service shares one signing secret, and regenerating it affects all of them at once.

**Why It Happens:** Firecrawl signs every webhook with a single account-level secret, and documents no overlap period when you regenerate it.

**Workarounds:**

* Keep the secret in one shared secret store, so a regeneration is one update.
* Use `metadata` to separate environments and tenants, since the secret can't.

**How Hookdeck Can Help:** With Hookdeck in front, the secret lives on the source. Update it once and every service behind it keeps receiving verified events.

### `id` is not a delivery id

**The Problem:** Deduplicating on `id` drops real events.

**Why It Happens:** `id` is the job id, shared by every event in the job. `webhookId` is described as a unique id for the delivery, but it is not sent with agent events, and Firecrawl does not say whether a retry keeps the same value.

**Workarounds:**

* Deduplicate on `webhookId` where it exists, and check how it behaves on a retried delivery before you rely on it alone.
* Where it is missing, build a key from `id`, `type` and identifying fields in `data`, such as the page URL.

**How Hookdeck Can Help:** Hookdeck's deduplication can use whichever body fields you choose, such as `webhookId`.

## Best practices

### Verify against the raw body

Use `express.raw()` in Express, `await request.text()` in a Next.js route handler and `await request.body()` in FastAPI. Firecrawl's own Express sample does the same.

### Use `metadata` to correlate

Firecrawl echoes the request's `metadata` in every event. Put your own task id there, and the receiver never has to map Firecrawl job ids back to your records.

### Check `success` before reading `data`

Failure events set `success` to `false` and add `error`. A partially failed monitor check also reports `success: false`. Check it before assuming `data` holds results.

### Treat `completed` as the end of a job

Use `crawl.completed` or `batch_scrape.completed` to finalise a job, rather than guessing from the page events you have received. A page whose delivery is being retried can still arrive after the job is complete.

### Acknowledge fast and process asynchronously

Firecrawl allows 10 seconds. Verify, store, return `200`, and process afterwards. See [why to process webhooks asynchronously](/webhooks/guides/why-implement-asynchronous-processing-webhooks) and our [guide to webhook idempotency](/webhooks/guides/implement-webhook-idempotency).

## Conclusion

Firecrawl signs every webhook with `X-Firecrawl-Signature: sha256=<hex>`, an HMAC-SHA256 of the raw body under one account-wide secret. Verify against the raw bytes, raise body size limits for page events, deduplicate on `webhookId` rather than the job `id`, use `metadata` to correlate jobs, and acknowledge within 10 seconds, because three retries leave little room for downtime.

[Hookdeck Event Gateway](https://hookdeck.com) sits in front of your receiver, checks every Firecrawl signature, smooths out crawl bursts, and holds each event until your service has processed it.

[Get started with Hookdeck](https://dashboard.hookdeck.com/signup) for free and handle Firecrawl webhooks reliably in minutes.