# Guide to Scrapfly Webhooks: Features and Best Practices

Scrapfly webhooks notify your application when a scrape (or extraction or screenshot) job completes, delivering the full job result to your endpoint. If you're building on Scrapfly, webhooks are how you receive results asynchronously instead of holding a request open.

This guide covers how Scrapfly webhooks work, the resource types you'll handle, how to verify the `X-Scrapfly-Webhook-Signature` (note the uppercase hex), the two-step registration, and the best practices for production.

## What are Scrapfly webhooks?

Scrapfly delivers one webhook per job on completion. Each is signed with an `X-Scrapfly-Webhook-Signature` header: an HMAC-SHA256 over the raw request bytes, hex-encoded in UPPERCASE. A lowercase duplicate is available in `X-Scrapfly-Webhook-Signature-Lowercase`. There's no timestamp header and no tolerance window, so the signature is authenticity-only. There's also no event-name field: the discriminator is the `X-Scrapfly-Webhook-Resource-Type` header.

## Scrapfly webhook features

| Feature | Details |
| --- | --- |
| Configuration | Two-step: name a webhook in the dashboard, then pass `webhook_name=<name>` per job |
| Signature header | `X-Scrapfly-Webhook-Signature` (UPPERCASE hex), plus `-Lowercase` duplicate |
| Signature scheme | HMAC-SHA256 over the raw bytes, hex, uppercased |
| Discriminator | `X-Scrapfly-Webhook-Resource-Type` header (`scrape`, plus `extraction` / `screenshot` on paid plans) |
| Replay window | None (no timestamp) |
| Retries | 30s, 1min, 5min, 30min, 1hr, 1day; auto-disabled after 100 consecutive failures |
| SDK | `scrapfly-sdk` (npm/pip); verify manually |

## Common events

Scrapfly has no event catalog. Instead, the `X-Scrapfly-Webhook-Resource-Type` header tells you what completed:

| `X-Scrapfly-Webhook-Resource-Type` | Fires when |
| --- | --- |
| `scrape` | A scrape job completes |
| `extraction` | An extraction job completes (paid plans) |
| `screenshot` | A screenshot job completes (paid plans) |

The body is the entire job response inline (`result`, `config`, `context`, `status`, `success`), plus `context.webhook{name, secret, consecutive_failed_count}` and `context.job.uuid`. Other headers include `X-Scrapfly-Webhook-Env`, `-Project`, `-Id`, `-Name`, and `-Job-Id`.

## Setting up Scrapfly webhooks

Registration is two-step, and easy to get wrong. First, create and name a webhook in the dashboard. Then, every individual scrape request must pass `webhook_name=<that name>`. There's no global "send all scrapes here" default, omitting `webhook_name` means no webhook fires at all. Store the webhook's signing secret as `SCRAPFLY_WEBHOOK_SECRET`.

## Securing Scrapfly webhooks

Compute an HMAC-SHA256 over the raw bytes, hex-encode it, and uppercase it to match the primary header (a lowercase digest won't match `X-Scrapfly-Webhook-Signature`). Compare in constant time. Don't parse and re-serialize the JSON, that changes the byte sequence and breaks the signature.

```javascript
const crypto = require("crypto");

const SECRET = process.env.SCRAPFLY_WEBHOOK_SECRET;

function verify(rawBody, signatureHeader) {
  if (!signatureHeader) return false;
  const expected = crypto.createHmac("sha256", SECRET).update(rawBody).digest("hex").toUpperCase();
  const received = signatureHeader.toUpperCase();
  try {
    return crypto.timingSafeEqual(Buffer.from(received, "hex"), Buffer.from(expected, "hex"));
  } catch {
    return false;
  }
}

app.post("/webhooks/scrapfly", express.raw({ type: "*/*" }), (req, res) => {
  if (!verify(req.body, req.headers["x-scrapfly-webhook-signature"])) {
    return res.sendStatus(401);
  }
  res.sendStatus(200); // acknowledge fast
  const resourceType = req.headers["x-scrapfly-webhook-resource-type"]; // scrape / extraction / screenshot
  processQueue.add({ resourceType, body: JSON.parse(req.body.toString()) }); // async
});

```

The same check in Python (the documented computation):

```python
import hashlib
import hmac
import os

SECRET = os.environ["SCRAPFLY_WEBHOOK_SECRET"]

def verify(raw_body: bytes, signature_header: str) -> bool:
    expected = hmac.new(
        SECRET.encode("utf-8"), raw_body, hashlib.sha256
    ).hexdigest().upper()
    return hmac.compare_digest(expected, (signature_header or "").upper())

```

## Scrapfly webhook limitations and pain points

### The primary signature is uppercase hex

The Problem: The primary `X-Scrapfly-Webhook-Signature` is uppercase hex. A constant-time compare against a lowercase digest fails against it.

Why It Happens: Scrapfly uppercases the hex digest (`.hexdigest().upper()`).

Workarounds:

* Uppercase your computed digest (or normalize both sides), or verify against the `-Lowercase` companion header.

How Hookdeck Can Help: Hookdeck verifies the signature at the edge, so casing doesn't trip your app.

### There's no event field

The Problem: Scrapfly has no event-name field in the body. Code that switches on a `type` field has nothing to switch on.

Why It Happens: The job type is carried in the `X-Scrapfly-Webhook-Resource-Type` header, not the body.

Workarounds:

* Dispatch on the `X-Scrapfly-Webhook-Resource-Type` header (`scrape` / `extraction` / `screenshot`).

How Hookdeck Can Help: Hookdeck's filters can route on the resource-type header, giving you type-based routing.

### Two-step registration is easy to miss

The Problem: Naming a webhook in the dashboard isn't enough. If a scrape request doesn't pass `webhook_name`, no webhook fires, silently.

Why It Happens: Scrapfly has no global default destination; delivery is opt-in per request.

Workarounds:

* Pass `webhook_name=<name>` on every job you want delivered.

How Hookdeck Can Help: Hookdeck gives you a stable endpoint and full delivery visibility, so you can see immediately whether jobs are arriving.

### Endpoints auto-disable after 100 failures

The Problem: After 100 consecutive failures Scrapfly auto-disables the webhook, and it must be re-enabled by hand in the dashboard. Deliveries stop until you notice.

Why It Happens: Scrapfly disables persistently failing endpoints.

Workarounds:

* Acknowledge reliably, and monitor for the disabled state.

How Hookdeck Can Help: Hookdeck absorbs delivery to your app behind a durable queue and retries, so a downstream blip doesn't march toward the 100-failure cutoff.

## Best practices

### Verify uppercase HMAC-SHA256 over the raw bytes

Compute the hex HMAC over the raw bytes, uppercase it, and compare against the primary header in constant time; never re-serialize the body first.

### Dispatch on the resource-type header

Route on `X-Scrapfly-Webhook-Resource-Type`, not a body field.

### Always pass webhook_name

Every scrape you want delivered must include `webhook_name=<name>`.

### Acknowledge fast, process asynchronously

Return 200 quickly and defer work to a queue; dedupe on `X-Scrapfly-Webhook-Id` / `context.job.uuid`. See [why to process webhooks asynchronously](/webhooks/guides/why-implement-asynchronous-processing-webhooks).

## Conclusion

Scrapfly delivers one webhook per job, verified with an `X-Scrapfly-Webhook-Signature` HMAC-SHA256 over the raw bytes, hex in UPPERCASE (with a lowercase duplicate available). There's no timestamp and no event field, dispatch on `X-Scrapfly-Webhook-Resource-Type`. Uppercase your digest to match, never re-serialize the body, pass `webhook_name` on every job, and watch for the 100-failure auto-disable.

[Hookdeck](https://hookdeck.com) verifies the signature, deduplicates, and durably queues every job result at the edge, so your app only ever processes verified, unique events.

[Get started with Hookdeck](https://dashboard.hookdeck.com/signup) for free and handle Scrapfly webhooks reliably in minutes.