# Openlayer Webhooks

[Openlayer](https://openlayer.com) is an AI evaluation and observability platform. Workspace webhooks POST a signed JSON event to your HTTPS endpoint when a test is created, updated or deleted, and when a test suite finishes a run. Signatures follow Standard Webhooks, so each delivery carries `webhook-id`, `webhook-timestamp` and `webhook-signature` headers.

## When to Use This Skill

* How do I receive Openlayer webhooks?
* How do I verify an Openlayer webhook signature (`webhook-signature`)?
* Where do I find my Openlayer webhook signing secret (`whsec_...`)?
* How do I handle `test.created`, `test.updated`, `test.deleted` or `tests.result.updated` events?
* How do I alert when an Openlayer test suite run has failing tests?
* Why is my Openlayer webhook verification failing, or failing only on retries and replays?

## Verification (core)

| Header | Value |
| --- | --- |
| `webhook-id` | Unique per message; retries reuse it. Your idempotency key |
| `webhook-timestamp` | Unix seconds |
| `webhook-signature` | One or more space-separated `v1,<base64>` entries |

Signed content is `{webhook-id}.{webhook-timestamp}.{raw body}`, HMAC-SHA256, base64 output. The key is the base64-**decoded** part of the `whsec_...` secret after the prefix. Compare against every `v1,` entry; ignore other versions. If `webhook-id` is absent, read `svix-id`, `svix-timestamp` and `svix-signature` instead, as Hookdeck does. The examples use the `standardwebhooks` library (npm and PyPI):

```javascript
const { Webhook } = require('standardwebhooks');

// rawBody: exact request body (Buffer or string), NOT re-serialised JSON
function verifyOpenlayer(rawBody, headers, secret = process.env.OPENLAYER_WEBHOOK_SECRET) {
  const p = headers['webhook-id'] ? 'webhook' : 'svix'; // fall back to the svix-* names
  new Webhook(secret).verify(rawBody, {
    'webhook-id': headers[`${p}-id`],
    'webhook-timestamp': headers[`${p}-timestamp`],
    'webhook-signature': headers[`${p}-signature`],
  }, { jsonParse: false }); // throws on a bad signature or stale timestamp → respond 401
}

```

The library always enforces a fixed **5-minute timestamp window**, which is the tolerance Openlayer suggests (for example 5 minutes). Hookdeck retries and replays resend the original `webhook-timestamp`, so the library rejects them once they are more than 5 minutes old. If you receive through Hookdeck and need late retries to pass, use the manual verifier in [references/verification.md](https://github.com/hookdeck/webhook-skills/blob/main/skills/openlayer-webhooks/references/verification.md), which can drop or widen the window. In Express, read the body with `express.raw({ type: '*/*' })` so any content type verifies, and parse JSON only after the check.

> **For complete handlers with tests**, see [examples/express/](https://github.com/hookdeck/webhook-skills/tree/main/skills/openlayer-webhooks/examples/express/), [examples/nextjs/](https://github.com/hookdeck/webhook-skills/tree/main/skills/openlayer-webhooks/examples/nextjs/), [examples/fastapi/](https://github.com/hookdeck/webhook-skills/tree/main/skills/openlayer-webhooks/examples/fastapi/).

## Event Type and Payload

Every body is `{ type, timestamp, data }`. The event name is the top-level `type`, `timestamp` is the ISO 8601 (UTC) time the event was generated, and the event fields sit under `data` (`data.test` for `test.*`, `data.tests` for `tests.result.updated`):

```json
{
  "type": "tests.result.updated",
  "timestamp": "2026-01-21T10:30:00Z",
  "data": {
    "tests": {
      "projectId": "3fa85f64-5717-4562-b3fc-2c963f66afa6",
      "projectVersionId": "3fa85f64-5717-4562-b3fc-2c963f66afa6",
      "inferencePipelineId": null,
      "total": 6, "passing": 4, "failing": 1, "skipped": 1, "running": 0
    }
  }
}

```

`data.test.type` (for example `integrity`) is the test's own category, not the event type.

## Common Event Types

| Event | Sent when |
| --- | --- |
| `test.created` | A test is created (`data.test` holds the full definition) |
| `test.updated` | A test's configuration is updated (`data.test` holds the new state) |
| `test.deleted` | A test is deleted (`data.test` has only `id`) |
| `tests.result.updated` | A test suite finishes a run: one event per run with `total`, `passing`, `failing`, `skipped`, `running` |

These 4 are the documented events; the list may not be exhaustive, so ignore types you don't recognise. See [references/overview.md](https://github.com/hookdeck/webhook-skills/blob/main/skills/openlayer-webhooks/references/overview.md).

## Delivery Semantics

Return any `2xx`; the connection timeout is 5 seconds and the response timeout 10 seconds. HTTPS only. Another status, a timeout or a redirect (not followed, so a `3xx` is a failure) is a failed attempt. Openlayer makes the initial attempt plus up to 3 retries after about 5 seconds, 5 minutes and 30 minutes (with up to 10% jitter), then marks the event `failed`. Events and delivery records are kept for 90 days. Events can repeat: deduplicate on `webhook-id`. Return `400` for missing headers and `401` for a bad signature.

## Environment Variables

```bash
OPENLAYER_WEBHOOK_SECRET=whsec_xxxxx   # full value shown once when the subscription is created; handlers fail closed (500) without it

```

## Local Development

```bash
# Express / Next.js (use 8000 for FastAPI). No install or account required.
npx hookdeck-cli listen 3000 openlayer --path /webhooks/openlayer

```

Register the printed URL as an HTTPS webhook subscription (workspace settings > Webhooks), then trigger one of the subscribed events, for example by creating a test.

## Reference Materials

* [references/overview.md](https://github.com/hookdeck/webhook-skills/blob/main/skills/openlayer-webhooks/references/overview.md) - Event catalog, envelope and payload fields
* [references/setup.md](https://github.com/hookdeck/webhook-skills/blob/main/skills/openlayer-webhooks/references/setup.md) - Create a subscription in the dashboard or API, copy the secret
* [references/verification.md](https://github.com/hookdeck/webhook-skills/blob/main/skills/openlayer-webhooks/references/verification.md) - Library and manual verification, gotchas, debugging