# Guide to Buildkite Webhooks: Features and Best Practices

Buildkite Pipelines webhooks report what your CI is doing as it happens: builds scheduled, failing and finished, jobs starting on agents, agents connecting or getting lost. Teams use them for build lights, chat notifications and deployment dashboards without polling the REST API.

This guide covers how Buildkite webhook notification services are configured, the events and payloads you will receive, the two ways Buildkite authenticates requests and which one Hookdeck supports, and how to handle what Buildkite doesn't document about delivery.

## What are Buildkite webhooks?

In Buildkite, an outgoing webhook is a notification service at the organisation level. Each one has a URL, a token, a set of events, and filters for pipelines, branches and queues. Buildkite sends each matching event as a JSON `POST` with the event name in an `X-Buildkite-Event` header and in the body's `event` field.

Every webhook has a token, and you choose how Buildkite sends it:

* **Token mode** (the default) sends the token as-is in `X-Buildkite-Token`. Buildkite's docs note that it is passed in clear text.
* **Signature mode** sends `X-Buildkite-Signature: timestamp=<unix>,signature=<hex>`, an HMAC-SHA256 of `<timestamp>.<raw body>` keyed with the token. The token itself is never sent.

## Buildkite webhook features

| Feature | Details |
| --- | --- |
| Configuration | **Settings** > **Notification Services** > **Webhook** > **Add** |
| Event header | `X-Buildkite-Event` |
| Authentication | `X-Buildkite-Token` (plain token, default) or `X-Buildkite-Signature` (HMAC-SHA256, hex) |
| Filters | Pipelines (all, some, exclude, by team, by cluster), branch patterns, and queues for job events |
| TLS | **Verify TLS Certificates** can be cleared for self-signed certificates |
| Request log | Last 20 requests and responses, under **Load recent requests** on the webhook's settings page |
| Retries and timeout | Not documented |
| Delivery ID | Not documented |
| Source IPs | Not documented |

## Common events

| Area | Events | Fires when |
| --- | --- | --- |
| Builds | `build.scheduled`, `build.running`, `build.failing`, `build.finished`, `build.skipped` | A build is scheduled, starts, starts failing, finishes or is skipped |
| Jobs | `job.scheduled`, `job.started`, `job.finished`, `job.activated`, `job.promised_exit_status` | A command job waits for or starts on an agent, finishes, a block step is unblocked, or a running job declares that it will fail |
| Agents | `agent.connected`, `agent.lost`, `agent.disconnected`, `agent.stopping`, `agent.stopped`, `agent.blocked` | An agent's connection changes, or its IP is no longer allowed by the agent token |
| Cluster tokens | `cluster_token.registration_blocked` | An agent registration is blocked by the token's IP allowlist |
| Settings | `ping` | The webhook's notification settings change |

Build events carry `build`, `pipeline` and `sender`. Job events add `job`. Here is the shape from Buildkite's docs, which only show the outer structure:

```json
{
  "event": "build.finished",
  "build": {
    "...": "...",
    "blocked": true
  },
  "pipeline": {
    "...": "..."
  },
  "sender": {
    "id": "0adfbc27-5f72-4a91-bf61-5693da0dd9c5",
    "name": "Some Person"
  }
}

```

The `build`, `job` and `pipeline` objects use the same schemas as Buildkite's REST API, with one difference: the webhook `build` object doesn't include job data, so subscribe to job events if you need it. A few details are easy to miss:

* A blocked build shows up as `build.finished` with `blocked: true`.
* Job events for retried jobs include a `retry_source` object with the original `job_id`, `retry_type` (`manual` or `automatic`) and `retried_by`.
* `job.promised_exit_status` arrives while the job is still `running`, with a top-level `promised_exit_status_reason`. A `job.finished` still follows.
* When a build changes state quickly, the payload may show a later state than the one that triggered the event.

## Setting up Buildkite webhooks

1. In Buildkite, select **Settings** > **Notification Services**, then **Add** on **Webhook**.
2. Enter a **Description** and the **Webhook URL**.
3. Under **Token**, copy the token and choose whether Buildkite sends it as `X-Buildkite-Token` or as an `X-Buildkite-Signature`. Choose `X-Buildkite-Token` if the URL is a Hookdeck source.
4. Select the **Events**, then narrow them with **Queues**, **Pipelines** and **Branch filtering** if you need to.
5. Select **Add Webhook Notification**. Buildkite sends `ping` when the settings change, which makes a quick test.
6. Check **Load recent requests** at the bottom of the webhook's settings page to see what your endpoint returned.

## Securing Buildkite webhooks

In token mode, compare `X-Buildkite-Token` with your stored token in constant time:

```javascript
const crypto = require('crypto');
const express = require('express');

function verifyBuildkiteToken(tokenHeader, expectedToken) {
  if (!tokenHeader || !expectedToken) return false;
  const received = Buffer.from(tokenHeader, 'utf8');
  const expected = Buffer.from(expectedToken, 'utf8');
  return received.length === expected.length && crypto.timingSafeEqual(received, expected);
}

const app = express();

app.post('/webhooks/buildkite', express.raw({ type: '*/*' }), (req, res) => {
  const token = process.env.BUILDKITE_WEBHOOK_TOKEN;
  if (!token) return res.sendStatus(500); // fail closed

  if (!verifyBuildkiteToken(req.get('x-buildkite-token'), token)) return res.sendStatus(401);

  const event = JSON.parse(req.body.toString('utf8'));
  // Route on req.get('x-buildkite-event') or event.event, then process async.
  res.sendStatus(200);
});

```

The same check in Python:

```python
import hmac
import os

def verify_buildkite_token(token_header, expected_token: str) -> bool:
    if not token_header or not expected_token:
        return False
    return hmac.compare_digest(token_header.encode("utf-8"), expected_token.encode("utf-8"))

# In FastAPI:
# verify_buildkite_token(request.headers.get("x-buildkite-token"), os.environ["BUILDKITE_WEBHOOK_TOKEN"])

```

If your endpoint receives Buildkite directly and you choose signature mode, parse `timestamp` and `signature` from `X-Buildkite-Signature`, compute a hex HMAC-SHA256 of `<timestamp>.<raw body>` with the token as the key, and compare in constant time:

```javascript
function verifyBuildkiteSignature(rawBody, signatureHeader, token) {
  if (!signatureHeader || !token) return false;
  const parts = Object.fromEntries(
    signatureHeader.split(',').map((pair) => pair.split('=', 2).map((value) => value.trim())),
  );
  if (!parts.timestamp || !parts.signature) return false;

  const signed_content = Buffer.concat([Buffer.from(`${parts.timestamp}.`, 'utf8'), rawBody]);
  const expected = crypto.createHmac('sha256', token).update(signed_content).digest('hex');
  const received = Buffer.from(parts.signature, 'utf8');
  const computed = Buffer.from(expected, 'utf8');
  return received.length === computed.length && crypto.timingSafeEqual(received, computed);
}

```

Buildkite suggests also rejecting signatures whose timestamp falls outside a window such as five minutes. That check is optional; if you add it, remember that a delivery replayed later keeps its original timestamp.

> **Make Buildkite webhooks production-ready.** [Hookdeck Event Gateway](/event-gateway) checks Buildkite's webhook token, records every build and job event, and gives you retries and replay that Buildkite doesn't document.

In Hookdeck, create a Buildkite source and paste the webhook token. Hookdeck verifies the token in `X-Buildkite-Token`, so set the Buildkite webhook to send the token, not a signature. Verification is optional on the source.

## Buildkite webhook limitations and pain points

### The default mode sends the secret in every request

**The Problem:** Anyone who can read a request can reuse the token.

**Why It Happens:** Token mode sends the token in clear text, and it is the default. Signature mode avoids this but needs HMAC code on the receiver.

**Workarounds:**

* Only use HTTPS URLs, and keep **Verify TLS Certificates** on.
* Don't log request headers in full.
* Switch to signature mode if the endpoint receives Buildkite directly.

**How Hookdeck Can Help:** With Hookdeck in front, the token only reaches Hookdeck's HTTPS endpoint. Your own service receives events from Hookdeck and doesn't need the Buildkite token at all.

### No documented retries or timeout

**The Problem:** You can't tell whether Buildkite will try again if your endpoint is down or slow.

**Why It Happens:** Buildkite's webhook docs describe no retry schedule, timeout or success criterion. The request log only keeps the last 20 requests.

**Workarounds:**

* Return `2xx` quickly and process asynchronously.
* After an outage, reconcile recent builds through the REST API.

**How Hookdeck Can Help:** Hookdeck stores every request it receives, keeps a full history beyond 20 entries, and retries delivery to your service on a schedule you set.

### No delivery ID

**The Problem:** There is no header or field that identifies a delivery, which makes duplicates hard to spot.

**Why It Happens:** Buildkite doesn't document an event or delivery ID.

**Workarounds:**

* Build a key from the event and the resource, such as `event` plus `build.id`, or `event` plus `job.id`, plus the state.
* Make handlers idempotent so a repeat does no harm.

**How Hookdeck Can Help:** Hookdeck's deduplication rules can match on body fields such as the event name and build ID, and drop repeats within a window you choose.

### Payloads can be ahead of the event

**The Problem:** A `build.finished` arrives showing the build as running again, or the reverse.

**Why It Happens:** Buildkite builds the payload from the current object, so fast transitions can produce a later state than the one that fired the event.

**Workarounds:**

* Route on the event name, and treat `build.state` as a hint.
* When the exact state matters, fetch the build from the REST API.

**How Hookdeck Can Help:** Hookdeck keeps the event history per source, so you can see the order in which Buildkite sent events when debugging state problems.

## Best practices

### Route on `X-Buildkite-Event`

The header and the body's `event` field carry the same value. Using the header lets you route before parsing JSON.

### Use queue and pipeline filters

Job events from a busy organisation add up quickly. Limit each webhook to the queues, pipelines and branches its consumer needs.

### Use payload timestamps for timing

The signature timestamp marks when the request was sent. For build and job timing, use `build.started_at`, `build.finished_at`, `job.started_at` and `job.finished_at`.

### Process asynchronously

Return `2xx` as soon as the event is stored. See [why to process webhooks asynchronously](/webhooks/guides/why-implement-asynchronous-processing-webhooks).

## Conclusion

Buildkite webhooks give you build, job and agent events with flexible filtering, authenticated by a token that is sent either in plain text or as an HMAC signature. Compare tokens in constant time, build your own deduplication key, and don't rely on the payload's state when transitions are fast.

[Hookdeck Event Gateway](https://hookdeck.com) stores every event and adds the retries and history that Buildkite leaves undocumented.

[Get started with Hookdeck](https://dashboard.hookdeck.com/signup) for free and handle Buildkite webhooks reliably in minutes.