# Guide to camelAI Webhooks: Features and Best Practices

[camelAI](https://camelai.com) runs AI agents for you on camelRun, its hosted agent runtime. An agent run can take a while, can stop to ask a person a question, and costs money with every model response, so camelRun's webhooks report exactly those moments: a run started, finished or failed, an agent is waiting for input, usage was recorded, or a balance is running low. (camelAI is not CAMEL-AI.org, the open-source multi-agent framework.)

Here you'll find camelRun's event envelope and its 20 types, how endpoints and secrets are managed through the API, how the Standard Webhooks signature works during a rotation, and what camelRun's 3-day retry window means for your receiver.

## What are camelAI webhooks?

A camelRun webhook endpoint is a URL with a list of event types, created through the API or the camelAI console. You can have up to 16. Each endpoint gets its own `whsec_` secret, which camelRun returns once, and every request is signed per Standard Webhooks.

The body is always an envelope of `id`, `type`, `created` and `data`. camelRun keeps `data` small, with IDs and key facts, and expects you to fetch anything larger, such as a run's reply, with your API key. One useful detail: the `metadata` you attach to a message when you start a run comes back on that run's events, so you can route an event to the right conversation or tenant without a lookup.

## camelAI webhook features

| Feature | Details |
| --- | --- |
| Configuration | `POST https://run.camelai.com/v1/webhooks` or the console. `GET`, `PATCH`, `DELETE` to manage |
| Endpoints | Up to 16. OAuth apps can't create or change them |
| Signature | Standard Webhooks HMAC-SHA256 |
| Headers | `webhook-id` (the event `id`), `webhook-timestamp` (Unix seconds), `webhook-signature` |
| Secret | `whsec_...` per endpoint, shown once. Replacing it keeps the old one signing for 24 hours |
| Body | JSON envelope `{id, type, created, data}` |
| Event routing | Body field `type` |
| Response window | `2xx` within 10 seconds |
| Retries | Exponential backoff, 5 seconds to an hour apart, for 3 days |
| Ordering | Not guaranteed. Retries can reorder events |
| Versioning | `data` only gains fields. Breaking changes come as new types |
| Source IPs | Not published |

## Events and payloads

| Event | Triggered When | Common Use Cases |
| --- | --- | --- |
| `run.started` | An agent run begins | Track run lifecycle |
| `run.completed` | A run finishes successfully | Collect results, notify users |
| `run.failed` | A run fails | Alerting, retries |
| `input.requested` | A run asks for input | Route a question to a person |
| `input.resolved` | A requested input is answered | Resume or audit a run |
| `usage.recorded` | Usage is recorded | Metering and cost tracking |
| `voice.session.started` | A voice session starts | Open a call record |
| `voice.session.ended` | A voice session ends | Close a call record |
| `voice.session.error` | A voice session errors | Alerting |
| `voice.utterance` | An utterance is produced | Live transcripts |
| `voice.speech.started` | Speech starts | Turn-taking, barge-in |
| `voice.speech.ended` | Speech ends | Turn-taking |
| `billing.balance.low` | The account balance is low | Top-up reminders |
| `billing.balance.depleted` | The account balance is used up | Pause workloads |
| `end_user.balance.low` | An end user's balance is low | Prompt the end user to top up |
| `end_user.credit.exhausted` | An end user's credit runs out | Block or upsell |
| `end_user.credit.granted` | Credit is granted to an end user | Ledger sync |
| `end_user.credit.refunded` | Credit is refunded to an end user | Ledger sync |
| `end_user.credit.adjusted` | An end user's credit is adjusted | Ledger sync |
| `end_user.allowance.low` | An end user's allowance is low | Warn the end user |

camelRun's documented envelope, for a completed run:

```json
{"id": "evt_3f9c0a1b2c3d4e5f60718293", "type": "run.completed", "created": 1790000000,
 "data": {"agentId": "client_...", "requestId": "...", "method": "prompt", "metadata": {"source": "web"},
          "replyIndex": 3, "messageCount": 4,
          "usage": {"responses": 2, "input": 2300, "output": 140, "cacheRead": 0, "cacheWrite": 0, "costUsd": 0.0081}}}

```

A few fields need attention:

* `run.completed` doesn't always mean the agent finished its task. If the run stopped early, `stopped` says why: `input_required` (with `inputIds`), `spend_limit` or `turn_limit`.
* `run.failed` adds `error` and, where the error has a name, `code`, such as `cancelled`, `aborted`, `model_stream_stalled` or `turn_limit`. `uncertain` marks a run that a camelRun restart cut short.
* `input.requested` carries `inputId`, `toolCallId`, `kind` and `expiresAt`, so a human-in-the-loop handler knows how long it has.
* `usage.recorded` has a `kind` of `response`, `compaction`, `transcription`, `image` or `speech`. camelRun says kinds may be added, so treat unknown ones as other usage.
* Billing amounts are integers in micro-USD. `billing.balance.low` fires when the balance crosses the threshold (default $2), not on every charge below it. A threshold change can also fire it, with `source: "threshold_changed"` and no `previousBalance`.

The voice and end-user events are described on camelAI's [voice calls](https://camelai.com/docs/camelrun/voice-calls) and [end-user credits](https://camelai.com/docs/camelrun/end-user-credits#webhooks) pages.

## Setting up camelAI webhooks

Register the endpoint and the event types it wants:

```bash
curl https://run.camelai.com/v1/webhooks \
  -H "Authorization: Bearer $CAMELAI_API_KEY" -H "Content-Type: application/json" \
  -d '{"url": "https://example.com/webhooks/camel-ai", "events": ["run.completed", "run.failed", "input.requested"], "description": "prod"}'

```

The response holds the endpoint's `id` (`we_...`) and its `secret`. Save the secret as `CAMELAI_WEBHOOK_SECRET` before doing anything else, because camelRun won't show it again. The console offers the same thing without code, and the Python SDK has `create_webhook`, `webhooks`, `update_webhook`, `delete_webhook` and `rotate_webhook_secret` on `AgentRuntime`.

Each endpoint records who set it in `setBy` (`token:<id>`, `console` or `oauth:<grant id>`). Revoking an API token doesn't stop the endpoints it created; the revoke response lists them in `left` so you can review them.

An endpoint only sees runs that start after it exists, so create it before you kick off the runs you want to watch. For local testing, the [Hookdeck CLI](/docs/cli) gives you a public URL:

```bash
npx hookdeck-cli listen 3000 camel-ai --path /webhooks/camel-ai

```

**With Hookdeck.** Make a Hookdeck source of type **camelAI**, paste the endpoint's `whsec_` secret, and register the source URL as the camelRun endpoint. The source only accepts `POST`. camelRun gets a `200` for every request; any request whose signature fails is kept in Hookdeck as failed verification and is not delivered onward.

## Securing camelAI webhooks

camelRun signs `{webhook-id}.{webhook-timestamp}.{raw body}` with HMAC-SHA256, keyed with the base64 part of the secret after `whsec_`, and sends `v1,<base64>` in `webhook-signature`. For 24 hours after you replace a secret, both the old and new secrets sign each request, and the header holds two space-separated signatures. The function below passes if any `v1` entry matches:

```javascript
const crypto = require('node:crypto');

function decodeBase64(value) {
  if (typeof value !== 'string' || !/^[A-Za-z0-9+/]+={0,2}$/.test(value)) return null;
  const bytes = Buffer.from(value, 'base64');
  return bytes.length > 0 && bytes.toString('base64').replace(/=+$/, '') === value.replace(/=+$/, '')
    ? bytes : null;
}

function verifySignature(rawBody, headers, secret) {
  if (!Buffer.isBuffer(rawBody) || typeof secret !== 'string' || !secret) return false;
  const prefix = headers['webhook-id'] !== undefined ? 'webhook' : 'svix';
  const id = headers[`${prefix}-id`];
  const timestamp = headers[`${prefix}-timestamp`];
  const signature = headers[`${prefix}-signature`];
  if (typeof id !== 'string' || !id || typeof timestamp !== 'string' ||
      !/^[0-9]+$/.test(timestamp) || typeof signature !== 'string') return false;
  const key = decodeBase64(secret.replace(/^whsec_/, ''));
  if (!key) return false;
  const message = Buffer.concat([Buffer.from(`${id}.${timestamp}.`, 'utf8'), rawBody]);
  const candidates = signature.split(/\s+/).filter((item) => item.startsWith('v1,')).map((item) => item.slice(3));
  const expected = crypto.createHmac('sha256', key).update(message).digest();
  return candidates.some((candidate) => {
    const received = decodeBase64(candidate);
    if (!received) return false;
    return received.length === expected.length && crypto.timingSafeEqual(received, expected);
  });
}

```

In the route, deduplicate on `id` and use the `metadata` you sent with the message to find the conversation:

```javascript
const express = require('express');

// Returns the parsed body when it is a JSON object, otherwise null.
function parseJsonObject(rawBody) {
  try {
    const value = JSON.parse(rawBody.toString('utf8'));
    return value && typeof value === 'object' && !Array.isArray(value) ? value : null;
  } catch {
    return null;
  }
}

const app = express();

app.post('/webhooks/camel-ai', express.raw({ type: '*/*' }), async (req, res) => {
  const secret = process.env.CAMELAI_WEBHOOK_SECRET;
  if (!secret) return res.sendStatus(500);
  if (!verifySignature(req.body, req.headers, secret)) return res.sendStatus(401);

  const event = parseJsonObject(req.body);
  if (!event) return res.sendStatus(400); // signed, but not a JSON object
  if (typeof event.type !== 'string' || !event.data || typeof event.data !== 'object') return res.sendStatus(400);
  try {
    if (!(await recordEventOnce(event.id))) return res.sendStatus(200);

    if (event.type === 'input.requested') {
      await askHuman(event.data.metadata, event.data.inputId, event.data.expiresAt);
    } else if (event.type === 'run.failed') {
      await notifyThread(event.data.metadata, event.data.code || event.data.error);
    }
  } catch {
    await releaseEvent(event.id).catch(() => {}); // free the claim so the retry is processed
    return res.sendStatus(503); // not processed, so don't acknowledge
  }
  return res.sendStatus(200);
});

```

`recordEventOnce`, `askHuman` and `notifyThread` are placeholders for your own storage and messaging.

camelAI's Python helper, `verify_webhook`, rejects any request sent more than 5 minutes ago, and the TypeScript `standardwebhooks` library it suggests does the same. Hookdeck has no timestamp tolerance and keeps camelRun's original `webhook-timestamp` when it retries or replays, so those helpers would reject events that were simply delayed. Use a signature-only check like the one above when Hookdeck is in front of your handler, and lean on deduplication by `id` for replay protection.

> **Make camelAI webhooks production-ready.** [Hookdeck Event Gateway](/event-gateway) verifies camelRun's signatures, stores every run and billing event, and delivers them to your workers at a pace they can handle.

## camelAI webhook limitations and pain points

### The secret appears once

**The Problem:** You need the signing secret again and the API won't return it.

**Why It Happens:** camelRun returns `secret` only in the create response.

**Workarounds:**

* Call `POST /v1/webhooks/{webhookId}/secret` to get a new one. The old secret keeps signing for 24 hours, so deploy the new one within that window.
* Accept both secrets in your verifier during the changeover, as the Python SDK does when you pass a list.

**How Hookdeck Can Help:** Behind Hookdeck, a rotation means updating the secret on one source rather than in every service.

### Retries reorder events

**The Problem:** `run.started` lands after `run.completed`, or two balance events arrive backwards.

**Why It Happens:** camelRun delivers at least once and says retries can reorder deliveries.

**Workarounds:**

* Order by `created` and ignore anything older than what you have stored.
* Deduplicate by `id`, which a retried event keeps.

**How Hookdeck Can Help:** Hookdeck's [deduplication](/docs/deduplication) can discard repeated event IDs before your handler sees them.

### `run.completed` isn't always done

**The Problem:** A run reported as completed hasn't produced the answer you expected.

**Why It Happens:** A run that stops for input or hits a spend or turn limit still ends as `run.completed`, with the reason in `stopped`.

**Workarounds:**

* Check `data.stopped` before marking a task as finished.
* Subscribe to `input.requested` if your product needs to answer agents' questions.

**How Hookdeck Can Help:** Hookdeck [filters](/docs/filters) can route completed runs that carry `stopped` to a different destination from those that don't.

### The deprecated usage webhook is a different format

**The Problem:** A handler written for endpoint events breaks on usage callbacks from an older integration.

**Why It Happens:** The legacy `PUT /v1/usage-webhook` posts each model response's usage as a bare body with no envelope and no `type`, and camelAI doesn't describe how, or whether, it is signed.

**Workarounds:**

* Move to an endpoint subscribed to `usage.recorded`, which camelAI recommends for new integrations.
* Keep any remaining usage-webhook traffic on a separate route.

**How Hookdeck Can Help:** A separate Hookdeck source for the legacy URL keeps that traffic out of the verified camelAI source.

## Best practices

### Answer within 10 seconds, then do the work

camelRun counts anything slower as a failure. Its own guidance is to queue the event and return `200`, then fetch the run's reply from `GET /v1/agents/{id}/requests/{requestId}` afterwards. See [why to process webhooks asynchronously](/webhooks/guides/why-implement-asynchronous-processing-webhooks).

### Keep the body raw

Signatures are computed over the exact bytes. Use `express.raw({ type: '*/*' })`, `await request.text()` in a Next.js route handler, or `request.get_data()` in Flask.

### Put routing hints in `metadata`

Send your own IDs as `metadata` when you start a run. They come back on its events, so a handler can act on the right thread or customer without a database lookup.

### Tolerate new fields and kinds

camelRun adds fields to `data` over time and may add `usage.recorded` kinds. Ignore fields you don't use and don't fail on unknown kinds. See our [guide to webhook idempotency](/webhooks/guides/implement-webhook-idempotency) for handling the duplicates that at-least-once delivery brings.

## Conclusion

camelRun's webhooks are built for agent workloads: small envelopes that point back to the run, your own `metadata` carried through, and dedicated events for human input and spend. Verify with the endpoint's secret and accept two signatures during a rotation, check `stopped` before trusting `run.completed`, order by `created`, and answer inside 10 seconds.

When your handler is down for longer than camelRun's 3 days of retries, events are gone. [Hookdeck Event Gateway](https://hookdeck.com) accepts and stores them first, so that limit stops applying to you. [Open a free Hookdeck account](https://dashboard.hookdeck.com/signup) to connect camelRun.