Gareth Wilson Gareth Wilson

Guide to Convox Webhooks: Features and Best Practices

Published


Convox is a platform-as-a-service that you install in your own AWS, GCP, Azure or DigitalOcean account. Each installation, called a Rack, runs your apps on Kubernetes and gives you a Heroku-style CLI and Console on top. Convox webhooks are how a Rack tells the outside world what it is doing: a build started or failed, a release was promoted, an app crossed its monthly budget, or the auto-shutdown countdown armed and scaled services to zero. Teams point them at Slack, Discord or PagerDuty, or at their own service for deploy tracking and cost control.

This guide covers the Rack's event catalogue, how to turn on signing with webhook_signing_key, how to verify Convox-Signature through a key rotation, and how to live with webhooks that are never retried.

What are Convox webhooks?

Webhooks are configured per app. In Console you add receiver URLs under App > Settings > Webhooks, and for budget-cap events you can also set budget.atCapWebhookUrl at the top level of convox.yml. The Rack stores them and sends every receiver every action as an HTTP POST with Content-Type: application/json. There is no event filter and no per-service URL.

Because the Rack runs in your own cloud account, the requests come from your infrastructure rather than from Convox. Signing arrived in Rack 3.24.6 and is opt-in: once you set the webhook_signing_key Rack parameter, each delivery carries Convox-Signature with a timestamp and one or more HMAC-SHA256 signatures.

Convox webhook features

FeatureDetails
ConfigurationConsole: App > Settings > Webhooks, or budget.atCapWebhookUrl in convox.yml
SubscriptionNone. Every receiver gets every action
Signaturehex HMAC-SHA256
Signature headerconvox-signature, as t=<unix>,v1=<hex>[,v1=<hex>...]
Signed content<t>.<raw body>
SecretThe webhook_signing_key Rack parameter, chosen by you
RotationUp to 4 active keys, one v1= each
BodyJSON
Event routingBody field action
Timeout30 seconds by default, configurable per receiver on 3.24.6+
RetriesNone. Best effort, fire-and-forget, even on 5xx
OrderingNot guaranteed. Duplicates are possible

Events and payloads

status qualifies some events: success or error on build:create (with data.message carrying the build error), start on release:promote, skipped on release:prometheus-skipped.

App lifecycle, builds and releases

ActionTriggered when
app:createconvox apps create succeeds
build:createA build starts, and again on build failure (status: "error")
build:import-image:startconvox build import-image starts
build:import-image:doneconvox build import-image finishes
build:cancelconvox builds cancel deletes the build pod (Rack 3.25.7+)
release:createA release is created from a successful build
release:promoteA promote starts (status: "start")
app:promote:completedThe rollout watcher sees the promote finish
app:promote:erroredThe rollout watcher sees the promote fail (data.message has the error)
app:promote:cancelledThe promote was superseded or cancelled (data.message has the detail)
release:autoscale-disabledA service requests autoscale on a Rack without keda_enable=true
release:manifest-advisoryA service configuration is invalid at render time
release:prometheus-skippedKEDA Prometheus trigger creation was skipped (status: "skipped")
release:imperative-patch-noteconvox scale patched a KEDA-managed service's ScaledObject

Budget cap and cost

ActionTriggered when
app:budget:setconvox budget set or a Console budget change
app:budget:resetconvox budget reset clears the circuit breaker
app:budget:clearconvox budget clear removes the budget config
app:budget:thresholdalertThresholdPercent is crossed
app:budget:capmonthlyCapUsd is crossed
app:budget:breaker-clearedA cap raise clears the block-new-deploys breaker
app:budget:per-service-truncatedThe per-service breakdown exceeded its cap and entries were dropped

Scale and trigger overrides

ActionTriggered when
app:scale-override:toggledThe per-service scale override is switched on or off
app:scale-override:honoredA deploy preserved an active scale override
app:triggers-override:toggledThe per-service autoscaler override is switched on or off
app:triggers-override:threshold-setA single trigger's threshold was edited

Auto-shutdown lifecycle

ActionTriggered when
app:budget:auto-shutdown:armedThe countdown begins
app:budget:auto-shutdown:firedThe countdown elapsed and services scaled to zero
app:budget:auto-shutdown:cancelledAn armed countdown was cancelled before :fired (data.cancel_reason)
app:budget:auto-shutdown:restoredServices were restored from the persisted shutdown state
app:budget:auto-shutdown:expiredManual-mode month rollover with the user absent
app:budget:auto-shutdown:flap-suppressedA cap fire was suppressed by the 24-hour cooldown
app:budget:auto-shutdown:noopThe reconciler decided no action was needed
app:budget:auto-shutdown:failedShutdown patch retries were exhausted
app:budget:auto-shutdown:simulatedconvox budget simulate-shutdown was run
app:budget:auto-shutdown:dismissedThe recovery banner was dismissed

cancel_reason is one of reset-during-armed, cap-raised, manual-detected, config-changed or countdown-lapsed (Rack 3.25.10+).

Every body carries data.app, data.rack and data.actor. The actor is the email from the user's JWT for actions someone took, such as convox budget reset, and "system" for events the Rack generates itself, such as threshold crossings and the auto-shutdown countdown. Rack-generated events also carry data.tick_id. Convox tells receivers to ignore actions they don't recognise, because the catalogue grows with each Rack release and some rows need a later Rack version.

Published payload example

This is the app.budget.auto-shutdown.cancelled sample from the webhook samples library. It is the one complete example in Convox's webhook reference.

{
  "action": "app:budget:auto-shutdown:cancelled",
  "status": "success",
  "timestamp": "2026-04-27T10:30:00Z",
  "data": {
    "app": "myapp",
    "rack": "rack1",
    "actor": "alice@example.com",
    "tick_id": "tick-2026-04-27T10:30:00Z-3a7b4c2d8e9f4a6b8c4d3e2f1a0b9c8d",
    "cancel_reason": "reset-during-armed"
  }
}

Setting up Convox webhooks

You need a Rack on 3.24.6 or later for signing, permission to set Rack parameters, and the Admin role in Console if you want to reveal an existing key.

Signing is opt-in: set the webhook_signing_key rack parameter (Rack 3.24.6 or later) and paste the same value. Without it, deliveries are unsigned and fail verification.

Choose a signing key

You choose the key. The Rack uses the value as-is, so any string with enough entropy works. Convox suggests 32 random bytes as hex:

convox rack params set webhook_signing_key=$(openssl rand -hex 32)

Console can also generate, reveal and rotate the key under Rack > Settings. Revealing needs the Admin role; other users see a masked value.

Put the same value in WEBHOOK_SECRET on the receiver. Even though the suggested value looks like hex, use it as the literal string; don't hex-decode it. CLIs older than 3.24.6 print the key in plain text from convox rack params, so upgrade the CLI first.

Add the receiver

  1. In Console, open your app, then App > Settings > Webhooks.
  2. Add your endpoint URL, for example https://example.com/webhooks/convox.
  3. For budget-cap events you can instead set budget.atCapWebhookUrl at the top level of convox.yml. It applies to the whole app.

A receiver config can also be a JSON object with url and timeout (Rack 3.24.6+), for example {"url":"https://audit.internal.corp/v1/events","timeout":"60s"}.

Rotate without downtime

A Rack supports up to 4 active keys at once. While more than one is active, it signs with all of them and sends one v1= per key.

  1. Generate a new key and set it on the Rack.
  2. Update your receivers to the new key one by one. A receiver holding either key keeps verifying, because it accepts the delivery if any v1= matches.
  3. Remove the old key from the Rack.

Setting more than 4 keys is rejected. If you downgrade a Rack below 3.24.6 it stops sending Convox-Signature, so set a fresh key after upgrading again.

To try it from your machine, forward deliveries with the Hookdeck CLI and add the forwarding URL as a receiver:

npx hookdeck-cli listen 3000 convox --path /webhooks/convox

Securing Convox webhooks

Split Convox-Signature on commas. Take the single t= value as a string and every v1= value. Compute an HMAC-SHA256 of t, a full stop and the raw body, keyed with the signing key's UTF-8 bytes, encode it as hex, and accept the request if any v1= matches in constant time. See the verification documentation.

const crypto = require('node:crypto');

function verifySignature(rawBody, headers, secret) {
  if (!Buffer.isBuffer(rawBody) || typeof secret !== 'string' || !secret) return false;
  const signature = headers['convox-signature'];
  if (typeof signature !== 'string') return false;
  const key = Buffer.from(secret, 'utf8');
  const parts = signature.split(',').map((part) => part.trim());
  const times = parts.filter((part) => part.startsWith('t=')).map((part) => part.slice(2));
  if (times.length !== 1 || !/^[0-9]+$/.test(times[0])) return false;
  const message = Buffer.concat([Buffer.from(`${times[0]}.`, 'utf8'), rawBody]);
  const candidates = parts.filter((part) => part.startsWith('v1=')).map((part) => part.slice(3));
  const expected = crypto.createHmac('sha256', key).update(message).digest();
  return candidates.some((candidate) => {
    if (!/^[0-9a-fA-F]{64}$/.test(candidate)) return false;
    const received = Buffer.from(candidate, 'hex');
    return received.length === expected.length && crypto.timingSafeEqual(received, expected);
  });
}

The handler verifies, skips repeated Rack ticks, and routes the actions that need a person's attention: failed builds and promotes, budget caps and an auto-shutdown that has actually fired.

const express = require('express');

const PAGE_ON = new Set(['app:promote:errored', 'app:budget:cap', 'app:budget:auto-shutdown:fired', 'app:budget:auto-shutdown:failed']);

function createConvoxApp({ seenTick, page, record }) {
  const app = express();
  app.post('/webhooks/convox', express.raw({ type: '*/*' }), async (req, res) => {
    const secret = process.env.WEBHOOK_SECRET;
    if (!secret) return res.sendStatus(500);
    if (!verifySignature(req.body, req.headers, secret)) return res.sendStatus(401);

    let parsed;
    try {
      parsed = JSON.parse(req.body.toString('utf8'));
    } catch {
      return res.sendStatus(400);
    }
    if (!parsed || typeof parsed !== 'object' || Array.isArray(parsed)) return res.sendStatus(400);
    const { action, status, timestamp, data } = parsed;
    if (typeof action !== 'string') return res.sendStatus(400);

    try {
      if (data?.tick_id && (await seenTick(data.tick_id, action))) return res.sendStatus(200);
      await record({ action, status, timestamp, app: data?.app, rack: data?.rack, actor: data?.actor });
      const buildFailed = action === 'build:create' && status === 'error';
      if (buildFailed || PAGE_ON.has(action)) await page(action, data);
      // Unknown actions are recorded and otherwise ignored, as Convox advises.
    } catch {
      return res.sendStatus(503);
    }
    return res.sendStatus(200);
  });
  return app;
}

The t= timestamp is signed, so your own receiver can reject old deliveries if you want. Convox recommends a window but doesn't enforce one, and because the Rack never retries, a late delivery is rare unless something replays it. If you add a check, keep it optional.

Hookdeck's Convox source type parses t=, signs <t>.<raw body> with your key as UTF-8, and accepts a match on any v1=, so rotations work without changes. It applies no timestamp window, which means retries and replays from Hookdeck verify whenever they run. Hookdeck answers the Rack with 200 in all cases, records a failed check as failed verification and does not forward that request.

Make Convox webhooks production-ready. Hookdeck Event Gateway receives every Rack event, verifies Convox-Signature across key rotations, and adds the retries and replay that Convox's fire-and-forget delivery leaves out.

Convox webhook limitations and pain points

Nothing is retried

The Problem: A deploy notification or a budget-cap alert disappears because your receiver was down for a minute.

Why It Happens: Convox webhooks are best effort and fire-and-forget. The Rack sends each event once and doesn't retry, not even on a 5xx. A 401 from a misconfigured receiver loses the event just as surely.

Workarounds:

  • Keep the receiver simple and highly available.
  • Get verification right before you depend on the feed.
  • For budget and shutdown state, check the Rack with the CLI rather than relying only on events.

How Hookdeck Can Help: Point the receiver URL at Hookdeck. Hookdeck accepts each event as it arrives and stores it, then retries your service as many times as you configure and lets you replay anything that failed.

Signing is off until you turn it on

The Problem: Your endpoint accepts requests that nobody can prove came from your Rack.

Why It Happens: Racks before 3.24.6 never sign, and later Racks only sign once webhook_signing_key is set. Unsigned deliveries carry no Convox-Signature header at all.

Workarounds:

  • Upgrade the Rack and set a key before exposing a receiver publicly.
  • Fail closed when the header is missing.

How Hookdeck Can Help: With verification on, Hookdeck's Convox source flags every unsigned or wrongly signed delivery, so a Rack that lost its key after a downgrade shows up straight away.

Rotation reads every v1=

The Problem: Verification breaks halfway through a key rotation.

Why It Happens: During rotation the header carries one v1= per active key, and the first entry may belong to a key the receiver no longer holds. Code that reads only the first signature fails.

Workarounds:

  • Accept a match on any v1=, as the verifier above does.
  • Keep t as the exact string from the header when building the signed content.

How Hookdeck Can Help: Hookdeck checks every v1= value, so rotating the Rack key needs only an update to the source's secret.

Everything goes to every receiver

The Problem: A Slack channel meant for budget alerts fills with build events.

Why It Happens: Receivers can't subscribe to a subset of actions. Each receiver URL gets the full catalogue.

Workarounds:

  • Put a small router in front of chat tools and forward only the actions each one needs.
  • Use budget.atCapWebhookUrl for a receiver that should only see budget-cap events.

How Hookdeck Can Help: Hookdeck filters on action let one Convox receiver feed several destinations, each with only the events it wants.

Best practices

Verify the exact bytes

Read the body raw with express.raw({ type: '*/*' }), await request.arrayBuffer() or await request.body(). Don't hex-decode the key, and don't parse t into a number and format it again.

Deduplicate Rack ticks

Events can arrive twice. For system events, data.tick_id plus action is a natural key. For user actions, combine action, data.app and timestamp. See our guide to webhook idempotency.

Order by timestamp

The Rack doesn't guarantee ordering, so an armed event can follow cancelled. Apply auto-shutdown state by timestamp, and read cancel_reason to understand why a countdown stopped.

Ignore what you don't know

New actions appear as Racks are upgraded. Record and acknowledge them without failing.

Set a timeout that matches your receiver

If your receiver needs longer than 30 seconds, use the JSON receiver form with an explicit timeout, or better, store the event and answer straight away. Asynchronous processing avoids the problem altogether.

Conclusion

A Convox Rack sends a detailed stream of build, release and cost events, but it sends each one once and only signs it if you've set a key. Set webhook_signing_key, accept any v1= match so rotations are painless, deduplicate on tick_id, and put something durable in front of anything that can't afford to miss an event.

Hookdeck Event Gateway is that durable layer: it verifies each Rack event and keeps it until every destination has processed it.

Get started with Hookdeck for free and handle Convox webhooks reliably in minutes.


Gareth Wilson

Gareth Wilson

Product Marketing

Multi-time founding marketer, Gareth is PMM at Hookdeck and author of the newsletter, Community Inc.