Gareth Wilson Gareth Wilson

Guide to FireHydrant Webhooks: Features and Best Practices

Published


FireHydrant webhooks push incident activity to your other systems as it happens: an incident's milestone changes, a role is assigned, a task is completed, a change event is recorded, or a Signals alert is opened, acknowledged or resolved. FireHydrant is an incident-management platform, and webhooks are how you keep status pages, ticketing systems, data warehouses and custom tooling in step with it.

This guide covers how FireHydrant webhooks are configured and signed, what the payloads look like, what the documentation leaves out, and the best practices for running a FireHydrant receiver in production.

What are FireHydrant webhooks?

A FireHydrant webhook is a configuration with one destination URL, a secret, a status and a set of event subscriptions. When something you subscribed to happens, FireHydrant sends a JSON payload to the URL with an fh-signature header: a hex HMAC-SHA256 of the request body, keyed with the secret.

FireHydrant uses the same webhook configurations for Command Extensions, which let you build custom Slack commands that call external systems. A webhook used only for Command Extensions can leave every event subscription unchecked. Runbooks also have a separate Send a Webhook step. This guide covers the event-triggered webhooks.

FireHydrant webhook features

FeatureDetails
ConfigurationSettings > Webhooks in FireHydrant, with Owner permission
DestinationsOne URL per webhook. Create several webhooks to send to several destinations
SubscriptionsChange Events, Incidents, Incidents (Private), Signals (BETA)
StatusActive or Inactive. Inactive webhooks keep their settings but don't send
Signature headerfh-signature
SchemeHMAC-SHA256, hex digest
Signed contentThe raw request body only
SecretA value you type in when you create the webhook
Replay windowNone. No timestamp is sent or signed
Event IDNone
HTTP method, timeout and retriesNot documented
Source IPsNot published

Common events

FireHydrant doesn't send named event types. Each payload has an event object with an operation and a resource_type, and the resource type tells you which subscription it came from.

Subscriptionresource_typeFires when
Change Eventschange_eventA new change event, such as a deploy, is created in FireHydrant
IncidentsincidentAnything changes on a public incident
Incidents (Private)Not documentedThe same as Incidents, for private incidents
Signals (BETA)signals_alertsAny Signals alert activity, including state changes

For the two incident subscriptions, FireHydrant lists nine kinds of change that trigger a webhook: chat messages posted to the timeline, follow-up updates, general updates to any field including custom fields, milestone changes, edits to milestone timestamps, notes and infrastructure updates, role assignments, task updates, and team assignments. They all arrive with resource_type: "incident" and the full incident under data.incident, so nothing in the payload says which kind of change it was.

Here is FireHydrant's Signals example:

{
  "data": {
    "alert": {
      "body": "This is the description of the Alert, if there is one",
      "decision": null,
      "id": "6f767579-0015-400e-8c6e-cb481d5760c2",
      "starts_at": "2024-03-29 22:50:07 UTC",
      "status": "opened",
      "summary": "This is the Summary/Title of the Alert"
    }
  },
  "event": {
    "operation": "CREATED",
    "resource_type": "signals_alerts"
  }
}

For Signals, event.operation is CREATED when the alert is created and UPDATED for everything after. data.alert.decision records what happened: null on creation, then ACKNOWLEDGE, ESCALATE, RESOLVE, DISMISS or TIMEOUT. data.alert.status is the result: opened, acknowledged, resolved or ignored.

The examples aren't consistent about the case of operation. The incident example uses UPDATED, the change event example uses updated, and the Signals example uses CREATED. Compare it case-insensitively. Timestamps also differ in format: incidents and alerts use 2023-07-25 13:34:49 UTC, while change events use ISO 8601 such as 2019-05-31T16:16:11.728Z.

Setting up FireHydrant webhooks

  1. In FireHydrant, go to Settings and click Webhooks. You need Owner permissions.
  2. Click + Add webhook.
  3. Enter the URL the webhook should send to.
  4. Enter a Secret. FireHydrant doesn't generate one, so use a long random value and store it server-side as well, for example as FIREHYDRANT_WEBHOOK_SECRET.
  5. Set Status to Active.
  6. Under Event Subscriptions, choose Change Events, Incidents, Incidents (Private) and Signals as needed.
  7. Click Add webhook.

Always set a secret. The form doesn't require one, and FireHydrant doesn't document what it sends when the secret is empty.

If you use Hookdeck, create a source with the FireHydrant source type, set the same secret on it, and use the Hookdeck source URL as the webhook URL.

Securing FireHydrant webhooks

Compute an HMAC-SHA256 of the raw request body with your secret, encode it as hex, and compare it in constant time with fh-signature. FireHydrant documents this as pseudocode: HMAC.hexdigest('sha256', secret, body) == signature.

const crypto = require('crypto');
const express = require('express');

function verifyFireHydrantSignature(rawBody, signatureHeader, secret) {
  if (!signatureHeader || !secret) return false;

  const expected = Buffer.from(crypto.createHmac('sha256', secret).update(rawBody).digest('hex'), 'utf8');
  const received = Buffer.from(signatureHeader.trim().toLowerCase(), 'utf8');
  return received.length === expected.length && crypto.timingSafeEqual(received, expected);
}

const app = express();

app.post('/webhooks/firehydrant', express.raw({ type: '*/*' }), (req, res) => {
  if (!verifyFireHydrantSignature(req.body, req.get('fh-signature'), process.env.FIREHYDRANT_WEBHOOK_SECRET)) {
    return res.sendStatus(401);
  }

  const payload = JSON.parse(req.body.toString('utf8'));
  const resourceType = payload.event?.resource_type;
  const operation = (payload.event?.operation || '').toUpperCase();
  // Route on resourceType and operation, then process async.
  res.sendStatus(200);
});

The same check in Python:

import hashlib
import hmac
import os


def verify_firehydrant_signature(raw_body: bytes, signature_header, secret: str) -> bool:
    if not signature_header or not secret:
        return False
    expected = hmac.new(secret.encode("utf-8"), raw_body, hashlib.sha256).hexdigest()
    received = signature_header.strip().lower().encode("utf-8")
    return hmac.compare_digest(received, expected.encode("utf-8"))


# In FastAPI: raw_body = await request.body()
# verify_firehydrant_signature(raw_body, request.headers.get("fh-signature"),
#                              os.environ["FIREHYDRANT_WEBHOOK_SECRET"])

To check your verifier, use the secret firehydrant-test-secret and this exact body:

{"data":{"alert":{"id":"6f767579-0015-400e-8c6e-cb481d5760c2","status":"opened"}},"event":{"operation":"CREATED","resource_type":"signals_alerts"}}

The fh-signature header should be:

517eae0c6dec210f1e738ca741765f9ab9f6bb18700fc9a6571b0094d4a8c49f

Make FireHydrant webhooks production-ready. Hookdeck Event Gateway verifies the fh-signature header on every FireHydrant request and durably queues incident, change event and alert payloads, so the systems you sync during an incident don't miss an update.

FireHydrant webhook limitations and pain points

Delivery behaviour is undocumented

The Problem: You don't know how long FireHydrant waits for a response, whether it retries, or what counts as a failure.

Why It Happens: FireHydrant's webhook page covers configuration, signing and payloads, but not timeouts, retries or ordering. It doesn't name the HTTP method either.

Workarounds:

  • Respond with a 2xx quickly, and assume that a slow or failed response may lose the event.
  • Treat incident webhooks as a prompt to re-read the incident from the FireHydrant API, which also covers any events you missed.

How Hookdeck Can Help: Hookdeck accepts each request as soon as FireHydrant sends it and stores it, then retries delivery to your service on a schedule you set, so your own downtime doesn't lose incident updates.

High volume during incidents

The Problem: A busy incident floods your endpoint at exactly the moment your systems are under strain.

Why It Happens: Every chat message, task update, note, milestone change and role assignment on an incident sends a webhook with the full incident. A busy incident therefore produces many payloads that differ only slightly.

Workarounds:

  • Queue payloads and process them in the background.
  • Coalesce updates per incident ID and only act on the latest state.

How Hookdeck Can Help: Hookdeck queues bursts and can rate-limit delivery to your service, so a busy incident doesn't overload the systems you're trying to keep in sync.

Payloads don't say what changed

The Problem: You receive an incident update and can't tell whether it was a milestone change, a new note or a role assignment.

Why It Happens: All nine incident triggers send the full incident with the same event object. There is no field naming the change and no event ID.

Workarounds:

  • Store the last state per incident and diff it against the new payload.
  • Watch fields such as current_milestone and updated_at for the changes you care about.

How Hookdeck Can Help: Hookdeck filters can match on payload fields, for example forwarding only incidents whose current_milestone is resolved, so downstream services only see the changes they act on.

Inconsistent casing and formats

The Problem: A handler that matches operation === 'UPDATED' misses change events, or a date parser fails on incident timestamps.

Why It Happens: FireHydrant's examples use UPDATED, updated and CREATED for operation, and two different timestamp formats.

Workarounds:

  • Normalise operation to upper case before comparing.
  • Parse timestamps with a library that accepts both 2023-07-25 13:34:49 UTC and ISO 8601.

How Hookdeck Can Help: A Hookdeck transformation can normalise operation and timestamps before delivery, so every consumer sees one format.

Best practices

Verify against the raw body

The signature covers the exact bytes FireHydrant sent. Read the raw body (express.raw(), await request.body()) and parse only after verification.

Route on resource_type, then operation

event.resource_type tells you which subscription produced the payload. Branch on it first, then on operation compared case-insensitively. Log and acknowledge anything you don't recognise, including private incident payloads until you have seen one.

Use one webhook per destination

Each webhook sends to one URL. If several systems need FireHydrant events, either create several webhooks with their own secrets, or send one webhook to Hookdeck and fan out from there.

Deduplicate deliveries

Without an event ID, build a key from the resource ID and its updated_at (for incidents) or id and status (for alerts). See our guide to webhook idempotency.

Acknowledge fast and process asynchronously

Return a 2xx as soon as the payload is verified and stored, and do the work afterwards. See why to process webhooks asynchronously.

Conclusion

FireHydrant signs its webhooks with a hex HMAC-SHA256 of the raw body in fh-signature, keyed with a secret you choose. Always set that secret, verify against the raw bytes, route on resource_type and a case-insensitive operation, and be ready for bursts of full-incident payloads. FireHydrant doesn't document retries, so plan to reconcile from its API.

Hookdeck Event Gateway sits in front of your receiver, verifies every FireHydrant signature, and queues incident and alert payloads until your service has processed them.

Get started with Hookdeck for free and handle FireHydrant webhooks reliably in minutes.


Gareth Wilson

Gareth Wilson

Product Marketing

Multi-time founding marketer, Gareth is PMM at Hookdeck and author of the newsletter, Community Inc.