Gareth Wilson Gareth Wilson

Guide to Axiom Webhooks: Features and Best Practices

Published


Axiom webhooks carry monitor alerts out of Axiom, the event and log data platform. When a monitor matches an event or crosses a threshold, a Custom webhook notifier can send a JSON notification to your own service, so you can open incidents in a tool Axiom doesn't integrate with, trigger automated remediation, or keep your own record of alerts.

This guide covers how the Custom webhook notifier works, how to authenticate its requests with a secret header, what the default template sends, and the practices that make an Axiom receiver dependable.

What are Axiom webhooks?

In Axiom, a monitor runs a query on a schedule and a notifier decides how you hear about the result. Axiom has notifiers for email, Slack, PagerDuty and other tools, plus a Custom webhook notifier that sends an HTTP POST with Content-Type: application/json to any URL you choose, "together with any other headers you specify".

The body is a Go template that you can rewrite completely. Axiom starts you from a default template that sends an action of Open or Closed and the monitor's details under event. Whatever your template renders is what your endpoint receives.

Axiom doesn't sign notifications. The way to tell a real Axiom request from a forged one is a secret header that you add to the notifier and check on your side. Axiom's API also has a plain webhook notifier that takes nothing but a URL. It can't send a header, so it can't be authenticated: use the Custom webhook notifier.

Axiom webhook features

FeatureDetails
ConfigurationMonitors > Manage notifiers > New notifier > Custom webhook
Method and formatPOST, Content-Type: application/json
BodyA Go template. The default sends action and event
Custom headersOptional, any name. The API splits them into headers and secretHeaders (masked in logs)
Signature and timestampNone
Event fieldNone of Axiom's own. The default template's action is Open or Closed
Timeout, retries, source IPsNot documented

Notifications and payloads

With the default template, there are two kinds of notification:

actionSent when
OpenA match monitor matches an event, or a threshold monitor triggers
ClosedA threshold monitor resolves

Match monitors only ever open. If you need a "resolved" signal, it comes from a threshold monitor.

This is Axiom's default template:

{
  "action": "{{.Action}}",
  "event": {
    "monitorID": "{{.MonitorID}}",
    "body": "{{.Body}}",
    "description": "{{.Description}}",
    "queryEndTime": "{{.QueryEndTime}}",
    "queryStartTime": "{{.QueryStartTime}}",
    "timestamp": "{{.Timestamp}}",
    "title": "{{.Title}}",
    "value": {{.Value}},
    "matchedEvent": {{jsonObject .MatchedEvent}},
    "groupKeys": {{jsonArray .GroupKeys}},
    "groupValues": {{jsonArray .GroupValues}}
  }
}

And this is what it renders when a threshold monitor triggers, from Axiom's docs:

{
  "action": "Open",
  "event": {
    "monitorID": "CabI3w142069etTgd0",
    "body": "Current value of 57347 is above or equal to the threshold value of 0",
    "description": "",
    "queryEndTime": "2024-06-28 14:55:57.631364493 +0000 UTC",
    "queryStartTime": "2024-06-28 14:45:57.631364493 +0000 UTC",
    "timestamp": "2024-06-28 14:55:57 +0000 UTC",
    "title": "Axiom Monitor Test Triggered",
    "value": 57347,
    "matchedEvent": null,
    "groupKeys": null,
    "groupValues": null
  }
}

A few details are easy to miss:

  • .Value only applies to threshold monitors, and it is 0 when a threshold monitor resolves because its query stopped returning data for a series, such as a grouped series that goes idle.
  • .MatchedEvent only applies to match monitors. .GroupKeys and .GroupValues only apply when the monitor's APL query groups by a non-time field.
  • The timestamps are strings in Go's default time format, not ISO 8601, so parse them explicitly.

Because you own the template, you can reshape the body for the receiving system. Axiom's own example builds an incident.io payload with "status": "{{ if eq .Action "Open" }}firing{{ else }}resolved{{ end }}" and the monitor ID as a deduplication key.

Inspect Axiom notifications as they arrive. Point a Custom webhook notifier at a Hookdeck Inspect URL to see exactly what your template renders, with no account or setup required.

Setting up Axiom webhooks

Choose a header name and a long random value first, for example Authorization and Bearer followed by the output of openssl rand -hex 32. Axiom doesn't generate a secret for you.

  1. Open the Monitors tab and click Manage notifiers.
  2. Click New notifier and name it.
  3. Click Custom webhook.
  4. Enter your HTTPS endpoint in Webhook URL.
  5. Optionally, edit the template. Keep a field that tells Open from Closed.
  6. Add your secret header.
  7. Click Create, then attach the notifier to your monitors.

If you manage notifiers through Axiom's API, put the secret in secretHeaders rather than headers. Axiom describes secretHeaders as "sensitive HTTP headers (tokens, keys) that should be masked in logs".

To route through Hookdeck, create a source with the Axiom type and use its URL as the webhook URL. Enter the header's name in API Key Header and its exact value, including any Bearer prefix, in API Key. The source accepts POST.

Securing Axiom webhooks

Read the header you chose, case-insensitively, and compare its whole value with the one you stored, in constant time. Don't strip a Bearer prefix: compare exactly what you typed into the notifier.

const crypto = require('crypto');
const express = require('express');

const HEADER = (process.env.AXIOM_WEBHOOK_HEADER || 'Authorization').toLowerCase();
const SECRET = process.env.AXIOM_WEBHOOK_SECRET; // the exact, whole header value

function safeEqual(received, expected) {
  const a = Buffer.from(received, 'utf8');
  const b = Buffer.from(expected, 'utf8');
  // timingSafeEqual throws on unequal lengths, so check first.
  return a.length === b.length && crypto.timingSafeEqual(a, b);
}

const app = express();

app.post('/webhooks/axiom', express.raw({ type: '*/*' }), (req, res) => {
  const received = req.get(HEADER);
  if (!SECRET || typeof received !== 'string' || !safeEqual(received, SECRET)) {
    return res.sendStatus(401);
  }

  let notification;
  try {
    notification = JSON.parse(req.body.toString('utf8'));
  } catch {
    return res.sendStatus(400);
  }

  // With the default template: notification.action is "Open" or "Closed".
  res.sendStatus(200);
});

The same check in Python:

import hmac
import os

HEADER = os.environ.get("AXIOM_WEBHOOK_HEADER", "Authorization")
SECRET = os.environ.get("AXIOM_WEBHOOK_SECRET", "")


def verify_axiom_request(headers) -> bool:
    if not SECRET:
        return False  # fail closed when unconfigured
    received = headers.get(HEADER)  # Starlette and Flask headers are case-insensitive
    if not received:
        return False
    # Compare bytes: compare_digest raises TypeError on a non-ASCII str.
    return hmac.compare_digest(received.encode("utf-8"), SECRET.encode("utf-8"))

A static header authenticates the sender, not the body. It can't detect a modified payload, and with no signed timestamp a captured request can be replayed and still pass. Serve the endpoint over HTTPS only, use a long random value, and replace it on the notifier and in your receiver if it might have leaked.

Make Axiom alerts production-ready. Hookdeck Event Gateway checks your secret header on every Axiom notification, queues each one durably, and lets you replay any alert after a fix.

Axiom webhook limitations and pain points

No signature

The Problem: Anyone who learns the header value can send convincing alerts to your endpoint, and a changed body goes unnoticed.

Why It Happens: Axiom computes nothing over the body. The header you add is sent unchanged on every request.

Workarounds:

  • Use a long random value, HTTPS only, and keep the header out of your logs.
  • Before taking an irreversible action, check the monitor's state in Axiom.

How Hookdeck Can Help: Hookdeck's Axiom source compares the header you name with the exact value you enter. Hookdeck answers Axiom with a 200 either way, but a request with a missing or wrong value is recorded as failed verification and never reaches your destination.

The plain webhook notifier can't be authenticated

The Problem: Requests arrive with nothing to check.

Why It Happens: Axiom's plain webhook notifier takes only a URL. Only the Custom webhook notifier sends headers.

Workarounds:

  • Recreate the notifier as a Custom webhook with a secret header, and move your monitors to it.

How Hookdeck Can Help: A Hookdeck Axiom source requires the header, so a misconfigured notifier shows up straight away as failed verifications in the request log rather than as silently trusted traffic.

Delivery behaviour is undocumented

The Problem: You can't tell from the docs whether an alert sent during an outage will arrive later.

Why It Happens: Axiom's notifier docs don't describe a timeout, retries or what counts as success.

Workarounds:

  • Respond with 200 as soon as the header checks out, and do the work afterwards.
  • Treat Axiom's monitor history as the source of truth after an outage on your side.

How Hookdeck Can Help: Hookdeck accepts each notification as soon as it arrives, stores it, and delivers it to your service with retries you configure, so a short outage on your side doesn't depend on whatever Axiom does.

The body is whatever the template says

The Problem: Someone edits the template and your parser breaks, or a value such as .Body produces JSON your parser rejects.

Why It Happens: The template is yours to change, and Axiom doesn't validate it against your receiver.

Workarounds:

  • Keep the template in version control and change it alongside your receiver.
  • Return 400 for a body that doesn't parse, and log the raw request so you can fix the template.

How Hookdeck Can Help: Hookdeck stores every request with its raw body and headers, so you can see exactly what a template rendered and replay it once your parser is fixed.

Best practices

Store the full header value

Keep the exact value, Bearer prefix included, as one string and compare it whole. Put the name and value in configuration, not code, so you can rotate them.

Route on your own field

Axiom adds no event type. Keep action from the default template, or your own equivalent, and route on it. Acknowledge anything you don't recognise.

Deduplicate on the monitor and time

There is no notification ID. Build a key from event.monitorID, action and event.timestamp, and store it before acting. See our guide to webhook idempotency.

Acknowledge fast and process asynchronously

Check the header, store the notification, return 200, and do the paging or ticketing afterwards. See why to process webhooks asynchronously.

Conclusion

Axiom sends monitor notifications through a Custom webhook notifier as a JSON POST whose body is your own Go template, authenticated only by a secret header you add. Use the Custom webhook notifier rather than the plain one, compare the whole header value in constant time, route on action or your own field, and deduplicate on the monitor ID and timestamp.

Hookdeck Event Gateway sits in front of your receiver, checks the header, and holds every alert until your service has processed it, so an outage on your side doesn't cost you a notification.

Get started with Hookdeck for free and handle Axiom webhooks reliably in minutes.


Gareth Wilson

Gareth Wilson

Product Marketing

Multi-time founding marketer, Gareth is PMM at Hookdeck and author of the newsletter, Community Inc.