Gareth Wilson Gareth Wilson

Guide to Flux CD Webhooks: Features and Best Practices

Published


Flux CD keeps Kubernetes clusters in sync with what is in Git. Its controllers pull manifests and Helm charts, apply them, and report how each reconciliation went: a Kustomization applied a new revision, a HelmRelease upgrade failed, a GitRepository couldn't be fetched. Flux's notification-controller can send those reports anywhere, and its generic-hmac provider sends them as signed webhooks to an HTTP endpoint of your choice. Platform teams use that to record deploys, update a release dashboard or open an incident when a reconciliation fails.

This guide covers how Flux's Provider and Alert resources produce webhooks, what a Flux Event looks like, how to verify X-Signature, and the gaps in Flux's delivery guarantees.

What are Flux CD webhooks?

Flux has no dashboard and no hosted service. Webhooks are configured with two Kubernetes resources in the cluster:

  • A Provider says where to send events and how. For webhooks, .spec.type is generic or generic-hmac, .spec.address is the URL, and .spec.secretRef points at a Secret.
  • An Alert says which events to send: which Flux objects to watch and whether to include info events or only error ones. It references the Provider.

Each event the Alert selects becomes one POST with a JSON body. With generic-hmac, Flux adds X-Signature: sha256=<hex>, an HMAC-SHA256 of the body keyed with the token from the Secret. Plain generic sends no signature. Because Flux runs inside your cluster, every cluster is a separate sender with its own secret.

Flux CD webhook features

FeatureDetails
ConfigurationKubernetes resources: Provider (type generic-hmac) and Alert
FilteringThe Alert selects event sources and eventSeverity (info or error)
Signaturehex HMAC-SHA256
Signature headerx-signature, as sha256=<hex>
Signed contentThe raw request body
SecretThe token key of the Secret in .spec.secretRef, chosen by you
Other headersGotk-Component (the reporting controller), plus custom headers from the Secret
BodyJSON
Event routingNo event type field. Route on involvedObject.kind, reason and severity
TimeoutConfigurable on the Provider
RetriesNot documented for generic webhooks
Source IPsYour cluster's egress

Events and payloads

severityTriggered WhenExample reason
errorA Flux object fails to reconcile or validateValidationFailed
infoA Flux object makes normal progressReconciliationSucceeded

severity is what an Alert filters on (eventSeverity: info or error). reason is controller-specific; ValidationFailed comes from the Flux docs, the other values are examples of what controllers report, so treat reason as an open set.

Event Payload Structure

FieldDescription
involvedObjectMetadata of the object that raised the event: apiVersion, kind, name, namespace, uid
severityinfo or error
reasonShort machine-readable cause, e.g. ValidationFailed
messageHuman-readable detail
metadataExtra key/values, e.g. kustomize.toolkit.fluxcd.io/revision
reportingControllere.g. kustomize-controller
reportingInstancePod name of the reporting controller
timestampRFC 3339 time of the event

The request also carries a Gotk-Component header naming the sending controller.

There is no list of event types to subscribe to. A Flux event is a Kubernetes event from one of its controllers, and reason is whatever that controller reported. For routing, involvedObject.kind tells you whether it concerns a Kustomization, a HelmRelease or a source, severity whether it is a problem, and metadata often carries the Git revision that was applied.

Published payload example

This is the generic-webhook sample from the webhook samples library. The name comes from the section of Flux's docs it appears in, not from a field on the wire.

{
  "involvedObject": {
    "apiVersion": "kustomize.toolkit.fluxcd.io/v1",
    "kind": "Kustomization",
    "name": "webapp",
    "namespace": "apps",
    "uid": "7d0cdc51-ddcf-4743-b223-83ca5c699632"
  },
  "metadata": {
    "kustomize.toolkit.fluxcd.io/revision": "main/731f7eaddfb6af01cb2173e18f0f75b0ba780ef1"
  },
  "severity": "error",
  "reason": "ValidationFailed",
  "message": "service/apps/webapp validation error: spec.type: Unsupported value: Ingress",
  "reportingController": "kustomize-controller",
  "reportingInstance": "kustomize-controller-7c7b47f5f-8bhrp",
  "timestamp": "2022-10-28T07:26:19Z"
}

Setting up Flux CD webhooks

Use a Flux Provider of type generic-hmac and paste the token from its Secret. Generic providers are unsigned and fail verification.

You need a cluster running Flux with the notification-controller and kubectl access to create a Secret, a Provider and an Alert.

Create the Secret

You choose the signing secret and store it under the token key of a Kubernetes Secret:

apiVersion: v1
kind: Secret
metadata:
  name: webhook-token
  namespace: default
stringData:
  token: <a long random string>

That token value is your WEBHOOK_SECRET. Use it as UTF-8 text; do not hex- or base64-decode it. The same Secret can also hold a headers key if you want Flux to send extra headers.

Create the Provider and Alert

apiVersion: notification.toolkit.fluxcd.io/v1
kind: Provider
metadata:
  name: webhook
  namespace: default
spec:
  type: generic-hmac
  address: https://example.com/webhooks/flux-cd
  secretRef:
    name: webhook-token
---
apiVersion: notification.toolkit.fluxcd.io/v1
kind: Alert
metadata:
  name: webhook
  namespace: default
spec:
  providerRef:
    name: webhook
  eventSeverity: info   # info sends info and error; error sends errors only
  eventSources:
    - kind: Kustomization
      name: '*'
    - kind: HelmRelease
      name: '*'

Apply with kubectl apply -f. Each event the Alert selects becomes one POST to address with an X-Signature: sha256=<hex> header. Start with eventSeverity: error if you only want failures; info adds every successful reconciliation, which can be a lot of traffic on a busy cluster.

To watch events arrive before you point Flux at production, run the Hookdeck CLI and use its URL as address:

npx hookdeck-cli listen 3000 flux-cd --path /webhooks/flux-cd

Securing Flux CD webhooks

Strip sha256= from X-Signature, compute an HMAC-SHA256 of the raw body with the token as UTF-8 bytes, encode it as hex and compare in constant time. See the verification documentation.

const crypto = require('node:crypto');

function verifySignature(rawBody, headers, secret) {
  if (!Buffer.isBuffer(rawBody) || typeof secret !== 'string' || !secret) return false;
  const signature = headers['x-signature'];
  if (typeof signature !== 'string') return false;
  const key = Buffer.from(secret, 'utf8');
  const message = rawBody;
  const candidates = [signature.startsWith('sha256=') ? signature.slice(7) : signature];
  const expected = crypto.createHmac('sha256', key).update(message).digest();
  return candidates.some((candidate) => {
    if (!/^[0-9a-fA-F]{64}$/.test(candidate)) return false;
    const received = Buffer.from(candidate, 'hex');
    return received.length === expected.length && crypto.timingSafeEqual(received, expected);
  });
}

The handler records every event against the cluster and Flux object it came from, and raises an incident for error events.

const express = require('express');

function createFluxApp({ recordDeploy, openIncident }) {
  const app = express();
  app.post('/webhooks/flux-cd', express.raw({ type: '*/*' }), async (req, res) => {
    const secret = process.env.WEBHOOK_SECRET;
    if (!secret) return res.sendStatus(500);
    if (!verifySignature(req.body, req.headers, secret)) return res.sendStatus(401);

    let event;
    try {
      event = JSON.parse(req.body.toString('utf8'));
    } catch {
      return res.sendStatus(400);
    }
    if (!event || typeof event !== 'object' || Array.isArray(event)) return res.sendStatus(400);
    if (
      !event.involvedObject ||
      typeof event.involvedObject !== 'object' ||
      typeof event.reason !== 'string'
    ) {
      return res.sendStatus(400);
    }
    const { kind, namespace, name } = event.involvedObject || {};
    const target = `${kind}/${namespace}/${name}`;
    try {
      await recordDeploy(target, event.reason, event.metadata, event.timestamp, req.get('gotk-component'));
      if (event.severity === 'error') await openIncident(target, event.reason, event.message);
    } catch {
      return res.sendStatus(503);
    }
    return res.sendStatus(200);
  });
  return app;
}

Flux signs only the body. The event's timestamp is inside it, but there's no signed header to check, so repeated events are handled by deduplication rather than by a time window.

Hookdeck's Flux CD source type reads the sha256= value from X-Signature and verifies a hex HMAC-SHA256 of the raw body with your token. If you use a plain generic Provider, leave verification off on the source. Hookdeck answers Flux with 200 whatever the result, records a failed check as failed verification and does not forward that request. It has no timestamp tolerance to configure.

Make Flux CD webhooks production-ready. Hookdeck Event Gateway collects signed events from every cluster, verifies X-Signature, and keeps each reconciliation event so your deploy history has no gaps.

Flux CD webhook limitations and pain points

No documented retries

The Problem: A deploy record or failure alert never arrives because your endpoint was briefly unavailable.

Why It Happens: Flux's provider documentation describes a timeout setting but no retry policy or acknowledgement rule for generic webhooks. Assume an event that fails to deliver is gone.

Workarounds:

  • Keep the receiver highly available and quick to answer.
  • Use Flux's own status on Kustomizations and HelmReleases, via flux get or Kubernetes events, as the source of truth.

How Hookdeck Can Help: Point the Provider at a Hookdeck source. Hookdeck accepts each event and stores it, then retries your service as needed, so your downtime doesn't lose Flux events.

generic sends nothing to verify

The Problem: Anyone can post fake deploy events to your endpoint.

Why It Happens: Only the generic-hmac Provider type signs. A generic Provider sends the same body with no X-Signature.

Workarounds:

  • Use generic-hmac for anything reachable from outside the cluster.
  • If you must use generic, add a shared secret header through the Secret's headers key and check it.

How Hookdeck Can Help: Hookdeck's Flux CD source verifies generic-hmac signatures and logs any delivery that fails, which shows up a Provider still set to generic.

The hash function may change

The Problem: A Flux upgrade could break verification.

Why It Happens: Flux says the hash function "currently defaults to sha256, which may change in the future", and its Go verification sample accepts sha224, sha256, sha384 and sha512 prefixes. Receivers that only handle sha256= would reject a different default. Hookdeck's Flux CD source type also verifies only sha256= today.

Workarounds:

  • Watch Flux release notes for changes to notification signing.
  • Test signature verification after each Flux upgrade.

How Hookdeck Can Help: Hookdeck's request log shows the exact X-Signature each cluster sends, so a change in prefix is easy to spot.

No event vocabulary

The Problem: You can't write a complete handler for "every Flux event".

Why It Happens: reason is an open set of Kubernetes event reasons from each controller. Flux doesn't publish a list.

Workarounds:

  • Route on severity and involvedObject.kind, and only special-case the reasons you know.
  • Log unfamiliar reasons for review.

How Hookdeck Can Help: Hookdeck filters can match on severity and involvedObject.kind, so error events can go to on-call while info events go to a deploy log.

Best practices

One secret per cluster

Give each cluster its own token and its own Hookdeck source or endpoint path. A leaked token then affects one cluster, and you always know which cluster sent an event.

Filter in the Alert

The Alert's eventSeverity and event sources are your only filters on Flux's side. Narrow them so only events you act on leave the cluster.

Use the revision in metadata

Keys such as kustomize.toolkit.fluxcd.io/revision tell you which Git revision was applied. Store them with the event for deploy history.

Verify the raw body

Hash the bytes exactly as received: express.raw({ type: '*/*' }), await request.arrayBuffer() or await request.body(). Check for 64 hex characters before decoding.

Deduplicate

Without an event ID, build a key from involvedObject.uid, reason, timestamp and the revision. The webhook idempotency guide covers storing it.

Conclusion

Flux's webhooks are Kubernetes events in JSON, sent from inside your cluster by an Alert and a Provider you define. Use generic-hmac, keep one token per cluster, route on severity and object kind, and don't count on Flux to retry.

Hookdeck Event Gateway gives every cluster one durable endpoint that verifies its signature and holds each event until your systems have recorded it.

Get started with Hookdeck for free and handle Flux CD webhooks reliably in minutes.


Gareth Wilson

Gareth Wilson

Product Marketing

Multi-time founding marketer, Gareth is PMM at Hookdeck and author of the newsletter, Community Inc.