Gareth Wilson Gareth Wilson

Guide to HCP Terraform Webhooks: Features and Best Practices

Published


HCP Terraform webhooks tell your systems what is happening to your infrastructure runs: a run is created, needs approval, starts applying, completes or errors, drift is detected, a health assessment fails, or a workspace is about to be destroyed. If you run Terraform in HCP Terraform (formerly Terraform Cloud) or Terraform Enterprise, these notifications let you react to runs, for example by paging someone when a run fails, without polling the runs API.

This guide covers HCP Terraform's generic notification configurations, the "Webhook" destination: how they are created, verified and signed, the triggers and their payloads, and the best practices for running a receiver in production. The Slack, Microsoft Teams and email destinations send different payloads and are not covered here.

What are HCP Terraform webhooks?

HCP Terraform calls its webhooks notification configurations. Each one has a destination type, a URL, an optional token and a list of triggers. With the generic destination type, HCP Terraform sends an HTTP POST with a JSON body to your URL whenever one of the selected triggers fires.

Configurations exist at three levels. A workspace can have up to 20 of its own. A project-level configuration applies to every workspace in the project. A team-level configuration notifies a team about change requests on workspaces it can access.

The token is what makes a notification verifiable. HCP Terraform uses it as the key for an HMAC-SHA512 of the request body and sends the result in X-TFE-Notification-Signature. The token is optional, and a configuration without one sends no signature at all.

HCP Terraform webhook features

FeatureDetails
ConfigurationWorkspace Settings > Notifications, or POST /workspaces/:workspace_id/notification-configurations
ScopeWorkspace (up to 20 per workspace), project and team configurations
Destination typegeneric ("Webhook" in the UI). The URL may be HTTP or HTTPS
Signature headerX-TFE-Notification-Signature, only when a token is set
SchemeHMAC-SHA512, lowercase hex
Signed contentThe request body, nothing else
SecretThe configuration's token: chosen by you, write-only, not viewable after saving
Replay windowNone. No timestamp is sent or signed
Enable-time checkA verification notification that must receive a 2xx
Delivery logEach response is kept in the configuration's delivery-responses
Retries and timeoutNot documented on the notification pages
Payload versionspayload_version 1 for run notifications, "2" for workspace and assessment notifications

Common events

HCP Terraform calls the events triggers. Workspace and project configurations can subscribe to these:

TriggerUI nameFires when
run:createdCreatedA run is created and enters the Pending stage
run:planningPlanningA run acquires the lock and starts to execute
run:needs_attentionNeeds AttentionA plan has changes and needs user input, such as plan approval or a policy override
run:applyingApplyingA run enters the Apply stage
run:completedCompletedA run completes successfully
run:erroredErroredA run ends early because of an error or a cancellation
assessment:driftedDrift detectedDrift is detected for the first time, or previously detected drift changes
assessment:check_failureCheck failedOne or more continuous validation checks fail
assessment:failedHealth assessment erroredA health assessment fails to run
workspace:auto_destroy_reminderAuto destroy reminder12 and 24 hours before a scheduled auto destroy run
workspace:auto_destroy_run_resultsAuto destroy resultsAn auto destroy run was attempted, successful or not
workspace:deletedWorkspace DeletedA workspace is deleted

The assessment:* triggers only fire when health assessments are enabled for the workspace. Team configurations have one trigger, change_request:created, sent when someone creates a change request for a workspace the team can access. Speculative plans, and workspaces using the Local execution mode, do not send notifications.

The payload depends on the kind of trigger. Run notifications use payload_version 1, with run details at the top level and the trigger inside a notifications array. Here is HashiCorp's run:errored example:

{
  "payload_version": 1,
  "notification_configuration_id": "nc-AeUQ2zfKZzW9TiGZ",
  "run_url": "https://app.terraform.io/app/acme-org/my-workspace/runs/run-FwnENkvDnrpyFC7M",
  "run_id": "run-FwnENkvDnrpyFC7M",
  "run_message": "Add five new queue workers",
  "run_created_at": "2019-01-25T18:34:00.000Z",
  "run_created_by": "sample-user",
  "workspace_id": "ws-XdeUVMWShTesDMME",
  "workspace_name": "my-workspace",
  "organization_name": "acme-org",
  "notifications": [
    {
      "message": "Run Canceled",
      "trigger": "run:errored",
      "run_status": "canceled",
      "run_updated_at": "2019-01-25T18:37:04.000Z",
      "run_updated_by": "sample-user"
    }
  ]
}

HashiCorp says the notifications array "is always one event" at present, but that it may roll several notifications for a run into one request in the future.

Assessment and workspace notifications use payload_version "2", written as a string, with the trigger at the top level beside trigger_scope, a message and a details object. Here is assessment:drifted, trimmed:

{
  "payload_version": "2",
  "notification_configuration_id": "nc-SZ3V3cLFxK6sqLKn",
  "notification_configuration_url": "https://app.terraform.io/api/v2/notification-configurations/nc-SZ3V3cLFxK6sqLKn",
  "trigger_scope": "assessment",
  "trigger": "assessment:drifted",
  "message": "Drift Detected",
  "details": {
    "new_assessment_result": {
      "id": "asmtres-vRVQxpqq64EA9V5a",
      "drifted": true,
      "resources_drifted": 4,
      "resources_undrifted": 55
    },
    "workspace_id": "ws-XdeUVMWShTesDMME",
    "workspace_name": "my-workspace",
    "organization_name": "acme-org"
  }
}

Change request notifications have a third shape, with change_request_* and workspace_* fields and no trigger field. Since change_request:created is the only team trigger, the shape itself tells you what it is.

Setting up HCP Terraform webhooks

Choose a token first. It is an arbitrary secret string, and you cannot view it again after saving, so generate it and store it server-side (for example as TERRAFORM_NOTIFICATION_TOKEN) before you create the configuration.

  1. In HCP Terraform or Terraform Enterprise, open the workspace and go to Settings > Notifications.
  2. Click Create a Notification.
  3. Set Destination to Webhook, enter a Name, your Webhook URL and the Token.
  4. Choose the Workspace Events and Run Events to send, either all or specific ones.
  5. Click Create a notification.

Through the API, send POST /workspaces/:workspace_id/notification-configurations with destination-type "generic", name, url, token, enabled and a triggers array such as ["run:completed", "run:errored"]. You need admin access to the workspace. To cover every workspace in a project, create the configuration at the project level instead.

When you save a configuration with enabled set to true, toggle it on, or call POST /notification-configurations/:id/actions/verify, HCP Terraform sends a verification notification to the URL. Its body is a run-shaped payload whose run fields are null and whose trigger is verification. Your endpoint must answer with a 2xx, or HCP Terraform will not enable the configuration and shows the error instead. The Last Response box on the configuration shows what your endpoint returned, and Send a Test sends another verification notification.

Make sure your endpoint is running and holds the token before you enable the configuration. HashiCorp says it signs "all notification requests" when a token is set, but does not say anything specific about the verification request, so check Last Response if enabling fails.

Securing HCP Terraform webhooks

Compute an HMAC-SHA512 of the raw request body with the token as the key, encode it as hex, and compare it with X-TFE-Notification-Signature in constant time. HashiCorp's Ruby sample compares with !=; use a constant-time comparison instead.

const crypto = require('crypto');
const express = require('express');

function verifyTerraformSignature(rawBody, signatureHeader, token) {
  if (!signatureHeader || !token) return false;
  const expected = crypto.createHmac('sha512', token).update(rawBody).digest('hex');
  const received = Buffer.from(signatureHeader.trim().toLowerCase(), 'utf8');
  const computed = Buffer.from(expected, 'utf8');
  return received.length === computed.length && crypto.timingSafeEqual(received, computed);
}

const app = express();

app.post('/webhooks/terraform', express.raw({ type: '*/*' }), (req, res) => {
  const token = process.env.TERRAFORM_NOTIFICATION_TOKEN;
  if (!token) return res.sendStatus(500); // fail closed

  if (!verifyTerraformSignature(req.body, req.get('x-tfe-notification-signature'), token)) {
    return res.sendStatus(401);
  }

  const payload = JSON.parse(req.body.toString('utf8'));
  // Run notifications carry the trigger in notifications[]; v2 payloads carry it at the top level.
  const triggers = Array.isArray(payload.notifications)
    ? payload.notifications.map((n) => n.trigger)
    : [payload.trigger ?? 'change_request:created'];
  // "verification" is the enable-time test; acknowledge it and do nothing else.
  res.sendStatus(200);
});

The same check in Python:

import hashlib
import hmac


def verify_terraform_signature(raw_body: bytes, signature_header, token: str) -> bool:
    if not signature_header or not token:
        return False
    expected = hmac.new(token.encode("utf-8"), raw_body, hashlib.sha512).hexdigest()
    received = signature_header.strip().lower().encode("utf-8")
    return hmac.compare_digest(received, expected.encode("utf-8"))


# In FastAPI: raw_body = await request.body()
# verify_terraform_signature(raw_body, request.headers.get("x-tfe-notification-signature"),
#                            os.environ["TERRAFORM_NOTIFICATION_TOKEN"])

The digest is 128 hex characters. A 64-character value in your logs means something is computing SHA-256 rather than SHA-512.

Make HCP Terraform webhooks production-ready. Hookdeck Event Gateway verifies the X-TFE-Notification-Signature header against your notification token, and durably queues every run, drift and workspace notification before your service sees it.

HCP Terraform webhook limitations and pain points

No token, no signature

The Problem: Notifications arrive without X-TFE-Notification-Signature, so there is nothing to verify, and anyone who learns the URL can send fake run results.

Why It Happens: The token is optional and off by default. Only configurations with a token are signed. The URL can also be plain HTTP.

Workarounds:

  • Always set a token, and reject requests without the header.
  • Use an HTTPS URL.
  • Store the token before saving, because you cannot read it back afterwards. To rotate it, update the configuration and your receiver together.

How Hookdeck Can Help: Set the same token on your Hookdeck HCP Terraform source and Hookdeck verifies every notification before it reaches your code. A configuration with no token sends unsigned notifications, which fail verification on a source that expects a signature, so set a token on the configuration first.

Enabling depends on your endpoint

The Problem: You cannot enable the configuration, and the error is not obvious.

Why It Happens: HCP Terraform sends a verification notification when you enable a configuration and refuses to enable it unless it gets a 2xx. A receiver that is not deployed yet, rejects the request, or answers slowly fails that check.

Workarounds:

  • Deploy the receiver, with the token, before enabling the configuration.
  • Recognise verification and answer it with a 2xx without side effects.
  • Read Last Response on the configuration, or the delivery-responses array through the API, to see exactly what your endpoint returned.

How Hookdeck Can Help: A Hookdeck source answers the verification request even while your own service is being deployed, and the request is stored for you to inspect.

Three payload shapes

The Problem: Your router misses drift alerts and change requests.

Why It Happens: Run notifications put the trigger in notifications[].trigger. Assessment and workspace notifications put it in a top-level trigger, and write payload_version as a string. Change request notifications carry no trigger at all.

Workarounds:

  • Read notifications[*].trigger, then trigger, and treat a body with change_request_* fields as change_request:created.
  • Loop over notifications rather than reading only the first element, since HashiCorp may roll several into one request.
  • Compare payload_version loosely, or as a string.

How Hookdeck Can Help: Hookdeck filters can match on either notifications[].trigger or the top-level trigger, so you can route run results and drift alerts to different destinations.

The auto destroy results trigger has two spellings

The Problem: A handler keyed on workspace:auto_destroy_run_results never fires.

Why It Happens: The trigger tables, which define the subscription values, say workspace:auto_destroy_run_results. Both sample payloads for that event carry "trigger": "workspace:auto_destroy_results". The docs do not settle which one is sent.

Workarounds:

  • Subscribe with the documented value, and accept both spellings in your handler.
  • Check the first real delivery and adjust.

How Hookdeck Can Help: A Hookdeck filter can match either spelling, so routing works whichever one arrives.

Delivery guarantees are not documented

The Problem: You do not know whether a notification missed during an outage will be resent.

Why It Happens: The notification pages describe the delivery log but not a retry policy, a timeout or source IP ranges. No timestamp is signed either, so there is no replay window.

Workarounds:

  • Respond quickly and do the work in the background.
  • Treat notifications as prompts: for anything that matters, read the run or assessment from the API using the IDs in the payload.
  • Make handlers idempotent, keyed on run_id and the trigger, or on the assessment result id.

How Hookdeck Can Help: Hookdeck acknowledges HCP Terraform immediately and stores each notification, then retries towards your service on your schedule, so a short outage on your side does not depend on whatever HCP Terraform does with failures.

Best practices

Verify against the raw body

The signature covers the exact bytes HCP Terraform sent. In Express, mount express.raw() on the webhook route and parse only after verification. In Next.js, use await request.text(). In FastAPI, read await request.body().

Subscribe to the triggers you act on

Every run can produce up to six run notifications. If you only page on failures, subscribe to run:errored and run:needs_attention rather than everything.

Use project-level configurations for consistency

When every workspace in a project should report to the same place, configure the notification once on the project. It applies to new workspaces automatically, and keeps you clear of the 20-per-workspace limit.

Acknowledge fast and process asynchronously

Return a 2xx as soon as the signature checks out, and do the follow-up work, such as triggering a pipeline or opening an incident, from a queue. See why to process webhooks asynchronously and our guide to webhook idempotency.

Conclusion

HCP Terraform signs generic notifications with a hex HMAC-SHA512 of the raw body in X-TFE-Notification-Signature, but only when the configuration has a token, so always set one. Have your receiver running before you enable a configuration, read the trigger from whichever place the payload version puts it, and accept both spellings of the auto destroy results trigger.

Hookdeck Event Gateway sits in front of your receiver, verifies every HCP Terraform notification, and holds each one until your service has processed it.

Get started with Hookdeck for free and handle HCP Terraform webhooks reliably in minutes.


Gareth Wilson

Gareth Wilson

Product Marketing

Multi-time founding marketer, Gareth is PMM at Hookdeck and author of the newsletter, Community Inc.