Guide To CircleCI Webhooks Features And Best Practices

CircleCI is one of the most widely used continuous integration and continuous delivery (CI/CD) platforms, helping development teams automate their build, test, and deployment workflows. Beyond running pipelines, CircleCI's webhook system enables teams to integrate their CI/CD events with external services in real-time, removing the need to poll the API for status updates.

This guide covers everything you need to know about CircleCI webhooks: their features, how to configure them, best practices for production deployments, and the common pain points developers face along with solutions to address them.

What are CircleCI webhooks?

CircleCI webhooks are HTTP callbacks that deliver event notifications to external endpoints whenever key CI/CD events occur. When a workflow completes, a job finishes, or a pipeline reaches a terminal state, CircleCI sends a JSON payload containing event details to URLs you configure. This enables integration with monitoring dashboards, incident management systems, notification platforms, and any service that can receive HTTP requests.

Webhooks in CircleCI exist in two contexts:

• Outbound webhooks — Notifications sent from CircleCI to your external endpoints when workflow or job events occur. This is the primary webhook integration most teams use.
• Custom webhooks (inbound) — Webhook triggers that allow external services to trigger a CircleCI pipeline. Available for projects integrated via the GitHub App.

This guide focuses primarily on outbound webhooks, as they are the most commonly used webhook integration, with a section covering custom inbound webhooks for pipeline triggering.

CircleCI webhook features

Supported event types

CircleCI outbound webhooks can notify you of the following events:

Each webhook delivery includes the current status, along with detailed information about the project, organization, workflow, pipeline, and (for job events) the job itself.

Webhook payload structure

When CircleCI sends a webhook notification, it delivers a JSON payload with a consistent top-level structure. Here is an example workflow-completed payload:

{
  "id": "3888f21b-eaa7-38e3-8f3d-75a63bba8895",
  "type": "workflow-completed",
  "happened_at": "2021-09-01T22:49:34.317Z",
  "webhook": {
    "id": "cf8c4fdd-0587-4da1-b4ca-4846e9640af9",
    "name": "Sample Webhook"
  },
  "project": {
    "id": "84996744-a854-4f5e-aea3-04e2851dc1d2",
    "name": "webhook-service",
    "slug": "github/circleci/webhook-service"
  },
  "organization": {
    "id": "f22b6566-597d-46d5-ba74-99ef5bb3d85c",
    "name": "circleci"
  },
  "workflow": {
    "id": "fda08377-fe7e-46b1-8992-3a7aaecac9c3",
    "name": "build-test-deploy",
    "created_at": "2021-09-01T22:49:03.616Z",
    "stopped_at": "2021-09-01T22:49:34.170Z",
    "url": "https://app.circleci.com/pipelines/github/circleci/webhook-service/130/workflows/fda08377-fe7e-46b1-8992-3a7aaecac9c3",
    "status": "success"
  },
  "pipeline": {
    "id": "1285fe1d-d3a6-44fc-8886-8979558254c4",
    "number": 130,
    "created_at": "2021-09-01T22:49:03.544Z",
    "trigger": {
      "type": "webhook"
    },
    "vcs": {
      "provider_name": "github",
      "origin_repository_url": "https://github.com/circleci/webhook-service",
      "target_repository_url": "https://github.com/circleci/webhook-service",
      "revision": "1dc6aa69429bff4806ad6afe58d3d8f57e25973e",
      "commit": {
        "subject": "Description of change",
        "body": "More details about the change",
        "author": {
          "name": "Author Name",
          "email": "author.email@example.com"
        },
        "authored_at": "2021-09-01T22:48:53Z",
        "committer": {
          "name": "Committer Name",
          "email": "committer.email@example.com"
        },
        "committed_at": "2021-09-01T22:48:53Z"
      },
      "branch": "main"
    }
  }
}

Key payload fields

Payload differences by VCS integration

The payload structure varies depending on your VCS integration type:

• GitHub OAuth and Bitbucket Cloud pipelines include a vcs map with provider_name, revision, commit, branch, and repository URLs.
• GitLab, GitHub App, and Bitbucket Data Center pipelines include a trigger_parameters map instead, with provider-specific data under gitlab, git, and circleci sub-keys.

The event payloads are open maps, meaning new fields may be added without it being considered a breaking change.

Webhook headers

CircleCI includes the following HTTP headers on outbound webhook requests:

Setting up CircleCI webhooks

Via the CircleCI UI

1. In the CircleCI web app, select your organization.
2. Select Projects in the sidebar.
3. Find your project in the list, select the ellipsis (…), and select Project Settings.
4. In the sidebar, select Webhooks.
5. Click Add Webhook.
6. Fill out the webhook form:
   • Webhook name: A descriptive name for your webhook.
   • URL: The HTTPS endpoint that will receive webhooks.
   • Certificate Validation: Ensures the receiving host has a valid SSL certificate. Only uncheck for testing.
   • Secret token: Used to validate incoming data is from CircleCI.
   • Events: Select at least one event (workflow-completed and/or job-completed).
7. Optionally, click Test Ping Event to send a test payload.
8. Click Add Webhook to save.

Via the API

CircleCI provides a full Webhooks API (v2) for programmatic webhook management:

Create a webhook:

curl -X POST https://circleci.com/api/v2/webhook \
  -H "Circle-Token: $CIRCLECI_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "incident-webhook",
    "url": "https://your-endpoint.com/webhooks/circleci",
    "events": ["workflow-completed", "job-completed"],
    "verify-tls": true,
    "signing-secret": "your-signing-secret",
    "scope": {
      "id": "your-project-id",
      "type": "project"
    }
  }'

Custom webhooks (inbound)

Custom webhooks allow external services to trigger a CircleCI pipeline. This is the reverse of outbound webhooks — instead of CircleCI notifying you, an external service notifies CircleCI.

Availability: Custom webhooks are only available for projects integrated via the GitHub App. They are not available on CircleCI server or for projects using GitHub OAuth or Bitbucket Cloud.

Setting up a custom webhook trigger

1. Navigate to Project Settings → Project Setup in the CircleCI web app.
2. Ensure a GitHub App pipeline is configured for the project.
3. Select Custom webhook + at the bottom of the pipeline box.
4. Enter a descriptive name for the webhook event.
5. Enter the name of the event source (e.g., "Datadog", "DockerHub").
6. Enter the branch to use for fetching the config file.
7. Click Save to generate a webhook URL and secret.

Test the trigger with curl:

curl -X POST -H "content-type: application/json" \
  'https://internal.circleci.com/private/soc/e/<webhook-id>?secret=<your-secret>'

Best practices when working with CircleCI webhooks

Verifying HMAC signatures

When processing webhooks from CircleCI, verify the HMAC signature to ensure requests genuinely originated from your CircleCI instance.

Node.js

const crypto = require('crypto');
const express = require('express');
const app = express();

app.use(express.json({
  verify: (req, res, buf) => {
    req.rawBody = buf.toString('utf8');
  }
}));

function verifyCircleCISignature(rawBody, signatureHeader, secret) {
  // Extract v1 signature from the header
  const signatures = Object.fromEntries(
    signatureHeader.split(',').map(pair => pair.split('='))
  );
  const signature = signatures['v1'];

  const hmac = crypto.createHmac('sha256', secret);
  const digest = hmac.update(rawBody).digest('hex');

  return crypto.timingSafeEqual(
    Buffer.from(digest),
    Buffer.from(signature)
  );
}

app.post('/webhooks/circleci', (req, res) => {
  const signatureHeader = req.headers['circleci-signature'];

  if (!signatureHeader || !verifyCircleCISignature(
    req.rawBody,
    signatureHeader,
    process.env.CIRCLECI_WEBHOOK_SECRET
  )) {
    return res.status(401).send('Invalid signature');
  }

  // Process webhook
  res.status(200).send('OK');
});

Python

import hmac
import hashlib
import os
from flask import Flask, request, abort

app = Flask(__name__)

def verify_circleci_signature(payload, signature_header, secret):
    # Extract v1 signature
    signatures = dict(
        pair.split('=', 1)
        for pair in signature_header.split(',')
    )
    signature = signatures.get('v1', '')

    computed = hmac.new(
        secret.encode('utf-8'),
        payload.encode('utf-8'),
        hashlib.sha256
    ).hexdigest()

    return hmac.compare_digest(computed, signature)

@app.route('/webhooks/circleci', methods=['POST'])
def handle_circleci_webhook():
    signature_header = request.headers.get('circleci-signature', '')

    if not verify_circleci_signature(
        request.get_data(as_text=True),
        signature_header,
        os.environ['CIRCLECI_WEBHOOK_SECRET']
    ):
        abort(401)

    # Process webhook
    return 'OK', 200

Respond quickly to avoid timeouts

CircleCI has a 10-second timeout for webhook deliveries. If your endpoint does not respond within that window, CircleCI considers the delivery failed and will retry. Always acknowledge the webhook immediately and process the payload asynchronously.

Use the event ID for idempotency

Each webhook includes a top-level id field that uniquely identifies the event. Since CircleCI may send duplicate webhook requests, use this ID to implement idempotent processing:

async function processCircleCIWebhook(payload) {
  const eventId = payload.id;

  // Check if already processed
  const exists = await redis.get(`processed:${eventId}`);
  if (exists) {
    console.log(`Event ${eventId} already processed, skipping`);
    return;
  }

  // Process the event
  await handleEvent(payload);

  // Mark as processed with TTL
  await redis.setex(`processed:${eventId}`, 86400, '1'); // 24 hour TTL
}

Decouple processing with a message queue

Push webhook payloads to a message queue like RabbitMQ or Amazon SQS and process them asynchronously. This ensures your endpoint responds within CircleCI's 10-second timeout while allowing complex processing downstream:

app.post('/webhooks/circleci', async (req, res) => {
  try {
    await sqs.sendMessage({
      QueueUrl: process.env.WEBHOOK_QUEUE_URL,
      MessageBody: JSON.stringify(req.body),
      MessageGroupId: req.body.project?.id || 'default',
      MessageDeduplicationId: req.body.id
    }).promise();

    res.status(200).send('OK');
  } catch (err) {
    console.error('Failed to enqueue webhook:', err);
    res.status(500).send('Internal error');
  }
});

CircleCI webhook limitations and pain points

Tight 10-second timeout

The Problem: CircleCI webhooks have a 10-second timeout that cannot be configured. If your endpoint doesn't respond within this window, CircleCI considers the delivery failed and will retry. This is one of the shortest webhook timeouts among CI/CD platforms, leaving very little room for endpoints that need to do any meaningful processing before acknowledging.

Why It Happens: The timeout is enforced server-side by CircleCI and is not exposed as a configurable setting in the webhook configuration UI or API.

Workarounds:

• Always acknowledge webhooks immediately with a 200 response and process asynchronously.
• Use a message queue (Hookdeck, SQS, RabbitMQ, etc.) as an intermediary between your endpoint and your processing logic.
• Ensure your endpoint infrastructure has low latency (deploy geographically close to CircleCI's servers).

How Hookdeck Can Help: Hookdeck accepts webhooks from CircleCI immediately and reliably, then delivers them to your endpoint with configurable timeouts. This removes the risk of CircleCI timing out while your endpoint processes the payload, and gives you full control over delivery timing.

Duplicate webhook deliveries

The Problem: CircleCI explicitly documents that webhook requests may be duplicated. If your endpoint is slow to respond, or if there are transient network issues, CircleCI may send the same event multiple times. Without careful handling, this can result in duplicate actions such as double notifications, duplicate records, or repeated deployments.

Why It Happens: CircleCI's retry mechanism fires when it does not receive a timely 2xx response. Combined with the tight 10-second timeout, even brief latency spikes can trigger retries that result in duplicate deliveries.

Workarounds:

• Implement idempotency checks using the id field from the webhook payload.
• Track processed event IDs in a persistent store (Redis, database) with a reasonable TTL.
• Design your webhook handlers to be safe to run multiple times for the same event.

How Hookdeck Can Help: Hookdeck's deduplication feature can automatically filter duplicate webhooks based on event content or custom identifiers, ensuring your endpoint only processes each unique event once.

Limited event types

The Problem: CircleCI only supports two outbound webhook event types: workflow-completed and job-completed. There are no webhooks for pipeline-started, job-started, workflow-started, approval-pending, or other intermediate states. Teams that need real-time visibility into when builds start or when approval gates are reached must resort to API polling.

Why It Happens: CircleCI introduced webhooks with a focused scope on terminal states. While the feature has been available since mid-2021, the event catalog has not expanded significantly since launch.

Workarounds:

• Poll the CircleCI API for intermediate states if you need real-time visibility into pipeline or job starts.
• Use the workflow-completed event with the status field to differentiate between success, failure, cancellation, and other terminal states.
• Combine webhooks for terminal events with API polling for in-progress states.

How Hookdeck Can Help: Hookdeck can receive and route CircleCI webhooks alongside data from other sources. You can use Hookdeck's transformations to enrich CircleCI webhook data with additional context from API calls, giving you a more complete picture of your pipeline lifecycle.

Five-webhook limit per project

The Problem: Each CircleCI project is limited to a maximum of 5 outbound webhooks. For teams that need to notify multiple services — such as a monitoring dashboard, a Slack notification system, a deployment tracker, an incident management tool, and a data warehouse — this limit can be quickly exhausted.

Why It Happens: The limit is a platform constraint imposed by CircleCI, likely to manage infrastructure load and prevent misconfiguration.

Workarounds:

• Build a fan-out service that receives a single webhook and distributes it to multiple downstream services.
• Consolidate webhook consumers where possible to reduce the number of endpoints needed.
• Use the CircleCI API to retrieve event data as a supplement to webhooks for lower-priority integrations.

How Hookdeck Can Help: Hookdeck can receive a single webhook from CircleCI and fan it out to multiple destinations using connections and rules. This means you only use one of your five webhook slots while still delivering events to as many services as you need.

No visibility into delivery status

The Problem: CircleCI does not provide a delivery log, dashboard, or any UI visibility into whether webhook deliveries succeeded or failed. You cannot see response codes, latency, or error details from CircleCI's side. The only feedback you get is from the test ping event during initial setup.

Why It Happens: CircleCI's webhook implementation does not include a delivery tracking or logging layer that is surfaced to users.

Workarounds:

• Implement logging on your receiving endpoint to track all incoming webhook requests.
• Monitor your endpoint's health independently using uptime monitoring tools.
• Set up alerts on your server for 4xx/5xx response rates to detect issues quickly.

How Hookdeck Can Help: Hookdeck's dashboard provides complete visibility into webhook delivery status, latency, response codes, and error details. You can inspect every request and response, set up alerts for delivery failures, and debug issues without modifying your CircleCI configuration.

No manual retry or dead letter queue

The Problem: CircleCI does not provide a way to manually retry a failed webhook delivery, nor does it maintain a dead letter queue for webhooks that fail after all retries. Once retries are exhausted, the event is lost. There is no way to replay events from the CircleCI UI or API.

Why It Happens: CircleCI's webhook system handles retries automatically but does not persist failed events for later inspection or manual replay.

Workarounds:

• Ensure your endpoint has high availability to minimise missed webhooks.
• Implement your own logging of received webhooks to detect gaps.
• Use the CircleCI API to query for workflow and job data to reconstruct any events missed during endpoint outages.

How Hookdeck Can Help: Hookdeck automatically preserves all webhook events, including failed deliveries, in a browsable log. You can manually retry any individual event or bulk-replay events from a specific time range, ensuring no CI/CD events are permanently lost.

Secret token cannot be viewed after creation

The Problem: When you create a webhook and set a secret token, CircleCI does not allow you to view the secret again. You can only reset it. If you lose the secret, you must generate a new one and update all of your webhook verification logic accordingly.

Why It Happens: This is a security design decision — secrets are stored hashed and are not retrievable. While this is good security practice, it can cause operational friction, especially in team environments.

Workarounds:

• Store the secret in a secrets manager (AWS Secrets Manager, HashiCorp Vault, etc.) at creation time.
• Document the secret securely and ensure team members know where to find it.
• If the secret is lost, reset it in CircleCI and update your webhook handler with the new value.

How Hookdeck Can Help: When using Hookdeck as your webhook endpoint, you manage the CircleCI secret in one place (between CircleCI and Hookdeck). Hookdeck handles verification centrally, so you don't need to distribute the secret to multiple downstream services.

Custom webhooks limited to GitHub App

The Problem: Custom webhooks (inbound triggers) are only available for projects integrated via the CircleCI GitHub App. Projects using GitHub OAuth, Bitbucket Cloud, or GitLab integrations cannot use custom webhooks to trigger pipelines from external services.

Why It Happens: Custom webhooks are built on CircleCI's newer GitHub App pipeline architecture and have not been back-ported to the legacy OAuth integration or other VCS providers.

Workarounds:

• Use the CircleCI API (POST /api/v2/project/{project-slug}/pipeline) to trigger pipelines programmatically from external services.
• Migrate your project to the GitHub App integration if custom webhooks are a requirement.
• Build a small intermediary service that receives webhooks from external services and uses the CircleCI API to trigger pipelines.

How Hookdeck Can Help: Hookdeck's transformations can receive webhooks from any external source, transform the payload, and forward it as an API call to CircleCI's pipeline trigger API, effectively giving you inbound webhook-like functionality regardless of your VCS integration type.

Testing CircleCI webhooks

Use the built-in test ping

The webhook configuration form includes a Test Ping Event button that sends a sample payload to your endpoint. Be aware that test payloads are abbreviated and may differ from real event payloads in structure and size.

Use a request inspector

Before building your handler, inspect real CircleCI payloads using a tool like Hookdeck Console:

1. Create a temporary webhook URL.
2. Configure it as your CircleCI webhook endpoint.
3. Trigger a real build by pushing a commit.
4. Inspect the payload structure, headers, and signature.

Validate with realistic scenarios

Test your webhook integration with realistic CI/CD scenarios:

• A workflow completing successfully
• A workflow failing
• A job being cancelled
• Multiple workflows running concurrently
• Rapid successive commits triggering multiple pipelines

Test locally with a tunnel

For local development, use a tunneling tool to expose your local endpoint to CircleCI. The Hookdeck CLI provides a convenient way to receive CircleCI webhooks locally:

hookdeck listen 3000 circleci --path /webhooks/circleci

This generates a public URL that tunnels webhooks to your local server running on port 3000, with a dashboard for inspecting every request.

Conclusion

CircleCI webhooks provide a useful foundation for integrating your CI/CD pipelines with external services like dashboards, incident management, notifications, and deployment automation. The payload structure is comprehensive, with detailed project, workflow, pipeline, and VCS information that enables sophisticated event processing and routing.

However, limitations around the tight 10-second timeout, duplicate deliveries, limited event types, the five-webhook-per-project cap, and the lack of delivery visibility or replay capabilities mean production deployments require careful consideration. Implementing proper signature verification, idempotent processing, and asynchronous handling will address most common issues.

For teams with simple notification needs and reliable endpoints, CircleCI's built-in webhook system works well when paired with proper error handling. For high-volume CI/CD environments, complex multi-destination routing, or mission-critical integrations where delivery guarantees matter, webhook infrastructure like Hookdeck can address CircleCI's limitations by providing configurable timeouts, automatic deduplication, fan-out to multiple destinations, delivery monitoring, and manual replay without modifying your CircleCI configuration.