Publish events to an Apache Kafka topic. Authentication uses SASL (PLAIN, SCRAM-SHA-256, or SCRAM-SHA-512) and TLS is enabled by default.

Creating a Kafka Destination

curl 'https://api.outpost.hookdeck.com/2025-07-01/tenants/<TENANT_ID>/destinations' \
--header 'Content-Type: application/json' \
--header 'Authorization: Bearer <API_KEY>' \
--data '{
  "type": "kafka",
  "topics": ["orders"],
  "config": {
    "brokers": "broker1.example.com:9092,broker2.example.com:9092",
    "topic": "events",
    "sasl_mechanism": "scram-sha-256"
  },
  "credentials": {
    "username": "outpost",
    "password": "<SASL_PASSWORD>"
  }
}'

Configuration

Config

Field	Type	Required	Description
config.brokers	string	Yes	Comma-separated list of Kafka broker addresses (e.g., broker1:9092,broker2:9092)
config.topic	string	Yes	The Kafka topic to publish messages to
config.sasl_mechanism	string	Yes	SASL mechanism: plain, scram-sha-256, or scram-sha-512
config.tls	string	No	Enable TLS: true or false (default: true)
config.partition_key_template	string	No	JMESPath expression for the message partition key (default: the event ID)

Credentials

Field	Type	Required	Description
credentials.username	string	Yes	SASL username
credentials.password	string	Yes	SASL password

Message Format

Events are published as Kafka messages:

• Key: The partition key — evaluated from partition_key_template, falling back to the event ID
• Value: The event's data field as raw JSON (no wrapper)
• Headers: content-type: application/json plus system metadata (event-id, topic, timestamp) and any event metadata from the published event

Messages are routed using a hash balancer over the key, so events with the same key land on the same partition.

Example

Publishing this event:

{
  "topic": "orders",
  "data": { "order_id": "123", "status": "created" },
  "metadata": { "source": "checkout-service" }
}

Results in:

Key: evt_123

Value:

{"order_id": "123", "status": "created"}

Headers:

Header	Value
content-type	application/json
event-id	evt_123
topic	orders
timestamp	1704067200
source	checkout-service

Partition Key

By default, the event ID is used as the partition key. Customize it with a JMESPath expression evaluated against the event payload (metadata and data):

{
  "config": {
    "brokers": "broker1.example.com:9092",
    "topic": "events",
    "sasl_mechanism": "scram-sha-256",
    "partition_key_template": "data.customer_id"
  }
}

If the template is empty, fails to evaluate, or returns an empty value, the event ID is used as the fallback key.

SASL Authentication

Outpost supports three SASL mechanisms, set via config.sasl_mechanism:

• plain — SASL/PLAIN
• scram-sha-256 — SASL/SCRAM with SHA-256
• scram-sha-512 — SASL/SCRAM with SHA-512

The username and password credentials are used for all mechanisms.

TLS

TLS is enabled by default. Set config.tls to "false" to connect without TLS (for example, against a local broker):

{
  "type": "kafka",
  "topics": ["orders"],
  "config": {
    "brokers": "localhost:9092",
    "topic": "events",
    "sasl_mechanism": "plain",
    "tls": "false"
  },
  "credentials": {
    "username": "admin",
    "password": "admin-secret"
  }
}