Guide to Dynatrace Webhooks: Features and Best Practices
Dynatrace webhooks carry problem notifications out of the observability platform. When Dynatrace detects a problem that affects real users, and again when the problem is resolved, a custom integration can POST a notification to your own service. Teams use this to open tickets in tools Dynatrace doesn't integrate with, page through their own on-call system, start automated remediation, or record incidents in an internal database.
This guide covers the Dynatrace Classic Custom integration for problem notifications: how to configure it, how the payload template works, how to authenticate requests with a secret header, and what to plan for because the documentation is silent on delivery behaviour.
What are Dynatrace webhooks?
Dynatrace groups related incidents into a single problem and keeps updating it while it is open. To avoid notification spam, it pushes problem notifications only when a problem is first detected and when it is finally resolved. A Custom integration sends those notifications as an HTTP POST to a URL you choose, and an alerting profile decides which problems qualify.
The body is entirely yours. You write a payload template and Dynatrace fills in placeholders such as {ProblemTitle} and {State}. There is no fixed schema and no event field unless your template includes one.
Problem notifications are a Dynatrace Classic feature. Dynatrace now recommends simple workflows for problem notifications in Latest Dynatrace; this guide covers the Classic custom webhook, which is what Hookdeck's Dynatrace source type links to.
Dynatrace doesn't sign these requests. Its Additional HTTP headers setting lets you add an Authorization header or any other header, and that secret header is how your receiver authenticates the request.
Dynatrace webhook features
| Feature | Details |
|---|---|
| Configuration | Settings Classic > Integration > Problem notifications > Add notification > Custom integration |
| Trigger | A problem is detected or resolved. Optionally, new events merge into an open problem |
| Filtering | An assigned alerting profile |
| Method | POST |
| Payload | Your template, with placeholders. JSON by default |
| Content type | JSON, unless you add a different Content-Type header such as text/plain or application/xml |
| Authentication | Additional HTTP headers you add. No signature |
| Secret webhook URL | Hides the URL in the notification settings |
| Certificates | Validated unless you turn on Accept any SSL certificate |
| Testing | Send test notification |
| Retries, timeout, source IPs | Not documented |
The payload template
Dynatrace's documented template looks like this:
{
"ImpactedEntities": {ImpactedEntities},
"ImpactedEntity": "{ImpactedEntity}",
"PID": "{PID}",
"ProblemDetailsHTML": "{ProblemDetailsHTML}",
"ProblemDetailsJSON": {ProblemDetailsJSON},
"ProblemID": "{ProblemID}",
"ProblemImpact": "{ProblemImpact}",
"ProblemTitle": "{ProblemTitle}",
"Problem URL": "https://example.com",
"State": "{State}",
"Tags": "{Tags}"
}
{ImpactedEntities} and {ProblemDetailsJSON} are JSON values, so they go in without quotes. The Available placeholders section of the configuration page lists the full set.
A test notification renders that template as:
{
"ImpactedEntities": [
{ "type": "HOST", "name": "MyHost1", "entity": "HOST-XXXXXXXXXXXXX" },
{ "type": "SERVICE", "name": "MyService1", "entity": "SERVICE-XXXXXXXXXXXXX" }
],
"ImpactedEntity": "MyHost1, MyService1",
"PID": "99999",
"ProblemDetailsHTML": "<h1>Dynatrace problem notification test run details</h1>",
"ProblemDetailsJSON": { "ID": "99999" },
"ProblemID": "999",
"ProblemImpact": "INFRASTRUCTURE",
"ProblemTitle": "Dynatrace problem notification test run",
"Problem URL": "https://example.com",
"State": "OPEN",
"Tags": "testtag1, testtag2"
}
Include {State} and {PID} or {ProblemID} in your template. Without them you can't tell an opening notification from a resolving one, or match the two up. The documentation shows only the OPEN value, so check the State a resolved problem sends before you depend on it.
By default Dynatrace validates the body as JSON and escapes placeholder values for JSON. If you add a different Content-Type header, it skips the validation and the escaping, so a quote in a problem title can break a body you still parse as JSON.
Inspect Dynatrace webhooks as they arrive. Point a custom integration at a Hookdeck Inspect URL and click Send test notification to see exactly what your template renders, with no account or setup required.
Setting up Dynatrace webhooks
Generate a long random value first, for example with openssl rand -hex 32.
- Go to Settings Classic > Integration > Problem notifications and select Add notification.
- Select Custom integration.
- Enter a Display name, and turn on Secret webhook URL if you want the URL hidden in the settings.
- Enter the Webhook URL.
- Under Additional HTTP headers, add a secret header, for example
Authorizationwith the valueBearer <your random value>. - Write the Custom payload.
- Leave Accept any SSL certificate off. Turn on Call webhook if new events merge into existing problems only if you want those extra notifications.
- Assign an Alerting profile.
- Select Send test notification, then save.
Dynatrace also lets you put an authentication token in the webhook URL's query string, but avoid it. As Hookdeck's setup note for this source says, send the secret in a custom header, not as a token in the URL: URLs end up in logs, and Hookdeck's source checks a header.
To route through Hookdeck, create a source with the Dynatrace type and use its URL in step 4. Enter the header name from step 5 in API Key Header and the exact value, including any Bearer prefix, in API Key. Hookdeck finds the header case-insensitively, compares the whole value, and accepts POST.
Securing Dynatrace webhooks
Look up the header you configured and compare its whole value with the stored secret, in constant time. Don't strip a Bearer prefix; compare exactly what you typed into Dynatrace.
const crypto = require('crypto');
const express = require('express');
const HEADER_NAME = (process.env.DYNATRACE_WEBHOOK_HEADER || 'authorization').toLowerCase();
const EXPECTED = process.env.DYNATRACE_WEBHOOK_SECRET; // e.g. "Bearer 3f9c..."
function safeEqual(received, expected) {
const a = Buffer.from(received, 'utf8');
const b = Buffer.from(expected, 'utf8');
// timingSafeEqual throws on unequal lengths, so check first.
return a.length === b.length && crypto.timingSafeEqual(a, b);
}
const app = express();
app.post('/webhooks/dynatrace', express.raw({ type: '*/*' }), (req, res) => {
const received = req.get(HEADER_NAME);
if (!EXPECTED || typeof received !== 'string' || !safeEqual(received, EXPECTED)) {
return res.sendStatus(401);
}
// The body is whatever your template renders. Parse it to suit your Content-Type.
let notification;
try {
notification = JSON.parse(req.body.toString('utf8'));
} catch {
return res.sendStatus(400);
}
// Queue the notification, keyed on notification.PID and notification.State.
res.sendStatus(200);
});
The same check in Python:
import hmac
import os
HEADER_NAME = os.environ.get("DYNATRACE_WEBHOOK_HEADER", "Authorization")
EXPECTED = os.environ.get("DYNATRACE_WEBHOOK_SECRET", "")
def verify_dynatrace_request(headers) -> bool:
if not EXPECTED:
return False # never accept when unconfigured
received = headers.get(HEADER_NAME) # most frameworks look headers up case-insensitively
if not received:
return False
# Compare bytes: compare_digest raises TypeError on a non-ASCII str.
return hmac.compare_digest(received.encode("utf-8"), EXPECTED.encode("utf-8"))
A secret header authenticates the sender, not the body. It cannot show that a notification was not modified, and with no signature or timestamp a captured request can be replayed and still pass. Use HTTPS, keep Accept any SSL certificate off, use a long random value, and change it in the integration settings if it might have leaked.
Make Dynatrace webhooks production-ready. Hookdeck Event Gateway checks your secret header, accepts every problem notification straight away, and keeps it until your incident tooling has processed it.
Dynatrace webhook limitations and pain points
No documented retries
The Problem: Your endpoint is down when a problem opens, and you never hear about it.
Why It Happens: The webhook integration page doesn't describe retries, timeouts or what happens to a notification your endpoint rejects. Dynatrace also sends only two notifications per problem, at detection and at resolution, so each one matters.
Workarounds:
- Keep the receiving endpoint simple and highly available, separate from the code that opens tickets.
- Reconcile open problems against the Dynatrace API periodically for anything critical.
How Hookdeck Can Help: Hookdeck accepts and stores each notification as it arrives, then delivers it to your service with retries on your schedule, so a short outage on your side no longer loses a problem. You can replay any notification after a fix.
The payload is whatever you wrote
The Problem: A template change breaks the receiver, or a resolved problem can't be matched to the notification that opened it.
Why It Happens: Dynatrace sends your template with placeholders filled in. There is no versioned schema, and anyone who edits the integration can change the body.
Workarounds:
- Keep the template in version control next to the receiver code.
- Always include
{PID}or{ProblemID}and{State}. - Use the default JSON content type unless your target needs something else, so Dynatrace validates and escapes the body.
How Hookdeck Can Help: Every notification is stored with its exact body in Hookdeck, so a broken template is easy to spot, and a transformation can normalise the body before it reaches your service.
Secrets in URLs leak
The Problem: A token in the webhook URL shows up in proxy logs, error messages and anywhere the URL is copied.
Why It Happens: Dynatrace offers a URL token as one way to authenticate, alongside custom headers.
Workarounds:
- Put the secret in an Additional HTTP headers entry instead.
- Turn on Secret webhook URL so the URL isn't displayed in the settings.
How Hookdeck Can Help: Hookdeck's Dynatrace source checks the header name and value you configure. Hookdeck answers Dynatrace with a 200 either way, but a request without the right header is recorded as failed verification and never reaches your service.
Merged events add noise
The Problem: One incident produces a stream of notifications.
Why It Happens: Call webhook if new events merge into existing problems sends an extra notification each time new events join an open problem.
Workarounds:
- Leave the option off unless you need those updates.
- Deduplicate on the problem identifier and state in your receiver.
How Hookdeck Can Help: Filters can drop notifications you don't need, and deduplication can discard repeats before they reach your service.
Best practices
Authenticate with a header, never the URL
Add a long random value as an Additional HTTP header and compare it whole, in constant time.
Put identifiers in the template
Include {PID} or {ProblemID}, {State} and {ProblemTitle} so every notification can be matched, routed and deduplicated. See our guide to webhook idempotency.
Keep JSON as the content type
Dynatrace validates and escapes JSON bodies by default. Switch only when the target needs a different format.
Test every template change
Use Send test notification after each edit, and check what your receiver logged.
Acknowledge fast and process asynchronously
Store the notification and respond, then open tickets or run remediation in the background. See why to process webhooks asynchronously.
Conclusion
A Dynatrace Classic custom integration sends a problem notification as a POST built from your own template, with no signature. Authenticate it with a secret header rather than a URL token, include the problem ID and state in the template, compare the header in constant time, and plan for undocumented retry behaviour by keeping the receiver simple and available.
Hookdeck Event Gateway checks the header, stores every notification as it arrives, and delivers it to your incident tooling with retries and replay.
Get started with Hookdeck for free and handle Dynatrace webhooks reliably in minutes.