Webhooks aren't always JSON: Binary delivery and Node.js transforms in Event Gateway

Not every webhook is JSON. Text-to-speech APIs deliver audio/mpeg, form tools send multipart/form-data uploads, analytics pipelines send gzip-compressed batches, and document services send PDFs.

The Event Gateway now handles all of these, and transformations can open them up or create new files from JSON:

• Receive files byte for byte. Audio, video, images, PDFs, archives, binary serialization formats and multipart uploads reach your destination exactly as they were sent.
• Run Node.js in transformations. Import built-in modules such as zlib, crypto and buffer, write ES modules and async handlers, and bundle npm packages when you need a library.
• Return any content type. A transformation can turn a JSON event into a PNG, a WAV, a PDF, an SVG or CSV, and the destination receives it with that Content-Type.

We built three demos to show you what that looks like: Deepgram text-to-speech callbacks, a test of 30 content types, and six transformations that generate files from JSON.

Receive audio, images, archives and multipart uploads

A Deepgram text-to-speech request with a callback parameter returns a request_id straight away. When the audio is ready, Deepgram POSTs it to the callback URL. Point that URL at a Hookdeck source, and the MP3 reaches your server with its original bytes, Content-Type and Content-Length, including on retries and replays.

The Deepgram demo runs both directions through the Event Gateway: speech-to-text callbacks arrive as JSON, and text-to-speech callbacks arrive as audio, both delivered to a local server with hookdeck listen.

Content types delivered as bytes

Requests are delivered as bytes when their Content-Type is one of:

• application/octet-stream, application/pdf, and any image/* type
• Audio and video: audio/mpeg, audio/wav, audio/ogg, audio/webm, audio/mp4, video/mp4, video/webm
• Archives: application/zip, application/gzip, application/x-tar, and any body sent with Content-Encoding: gzip
• Serialization formats: application/protobuf, application/x-protobuf, application/cbor, application/msgpack, application/vnd.apache.avro+binary
• application/wasm and font/woff2

JSON, XML, form-encoded and other text formats are accepted as before (see content types). The dashboard previews images and lets you download the original payload.

multipart/form-data requests keep their original bytes too, and their fields become a structured object. Filters can match on a field such as body.event, search indexes the fields, and the dashboard lists the attached files. The multipart requests guide covers the details.

To see every type in one place, the binary payloads demo sends a real file for each of 30 content types through a source to hookdeck listen, compares the SHA-256 of what arrives with what was sent, and previews the result: images, audio and video players, and the contents of zip and tar archives.

Run Node.js in transformations: built-ins, ES modules and bundled packages

Transformations now run on a Node.js-compatible runtime. Built-in modules such as crypto, zlib, buffer, querystring and url run natively, and binary bodies arrive in request.body as a Buffer.

You can write a transformation as a classic script with addHandler, or as an ES module that exports the handler as export default { transform }. Handlers can be async. When you need a library, bundle it with esbuild into a single file. See which modules and libraries are supported.

Turn files into events: gzip batches, PDFs, multipart uploads and JWTs

A transformation can unpack a file and hand the destination JSON that you can filter and search. This one handles a provider that sends gzip-compressed JSON:

import { gunzipSync } from 'node:zlib';

export default {
  transform(request) {
    request.body = JSON.parse(gunzipSync(request.body).toString('utf8'));
    request.headers['content-type'] = 'application/json';
    delete request.headers['content-encoding'];
    return request;
  },
};

We've added some guides that cover more of these:

• Decompress gzip and zip payloads, and compress a payload before delivery
• Extract text and data from a PDF
• Transform multipart file uploads, such as converting an uploaded CSV to JSON
• Decode and verify JWTs with node:crypto or jose

Generate files from events: PNG, WAV, PDF and SVG

A transformation can also return a different content type from the one it received. Return a Buffer and set content-type, and the destination receives those bytes. Return a string to send text in any format, such as CSV, YAML or SVG.

The transformations demo has six connections, each with a transformation that uses only Node.js built-ins, with no npm packages and no bundling. You send a JSON event and follow it from the source, through the transformation and its console output, to the file that hookdeck listen delivers:

Transformation limits: 1 second, 20 MiB, no network or filesystem

Transformations still can't make network calls or read and write files: fs, http and net can be imported, but their functions throw. Each run is limited to 1 second, which includes time spent waiting on timers. Binary bodies up to 20 MiB are passed to transformations, and larger ones are delivered unchanged.

Try the demos

Each demo runs locally against your own Hookdeck project and uses the Hookdeck CLI from npm. Delivering binary bodies to hookdeck listen needs CLI 3.1.0 or later.

• Deepgram speech-to-text and text-to-speech: needs a Deepgram API key.
• Binary payloads: sends 30 content types and checks each arrives byte-exact.
• Transformations with Node.js built-ins: the six examples above, with a setup script that creates the connections through the API.

For more details, read the changelogs: binary and multipart and Node.js transforms.

Feedback: which formats do you need?

Files now go in and out of the Event Gateway as bytes, and a transformation between the two can read them, rewrite them, or create new ones. If a provider sends you a content type that isn't on the list, or there's a file you'd like to generate from your events, tell us.