Transformations

A transformation lets you modify the payload of a event prior to delivery.

How transformations work

You can use transformations to solve many problems. Here are a few common scenarios:

• Change payload formats: For example, converting XML to JSON
• Unify models across sources: For example, conforming WooCommerce orders and Shopify orders to a single standard
• Add compatibility when routing to an API: For example, adding keys and reformatting the payload
• Add additional properties to payloads: For example, to more clearly indicate the event type for use later in the pipeline in a filter or in your own application logic.

Filter rules are run after any transformation has been performed.

Syntax

Hookdeck allows for arbitrary transformations on request data using JavaScript (ES6). For a complete overview, see transformations syntax.

Limitations

Transformations have some important limitations to keep in mind.

• The transformation runtime cannot perform any IO, or access any external resources such as the network or file system.
• The transformation runtime does not support promises and async/await.
• The execution runtime is limited to 1 second.
• Transformation code is limited to 5 MB.
• Transformations run in environments known as isolates, which carry their own set of limitations, detailed below.

Environment

Hookdeck sandboxes your transformation process within a V8 runtime environment known as an isolate. Isolates allow multiple JavaScript VM instances to operate in parallel, sharing low-level resources while remaining fundamentally distinct and secure.

Isolates deliver a host of performance and security advantages, but they also come with some noteworthy limitations.

While external JS libraries can be bundled with webpack for use inside your transformations, not all bundled libraries will function out of the box. Libraries that call unsupported APIs (such as the Node.js API) may require shimming using polyfills as a means of adding support. And for some libraries, this workaround may not function due to other limitations, such as:

• Dependencies for which polyfills cannot add support.
• Resource limitations.
• Process timeouts.

In the event that memory or time limits are exceeded, the isolate will terminate the execution of your code.

Supported methods

The following methods are supported out of the box.

• JSON.parse
• JSON.stringify

Transformations syntax

Hookdeck allows for arbitrary transformations on request data using JavaScript (ES6).

Handler function

The transformation handler is used to apply changes to a request. For example, the transformation code below adds a new header to the payload.

addHandler("transform", (request, context) => {
  request.headers["example-header"] = "Hello World";
  return request;
});

Let's take a look at the handler function's component parts.

Request object

request is the request object received on the connection. Its structure is denoted below.

{
  headers: { [key: string]: string };
  body: string | boolean | number | object | null;
  query: string;
  parsed_query: object;
  path: string;
}

Context object

context is an object containing the connection information. Its structure is denoted below.

{
  connection: Connection;
}

Where Connection is an object, as defined in the response of the GET /connections/:id endpoint.

Return value

The handler must return a valid request object, containing a valid content-type header.

The parsed_query object in the returned request can safely be ignored – Hookdeck will recreate this parsed query on its own.

The following transformation formats an order based on its source:

const formatOrder = (order, type) => {
  if (type === 'shopify') {
    // Format for shopify here
    return order;
  } if (type === 'woocommerce') {
    // Format for woocomerce here
    return order;
  }
}

// Handle that formats the order from Shopify and Woocommerce
addHandler('transform', (request, context) => {
  const { body, headers, path, query } = request;
  const { connection } = context;
  return {
    body: formatOrder(body, connection.source.name)
    headers,
    path,
    query
  };
});

Environment variables

Transformations support environment variables for storing secrets. Access your variable on the process.env object. For example, to access a variable called TEST use process.env.TEST.

Environment variables are managed in the transformations editor via the Variables drop down.

Create a transformation

Transformations are applied to a connection, just like any other rule. But unlike other rules, transformations can be shared between multiple connections.

To version control your transformations, consider creating or updating your connection using the API or the Hookdeck Terraform provider.

1. Open the connection rules configuration.

2. Click Transform to add a transformation rule.

3. Click Create new transformation.

4. Configure the transformation using the supported transformation syntax in the right-hand pane.

5. Once written, test your transformation.

   • The left pane's Input tab shows the request your transformation will be tested against. To change this request, click Change and select a recent payload, or edit the text directly.
   • Click Run to test the transformation. Once run, the left pane's Output tab shows the resulting payload, whereas the Diff tab shows the difference between the test input and test output.

6. Name the transformation using the input field in the upper left, clicking Confirm to update the name.

7. Once satisfied with the transformation, click Confirm.

8. Make sure to click Save again on the connection form to apply your changes.

From this point forward, events received on the connection are transformed prior to delivery to their destination.

Edit a transformation

Editing a transformation changes how payload data is transformed before delivery.

1. Open the connection rules configuration.

2. Next to the transformation rule, click Editor.

3. Configure the transformation using the supported transformation syntax in the right-hand pane.

4. Once written, test your transformation.

   • The left pane's Input tab shows the request your transformation will be tested against. To change this request, click Change and select a recent payload, or edit the text directly.
   • Click Run to test the transformation. Once run, the left pane's Output tab shows the resulting payload, whereas the Diff tab shows the difference between the test input and test output.

5. Optionally, you may rename the transformation using the input field in the upper left. Click Confirm to confirm the name change.

6. Once satisfied, click Confirm.

7. Make sure to click Save again on the connection form to apply your changes.

Transformations are updated immediately, and any events received going forward are run through the updated transformation.

Keep in mind that updating a transformation will affect all connections that use the same transformation. If this is not your desired behavior, you can delete a transformation and create a new one.

Delete a transformation

A Transformation can only be deleted if it is not being used by any Connections.

To remove a transformation from a connection, follow the instructions for configuring connection rules and click the trash icon to remove the transformation from the connection rules.

Once a Transformation has been removed from all Connections, click on Transformations in the sidebar under Connections and select the transformation you wish to delete. Scroll to the bottom of the page and click Delete. Confirm the deletion by clicking Delete in the modal that appears.

Troubleshoot a transformation

Execution logs are kept for every transformation, in order to facilitate troubleshooting.

1. Open the connection rules configuration.

2. Next to the transformation rule, click Editor.

3. Click View Executions.

4. Click an event to open its transformation execution.

   • The Input tab shows the unmodified request received by Hookdeck.
   • The Output tab shows the request after the transformation was applied.
   • The Diff tab shows the difference between the input and output requests.
   • The Console section shows any lines logged to the console in the course of applying the transformation.

Transformation issues

When your transformation fails Hookdeck will log a FATAL execution which will appear in the logs described above. Additionally, a new Issue will be opened. An issue will also be opened if you transformation does execute but logs a warning or an error with console.error or console.warn.

In the case of a fatal failure, Events may have not been created. Hookdeck keeps tracks of those as Ignored Events in the associated issue. Ignored events can be inspected and retried directly within the issue.

1. Open the Issue page.
2. Open the issue of type transformation
3. Inspect the histogram of "Ignored Events" to see the number of events that were not created.
4. Select the latest execution and click "Edit"
5. Make the necessary changes to your transformation code and click "run" to validate that the transformation issue as been resolved.
6. Go back to the original issue, click and submit the "Bulk Retry".