Multi-Step Functions

Use Inngest's multi-step functions to safely coordinate between events, delay execution for hours or days, and conditionally run code based on the result of previous steps and incoming events.

Critically, multi-step functions are written in code, not config, meaning you create readable, obvious functionality that's easy to maintain.

Benefits

Writing multi-step functions provide you with some easy-to-use tools to create intuitive flows for your system.

  • Run retryable blocks of code to maximum reliability
  • Pause execution and Wait for an event matching rules before continuing
  • Pause for an amount of time or until a specified time

This makes building reliable, distributed code simple. By wrapping asynchronous actions such as API calls in retryable blocks, we can ensure reliability when coordinating across many services.

Writing

Multi-step functions are written using either the createStepFunction() method on the Inngest client.

First, let's look at a simple single-step function.

import { Inngest } from "inngest";

const inngest = new Inngest({ name: "My App" })

export default inngest.createFunction(
  "Activation email",
  "app/user.created",
  async ({ event }) => {
    await sendEmail({ email: event.user.email, template: "welcome" });
  }
);

This function will send a user an email when they sign up. Nice and simple.

We have a new requirement, though, that we should send the user another email if they haven't created a post on our platform within 24 hours of signing up. We have a app/post.created event that is fired when this happens, so we can use that (or here, the absence of that) to trigger the second email.

First, let's convert out function to a multi-step function. To do this, we'll do a few things:

  • Use createStepFunction() instead of createFunction()
  • Change our provided handler to be non-async
  • Add a new tools argument
  • Wrap our sendEmail() call in a tools.run() call
export default inngest.createStepFunction(
  "Activation email",
  "app/user.created",
  ({ event, tools }) => {
    tools.run("Send welcome email", () =>
      sendEmail({ email: event.user.email, template: "welcome" })
    );
  }
);

Great! Now we have a multi-step function.

The main difference is that we've wrapped our sendEmail() call in a tools.run() call. This is how we tell Inngest that this is an individual step in our function. This step can be retried independently, just like a single-step function would.

Once our welcome email is sent, we want to wait at most 24 hours for our user to create a post. If they haven't created one by then, we want to send them a reminder email.

To do this, we can use the waitForEvent tool. This tool will wait for a matching event to be fired, and then return the event data. If the event is not fired within the timeout, it will return null, which we can use to decide whether to send the reminder email.

export default inngest.createStepFunction(
  "Activation email",
  "app/user.created",
  ({ event, tools }) => {
    tools.run("Send welcome email", () =>
      sendEmail({ email: event.user.email, template: "welcome" })
    );

    // Wait for an "app/post.created" event
    const postCreated = tools.waitForEvent("app/post.created", {
      match: "data.user.id", // the field "data.user.id" must match
      timeout: "24h", // wait at most 24 hours
    });
  }
);

Now we have our postCreated variable, which will be null if the user hasn't created a post within 24 hours, or the event data if they have.

Finally, we can use this to send the reminder email if the user hasn't created a post by running another block of code with tools.run().

export default inngest.createStepFunction(
  "Activation email",
  "app/user.created",
  ({ event, tools }) => {
    tools.run("Send welcome email", () =>
      sendEmail({ email: event.user.email, template: "welcome" })
    );

    // Wait for an "app/post.created" event
    const postCreated = tools.waitForEvent("app/post.created", {
      match: "data.user.id", // the field "data.user.id" must match
      timeout: "24h", // wait at most 24 hours
    });

    if (!postCreated) {
      // If no post was created, send a reminder email
      tools.run("Send reminder email", () =>
        sendEmail({ email: event.user.email, template: "reminder" })
      );
    }
  }
);

That's it! We've now written a multi-step function that will send a welcome email, and then send a reminder email if the user hasn't created a post within 24 hours.

Most importantly, we had to write no config to do this. We can use all the power of JavaScript to write our functions and all the power of Inngest's tools to coordinate between events and steps.

Tools

Inngest provides a number of tools to help you write multi-step functions. These tools are available in the tools argument of your function.

  • run - Run synchronous or asynchronous code as a retryable step in your function
  • waitForEvent - Wait for an event to be fired, and return the event data
  • sleep - Sleep for a given amount of time
  • sleepUntil - Sleep until a given time

run()

Use tools.run() to run synchronous or asynchronous code as a retryable step in your function.

It takes a name for the block (for you, so you can easily see which steps ran) and a function to run when the block is triggered, then returns a Promise that resolves to the return value of the given function.

For example, I could fetch a random user from the database in a lottery draw:

const user = await tools.run("Get random user", () => getRandomUser());

waitForEvent()

Use tools.waitForEvent() to wait for a particular event to be received before continuing. It returns a Promise that is resolved with the received event.

const postCreated = tools.waitForEvent("app/post.created", {
  timeout: "30 minutes",
  match: "data.post_id",
});

A timeout must be provided as the maximum time to wait for the event. If this timeout is reached, the tool will resolve with null instead of the event data.

To match a particular event, you can use the match option. This is as key which is used to compare the function's initial event payload and the event payload that you're waiting for. The above example will make sure that the initial event's data.post_id value exactly matches the app/post.created's data.post_id - Any app/post.created that does not match will be ignored by this wait.

sleep()

Use tools.sleep() to wait for a specified amount of time before continuing.

tools.sleep("30 minutes");
tools.sleep(1000 * 60 * 30);

time can be specified using a number of milliseconds or an ms-compatible time string like "1 hour", "30 mins", or "2.5d". See the ms package for more information.

To wait until a particular date, use tools.sleepUntil() instead.

sleepUntil()

Use tools.sleepUntil() to wait until a particular date before continuing by passing a Date.

Any dates passed to sleepUntil() must be static for every call to your function. This is best achieved by calculating dates within tools.run() instead of in the main function body.

const userBirthday = tools.run("Get user birthday", async () => {
  const user = await getUser(1);
  return user.birthday;
});

tools.sleepUntil(userBirthday);

To wait a particular amount of time from now, always use tools.sleep() instead.

Gotchas

My function is running twice

Inngest will communicate with your function multiple times throughout a single run and will use your use of tools to smartly memoise state.

For this reason, placing business logic outside of a tools.run() call is a bad idea, as this will be run every time Inngest communicates with your function.

I want to run asynchronous code

Easy! To run some asynchronous code, perform a call to tools.run(), providing an async function, like so:

tools.run("Do something", async () => {
  // your code
});

Ideally, each call to tools.run() is a single retryable action, so it's usually a good idea provide a synchronous function and return a promise.

For example, the below code is problematic.

tools.run("Create alert", async () => {
  const alertId = await createAlert();
  await sendAlertLinkToSlack(alertId);
});

If createAlert() succeeds but sendAlertLinkToSlack() fails, the code will be retried and an alert will be created every time the step is retried.

Instead, we should split out asynchronous actions in to multiple steps so they're retried independently.

const alertId = tools.run("Create alert", () => createAlert());

tools.run("Send alert link", () => sendAlertLinkToSlack(alertId));

My variable isn't updating

Because Inngest communicates with your function multiple times, memoising state as it goes, code within calls to tools.run() is not called on every invocation.

This can be confusing if you're using steps to update variables within the function's closure, like so:

// THIS IS WRONG!  tools.run only runs once and is skipped for future
// steps, so userID will not be defined.
let userId;

// Do NOT do this!  Instead, return data from tools.run.
tools.run("Get user", async () => {
  userId = await getRandomUserId()
});

console.log(userId); // undefined

Instead, make sure that any variables needed for the overall function are returned from calls to tools.run():

// This is the right way to set variables within tools.run :)
const userId = tools.run("Get user", () => getRandomUserId());

console.log(userId); // 123

sleepUntil() isn't working as expected

Make sure to only to use sleepUntil() with dates that will be static across the various calls to your function.

Always use sleep() if you'd like to wait a particular time from now.

// ❌ Bad
const tomorrow = new Date();
tomorrow.setDate(tomorrow.getDate() + 1);
tools.sleepUntil(tomorrow);

// ✅ Good
tools.sleep("1 day");
// ✅ Good
const userBirthday = tools.run("Get user birthday", async () => {
  const user = await getUser();
  return user.birthday; // Date
});

sleepUntil(userBirthday);