Sometimes you need to reliably trigger hundreds or thousands of jobs at once, making sure that every job succeeds independently. For example, you might have a cron which triggers importing thousands of products, or a user might require 100 different OpenAPI calls at once.
You might start by running everything in a single function, or within steps. However, as the number of jobs grows this becomes slower, more difficult to observe, and more prone to temporary failures. Instead of doing everything in a single job you should fan-out by sending events to trigger new jobs.
§How this works
In order to manage thousands of jobs at once you'll need to fan-out to run functions in parallel. The fan-out pattern works as follows:
- The job trigger runs as a single function (eg. a cron for triggering product imports)
- The function loads the data needed for all future jobs (eg. loading all product IDs)
- It sends new events to trigger jobs in parallel for each job needed.
When you schedule each job independently by sending events, each job gets its own retries and runs in parallel. This makes your system faster, more observable, and more resilient to temporary failures.
§With Inngest
Inngest supports scheduled functions and event-triggered functions. Combining the two enables you to fan-out functions to run in parallel. We'll define two these two functions:
01import { inngest } from "./client";0203// A scheduled function uses the current time to find notifications to send04const slackCron = inngest.createFunction(05 { id: "slack-notification-cron", triggers: [{ cron: "0 9,12 * * MON,FRI" }] },06 async () => {07 const notifications = await getNotificationsToRun();0809 const events = notifications.map((notification) => ({10 name: "app/notification.dispatched",11 data: { notification },12 }));1314 // Send an array of events to Inngest, triggering many jobs in parallel.15 await inngest.send(events);1617 return `${notifications.length} notifications dispatched`;18 }19);2021// A function runs for every app/notification.dispatched event to22// post the notification to Slack23const postSlackNotification = inngest.createFunction(24 { id: "send-slack-notification", triggers: [{ event: "app/notification.dispatched" }] },25 async ({ event }) => {26 const reportData = getAccountReportData(event.data.notification.accountId);2728 await app.client.chat.postMessage({29 channel: event.data.notification.slackChannelId,30 blocks: generateReportSlackBlocks(reportData),31 // ...32 });33 }34);This is the system — both functions can even be defined in the same file to keep things simple and maintainable. This approach works well for systems that have commonly scheduled times, but for more flexible systems that are scheduled as one-off, non-repeated tasks you should review Patterns: Running at specific times.
§Alternative approaches
You can wire cron-driven fan-out yourself, but each component carries its own failure modes:
- Cron + queue (SQS, BullMQ, Sidekiq): the cron enqueues jobs, workers consume them. Workable, but you own the worker fleet, the queue's dead-letter routing, and the observability story across both halves.
- Cron + serverless function: simpler to run, but most serverless platforms cap function duration at 5–15 minutes. A single cron that fans out across 10,000 tenants can't finish before the platform kills it.
- All-in-one in the cron: what people start with. Works at 100 tenants, breaks at 10,000 when the cron itself runs longer than its interval and starts lapping itself.
The Inngest version above keeps the cron lightweight: it only loads IDs and sends events. Each tenant's work runs as an independently-retried function with its own logs, retries, and concurrency budget.