Launch-week

·

Batch processing

Eric Allam

Eric Allam

CTO, Trigger.dev

Image for Batch processing

Today we're announcing a slew of improvements to our batch trigger feature: New limits, a dedicated dashboard page, new SDK methods, and more

Increased batch size

We've heard from quite a few people that our batch size limit of 100 items is too low, so we've built a new batch endpoint that can handle substantially more items, and we've increased the limit to 500 items.

We achieved this by changing the way the endpoint works to process items asynchronously. This means that the endpoint will return a response immediately after receiving the batch, and the items will be processed in the background. Previously the endpoint would wait for all items to be processed before returning a response, which would sometimes cause timeouts or just very slow responses.

NOTE

To take advantage of the new batch size limit, you'll need to be on version 3.3.0 or later of the SDK.

New batches page in the dashboard

See all your batches together in a dedicated dashboard page and view all runs filtered by batch. Here's a short run-through:

New batch trigger SDK methods

We have added some additional methods to the SDK to make working with batches even easier:

batch.trigger

batch.trigger allows you to trigger multiple tasks in a single batch. You can pass an array of objects, each with an id for the task you want to trigger and a payload object with the data you want to pass to the task.

src/app/routes/batch.ts

import { batch } from "@trigger.dev/sdk/v3";
// 👆 this is new
import type { myTask1, myTask2 } from "~/trigger/myTasks";
export async function POST(request: Request) {
//get the JSON from the request
const data = await request.json();
// Pass a union of the tasks to `trigger()` as a generic argument, giving you full type checking
const result = await batch.trigger<typeof myTask1 | typeof myTask2>([
// Because we're using a union, we can pass in multiple tasks by ID
{ id: "my-task-1", payload: { some: data.some } },
{ id: "my-task-2", payload: { other: data.other } },
]);
//return a success response with the result
return Response.json(result);
}

batch.triggerAndWait

batch.triggerAndWait is similar to batch.trigger, but it will wait for all tasks to complete before returning a response (and can only be used inside another task)

src/trigger/tasks.ts

import { batch, task } from "@trigger.dev/sdk/v3";
export const parentTask = task({
id: "parent-task",
run: async (payload: string) => {
// 👇 Pass a union of all the tasks you want to trigger
const results = await batch.triggerAndWait<
typeof childTask1 | typeof childTask2
>([
{ id: "child-task-1", payload: { foo: "World" } }, // 👈 The payload is typed correctly based on the task `id`
{ id: "child-task-2", payload: { bar: 42 } }, // 👈 The payload is typed correctly based on the task `id`
]);
for (const result of results) {
if (result.ok) {
// 👇 Narrow the type of the result based on the taskIdentifier
switch (result.taskIdentifier) {
case "child-task-1":
console.log("Child task 1 output", result.output); // 👈 result.output is typed as a string
break;
case "child-task-2":
console.log("Child task 2 output", result.output); // 👈 result.output is typed as a number
break;
}
} else {
console.error("Error", result.error); // 👈 result.error is the error that caused the run to fail
}
}
},
});
export const childTask1 = task({
id: "child-task-1",
run: async (payload: { foo: string }) => {
return `Hello ${payload}`;
},
});
export const childTask2 = task({
id: "child-task-2",
run: async (payload: { bar: number }) => {
return bar + 1;
},
});

batch.triggerByTask

You can batch trigger multiple different tasks by passing in the task instances. This function is especially useful when you have a static set of tasks you want to trigger:

src/trigger/tasks.ts

import { batch, task, runs } from "@trigger.dev/sdk/v3";
export const parentTask = task({
id: "parent-task",
run: async (payload: string) => {
const results = await batch.triggerByTask([
{ task: childTask1, payload: { foo: "World" } }, // 👈 The payload is typed correctly based on the task instance
{ task: childTask2, payload: { bar: 42 } }, // 👈 The payload is typed correctly based on the task instance
]);
// 👇 results.runs is a tuple, allowing you to get type safety without needing to narrow
const run1 = await runs.retrieve(results.runs[0]); // 👈 run1 is typed as the output of childTask1
const run2 = await runs.retrieve(results.runs[1]); // 👈 run2 is typed as the output of childTask2
},
});
export const childTask1 = task({
id: "child-task-1",
run: async (payload: { foo: string }) => {
return `Hello ${payload}`;
},
});
export const childTask2 = task({
id: "child-task-2",
run: async (payload: { bar: number }) => {
return bar + 1;
},
});

batch.triggerByTaskAndWait

batch.triggerByTaskAndWait is similar to batch.triggerByTask, but it will wait for all tasks to complete before returning a response (and can only be used inside another task)

src/trigger/tasks.ts

import { batch, task, runs } from "@trigger.dev/sdk/v3";
export const parentTask = task({
id: "parent-task",
run: async (payload: string) => {
const { runs } = await batch.triggerByTaskAndWait([
{ task: childTask1, payload: { foo: "World" } }, // 👈 The payload is typed correctly based on the task instance
{ task: childTask2, payload: { bar: 42 } }, // 👈 The payload is typed correctly based on the task instance
]);
if (runs[0].ok) {
console.log("Child task 1 output", runs[0].output); // 👈 runs[0].output is typed as the output of childTask1
}
if (runs[1].ok) {
console.log("Child task 2 output", runs[1].output); // 👈 runs[1].output is typed as the output of childTask2
}
// 💭 A nice alternative syntax is to destructure the runs array:
const {
runs: [run1, run2],
} = await batch.triggerByTaskAndWait([
{ task: childTask1, payload: { foo: "World" } }, // 👈 The payload is typed correctly based on the task instance
{ task: childTask2, payload: { bar: 42 } }, // 👈 The payload is typed correctly based on the task instance
]);
if (run1.ok) {
console.log("Child task 1 output", run1.output); // 👈 run1.output is typed as the output of childTask1
}
if (run2.ok) {
console.log("Child task 2 output", run2.output); // 👈 run2.output is typed as the output of childTask2
}
},
});
export const childTask1 = task({
id: "child-task-1",
run: async (payload: { foo: string }) => {
return `Hello ${payload}`;
},
});
export const childTask2 = task({
id: "child-task-2",
run: async (payload: { bar: number }) => {
return bar + 1;
},
});

Full stack example

We've put together a full stack example that uses the following:

  • A Next.js app with Prisma for the database.
  • Trigger.dev Realtime to stream updates to the frontend.
  • Work with multiple LLM models using the AI SDK
  • Distribute tasks across multiple tasks using the new batch.triggerByTaskAndWait method.

This demo runs a prompt through three different LLM models and then streams the results to the frontend using Realtime streams.

View the GitHub repository for the full code.

Additional SDK improvements

runs.list

You can now list all runs for a batch trigger using the runs.list method:


import { runs } from "@trigger.dev/sdk/v3";
// List all runs in a batch
const runs = await runs.list({ batch: "batch_1234" });

runs.subscribeToBatch

You can always subscribe to realtime updates for all runs in a batch using the runs.subscribeToBatch method:


import { runs } from "@trigger.dev/sdk/v3";
// Subscribe to realtime updates for all runs in a batch
for await (const run of runs.subscribeToBatch("batch_1234")) {
console.log(run);
}

useRealtimeBatch

You can use the useRealtimeBatch hook to subscribe to realtime updates for all runs in a batch in a React component:

src/components/MyComponent.tsx

"use client"; // This is needed for Next.js App Router or other RSC frameworks
import { useRealtimeBatch } from "@trigger.dev/react-hooks";
export function MyComponent({ batchId }: { batchId: string }) {
const { runs, error } = useRealtimeBatch(batchId);
if (error) return <div>Error: {error.message}</div>;
return (
<div>
{runs.map((run) => (
<div key={run.id}>Run: {run.id}</div>
))}
</div>
);
}

See our React Hooks documentation for more information.

batch.retrieve

You can now retrieve a batch by ID using the batch.retrieve method:


import { batch } from "@trigger.dev/sdk/v3";
// Retrieve a batch by ID
const batch = await batch.retrieve("batch_1234");

Batch Idempotency

We've added support for idempotency keys to the batch trigger endpoint. This means that you can pass an idempotencyKey and we'll ensure that the batch is only processed once, even if the request is retried.

src/app/routes/batch.ts

import { batch } from "@trigger.dev/sdk/v3";
export async function POST(request: Request) {
//get the JSON from the request
const data = await request.json();
// Pass an idempotency key to `trigger()` to ensure the batch is only processed once
const result = await batch.trigger<typeof myTask1 | typeof myTask2>(
[
{ id: "my-task-1", payload: { some: data.some } },
{ id: "my-task-2", payload: { other: data.other } },
],
{ idempotencyKey: "my-idempotency-key" }
);
//return a success response with the result
return Response.json(result);
}

By default, idempotency keys will expire after 30 days, but you can pass an idempotencyKeyTTL option to set a custom expiration duration:

src/app/routes/batch.ts

import { batch } from "@trigger.dev/sdk/v3";
export async function POST(request: Request) {
//get the JSON from the request
const data = await request.json();
// Pass an idempotency key to `trigger()` to ensure the batch is only processed once
const result = await batch.trigger<typeof myTask1 | typeof myTask2>(
[
{ id: "my-task-1", payload: { some: data.some } },
{ id: "my-task-2", payload: { other: data.other } },
],
{ idempotencyKey: "my-idempotency-key", idempotencyKeyTTL: "1d" } // 1 day
);
//return a success response with the result
return Response.json(result);
}

To learn more about using idempotency keys, see our Idempotency documentation.

Getting started

Head over to our Batch triggering documentation to learn more.

As always, we're here to help if you have any questions or need assistance. You can reach out to us on Discord or X.

Ready to start building?

Build and deploy your first task in 3 minutes.

Get started now
,