September 17, 2024
Trigger.dev v3 is now out of beta and at v3.0.0. This release includes a new CLI, a revamped build system, and more.
Trigger.dev v3, the open-source background job platform with no timeouts, is now generally available and out of beta. Get started today by running npx trigger.dev@latest init
and deploying your first job in minutes.
Before we dive into the details of this release, a quick recap of what Trigger.dev v3 is and how it can help you (or skip right to the What's new in 3.0.0 section):
Trigger.dev v3
Trigger.dev v3 is not just a rewrite—it's a complete reimagining of the platform. You still write your typesafe tasks right next to your application code. However, instead of merely orchestrating your tasks, Trigger.dev v3 now builds and deploys them to an elastic, serverless runtime. This also means you aren't double billed, once for your orchestrator and once for your serverless runtime.
A new runtime built for long-running tasks
Unlike existing runtimes like AWS Lambda or Google Cloud Functions, we've built a new runtime from the ground up specifically for writing and executing long-running tasks, featuring a checkpoint-restore system that allows for tasks to wait indefinitely without wasting resources or forcing a clunky programming model.
_32import { task, wait } from "@trigger.dev/sdk/v3";_32_32export const parentTask = task({_32 id: "parent-task",_32 run: async () => {_32 console.log("Starting parent task");_32_32 // This will cause the parent task to be checkpointed and suspended_32 const result = await childTask.triggerAndWait({ data: "some data" });_32_32 console.log("Child task result:", result);_32_32 // This will also cause the task to be checkpointed and suspended_32 await wait.for({ seconds: 30 });_32_32 console.log("Resumed after 30 seconds");_32_32 return "Parent task completed";_32 },_32});_32_32export const childTask = task({_32 id: "child-task",_32 run: async (payload: { data: string }) => {_32 console.log("Starting child task with data:", payload.data);_32_32 // Simulate some work_32 await sleep(5);_32_32 return "Child task result";_32 },_32});
This comes with a built-in way to build and deploy your tasks using our CLI:
_10npx trigger.dev@latest deploy
And a way to make sure your tasks run locally, before they're deployed:
_10npx trigger.dev@latest dev
To learn more about how the runtime works, check out our How it works guide.
Queueing and scheduling
On top of this runtime, we've added a queueing and scheduling system. This allows you to queue up work to be done later, schedule tasks to run at specific times, and even dynamically create queues on the fly.
Multi-tenant ready
Trigger.dev v3 is built from the ground up to be multi-tenant ready, without having to manually manage multiple accounts or workspaces. For example, you can easily create on-the-fly tenant concurrency limits when triggering tasks:
_16import { task } from "@trigger.dev/sdk/v3";_16_16export const tenantTask = task({_16 id: "tenant-task",_16 run: async () => {_16 console.log("Starting tenant task");_16_16 return "Tenant task completed";_16 },_16});_16_16// Trigger the task with a tenant concurrency limit of 5, but just for this tenant_16await tenantTask.trigger({_16 concurrencyKey: "my-tenant-id",_16 queue: { concurrencyLimit: 5 },_16});
We also support scheduling tasks to be triggered at a specific time for a specific tenant:
_21import { schedules } from "@trigger.dev/sdk/v3";_21_21export const firstScheduledTask = schedules.task({_21 id: "first-scheduled-task",_21 run: async (payload) => {_21 // payload.externalId will be "my-tenant-id"_21 },_21});_21_21// Create a schedule for the task to run at 8am every day for this tenant_21const createdSchedule = await schedules.create({_21 task: reminderTask.id,_21 //8am every day_21 cron: "0 8 * * *",_21 //the user's timezone_21 timezone: "America/New_York",_21 //the user id_21 externalId: "my-tenant-id",_21 //this makes it impossible to have two reminder schedules for the same tenant_21 deduplicationKey: "my-tenant-id/reminder",_21});
To learn more, check out our Concurrency & Queues guide, and our Scheduling guide.
Extreme visibility with OpenTelemetry
Our dashboard is powered by OpenTelemetry, which means you can see exactly what your tasks are doing in real-time. This includes logs and traces auto-correlated across your tasks. And we make it easy to add custom instrumentation to your tasks, so you can see exactly what you need to see.
Learn more over at our Logging & Tracing guide.
Key Features
- No Timeouts: tasks can run indefinitely without being interrupted.
- Open-source: The entire platform is open-source, allowing you to run it on your own infrastructure if you prefer.
- Dedicated Execution: Each job runs in its own isolated environment, eliminating interference between jobs.
- Deploy and Scale: Trigger.dev handles the execution and scaling of your jobs, letting you focus on writing code.
They say a picture is worth a thousand words, so here are about 1200 pictures stitched together (that's 1.2m words) to show you what Trigger.dev v3 is all about:
To dig even deeper, check out our How it works guide or our Quick Start guide.
What's new in 3.0.0
We released the beta version of Trigger.dev v3 in early April of this year. Since then, we've been working on various improvements and bug fixes based on the feedback we received from our beta users.
One major area of focus has been on the build system, which has been completely rewritten as part of the v3 latest release. The main features of the new build system are:
- Bundling by default: All dependencies are bundled by default, so you no longer need to specify which dependencies to bundle. This solves a whole bunch of issues related to monorepos.
- Build extensions: A new way to extend the build process with custom logic. This is a more flexible and powerful way to extend the build process compared to the old system (including custom esbuild plugin support).
- Multi-runtime support: We now support multiple runtimes, including Bun, which is a new runtime for Trigger.dev.
- Improved configuration: We've migrated to using c12 to power our configuration system.
- Better deployment error reporting: We now do a much better job of reporting any errors that happen during the deployment process.
Bundling by default
Our new build system is powered by esbuild, and we've moved to a model where all dependencies are bundled by default. This means you no longer need to specify which dependencies to bundle, and you can just import them as you normally would in your code. This should make it much easier to work with monorepos and other complex project structures. The biggest change is that you will now need to specify any dependencies that you don't want to bundle, and instead load externally. This can be done like this:
_10import { defineConfig } from "@trigger.dev/sdk/v3";_10_10export default defineConfig({_10 build: {_10 external: ["sharp"],_10 },_10});
We also auto-detect dependencies that will need to be external to enable OpenTelemetry instrumentation, so you don't need to worry about adding them manually:
_10import { defineConfig } from "@trigger.dev/sdk/v3";_10import { OpenAIInstrumentation } from "@traceloop/instrumentation-openai";_10_10export default defineConfig({_10 // `openai` will be automatically added to the external list_10 instrumentations: [new OpenAIInstrumentation()],_10});
Build extensions
Build extensions are a powerful new feature in Trigger.dev v3 that allow you to extend the build process with custom logic. You can now fully customize the bundled code through adding esbuild plugins, and you can also add and modify the deployed image to add custom system dependencies and more.
One example of a build extension is the prismaExtension
, which will prepare your deployment to be able to use Prisma. It can be used like this:
_13import { defineConfig } from "@trigger.dev/sdk/v3";_13import { prismaExtension } from "@trigger.dev/build/extensions/prisma";_13_13export default defineConfig({_13 build: {_13 extensions: [_13 prismaExtension({_13 version: "5.19.0", // optional, we'll automatically detect the version if not provided_13 schema: "prisma/schema.prisma",_13 }),_13 ],_13 },_13});
The extension handles generating the Prisma client and optionally allows you to run migrations on deployment. It also has built-in support for multi-schema Prisma projects and the newly released TypedSQL feature.
Another example is the ffmpeg
extension, which allows you to add the ffmpeg
binary to your deployment. It can be used like this:
_10import { defineConfig } from "@trigger.dev/sdk/v3";_10import { ffmpeg } from "@trigger.dev/build/extensions/core";_10_10export default defineConfig({_10 build: {_10 extensions: [ffmpeg()],_10 },_10});
Our syncEnvVars extension allows you to easily sync secrets when you deploy. An example of how you can use it with Infisical:
_34import { InfisicalClient } from "@infisical/sdk";_34import { syncEnvVars } from "@trigger.dev/build/extensions/core";_34import { defineConfig } from "@trigger.dev/sdk/v3";_34_34export default defineConfig({_34 build: {_34 extensions: [_34 syncEnvVars(async (ctx) => {_34 if (_34 !process.env.INFISICAL_CLIENT_ID ||_34 !process.env.INFISICAL_CLIENT_SECRET ||_34 !process.env.INFISICAL_PROJECT_ID_34 ) {_34 return;_34 }_34_34 const client = new InfisicalClient({_34 clientId: process.env.INFISICAL_CLIENT_ID,_34 clientSecret: process.env.INFISICAL_CLIENT_SECRET,_34 });_34_34 const secrets = await client.listSecrets({_34 environment: ctx.environment,_34 projectId: process.env.INFISICAL_PROJECT_ID,_34 });_34_34 return secrets.map((secret) => ({_34 name: secret.secretKey,_34 value: secret.secretValue,_34 }));_34 }),_34 ],_34 },_34});
You can easily add esbuild plugins to your build process using the esbuildPlugin
extension. Here's an example of how you can use the @sentry/esbuild-plugin
with Trigger.dev for automatically uploading sourcemaps to Sentry on deploy:
_18import { sentryEsbuildPlugin } from "@sentry/esbuild-plugin";_18import { esbuildPlugin } from "@trigger.dev/build";_18import { defineConfig } from "@trigger.dev/sdk/v3";_18_18export default defineConfig({_18 build: {_18 extensions: [_18 esbuildPlugin(_18 sentryEsbuildPlugin({_18 org: process.env.SENTRY_ORG,_18 project: process.env.SENTRY_PROJECT,_18 authToken: process.env.SENTRY_AUTH_TOKEN,_18 }),_18 { placement: "last", target: "deploy" }_18 ),_18 ],_18 },_18});
You can also write your own custom extensions to add any custom logic you need to the build process. Extensions are written in TypeScript and can be as simple or complex as you need them to be:
See our docs for more on our built-in extensions and how to write your own.
Bun runtime support
We've added support for the Bun runtime in Trigger.dev v3:
_10import { defineConfig } from "@trigger.dev/sdk/v3";_10_10export default defineConfig({_10 runtime: "bun",_10});
Allowing you to use Bun APIs and features in your Trigger.dev tasks:
_15import { Database } from "bun:sqlite";_15import { task } from "@trigger.dev/sdk/v3";_15_15export const bunTask = task({_15 id: "bun-task",_15 run: async (payload: { query: string }) => {_15 const db = new Database(":memory:");_15 const query = db.query("select 'Hello world' as message;");_15 console.log(query.get()); // => { message: "Hello world" }_15_15 return {_15 message: "Query executed",_15 };_15 },_15});
To get started using Bun, check out our Bun quickstart guide.
Other improvements
We've completely rewritten our dev
and deploy
CLI commands as well, including a much simpler architecture and more shared code. During the beta these commands did not share much code when it came to building and running, which led to regressions and inconsistencies. We've now unified the codebase and made it much easier to maintain and extend. We've also added an end-to-end testing suite for the build system to ensure that we don't introduce regressions in the future and make it easier to refactor and improve the build system. Additionally:
- Faster cold start times - We no longer load all your trigger task files when executing a task, instead we load them dynamically as needed. This should lead to faster cold start times.
- Faster deploys - We've improved the deploy process to be much faster by optimizing the way we build and push images.
- ESM builds - We now output builds as ESM instead of CommonJS, improving the ability to tree-shake out unused code and reducing the size of the final build. This also means we're prepared for the future of JavaScript & TypeScript.
- Improved configuration - Our configuration loading is now powered by c12, which is used by Nuxt, Nitro, and other large projects. This should mean fewer bugs as we're using a well-tested library, instead of rolling our own.
Self-hosting updates
-
Full support for our latest CLI - We updated our repo and self-hosting guide to add full support for our latest tag packages. Goodbye
@beta
. -
Easy version locking - You can now more easily lock your CLI and self-hosted instance to the same version tag to ensure compatibility and prevent nasty bugs:
_10TRIGGER_IMAGE_TAG=v3.0.4
- Better self-hosted deploys - We've improved the
deploy
command for self-hosters of Trigger.dev.deploy
now supports pushing to a custom registry:
_10npx trigger.dev@latest deploy \_10 --self-hosted \_10 --load-image \_10 --registry docker.io \_10 --namespace mydockerhubusername
- Deploy dry runs - You can now preview what your deployment will look like before actually deploying it:
_10npx trigger.dev@latest deploy --dry-run
Upgrading from the beta
If you are currently running on the beta version of Trigger.dev v3 packages, you can upgrade to the latest version by running npx trigger.dev@latest update
. Make sure to check out the upgrade guide for more information on what has changed and how to update your config. If you have any questions or need help with the upgrade, feel free to reach out to us on Discord.
NOTE
If you are self-hosting v3 and want to upgrade from the beta, checkout our self-hosting docs for instructions on how to upgrade.
Upgrading from v2
We announced the end-of-life for Trigger.dev v2 last month, which is coming up on Jan 31st, 2025. So you still have some time to upgrade to v3. We have a migration guide to help you with the upgrade process. You can still use the @trigger.dev/sdk
package and the @trigger.dev/cli
, and both have matching 3.0.x releases. The same goes for all our v2 integration and framework packages, like @trigger.dev/nextjs
, @trigger.dev/openai
, etc. All the code for these v2 packages can be found in our new v2 Legacy repo.
If you have any questions or need help with the migration, feel free to reach out to us on Discord.
Moving forward
Going forward, our @trigger.dev/sdk
package and trigger.dev
CLI will follow semantic versioning, which you can stay up to date on in our changelog and GitHub Releases. We're also working on some big new features, like Realtime Notifications, to bridge the gap between your tasks and your users. Stay tuned for more updates on that and other new features in the coming months.