---
title: "AI agent observability: LangSmith vs Langfuse vs Trigger.dev"
description: "LangSmith, Langfuse, Phoenix, Braintrust, Datadog and Trigger.dev compared for monitoring AI agents in production: tracing, evals, alerts, self-hosting, pricing."
date: 2026-10-30
---

An agent can fail in two different places. The model can return a bad answer while the run finishes perfectly. Or the model can return a good answer and the run dies afterward, because a worker restarted, a tool call timed out, or an approval never came back. Dedicated LLM tracing tools record the first failure in detail. The second one only shows up if something also traces the runtime around the model call.

This page compares six tools for observing AI agents in production: LangSmith, Langfuse, Arize Phoenix, Braintrust, Datadog Agent Observability and Trigger.dev. Trigger.dev publishes this page, and we've listed our own limits alongside everyone else's. Every claim was checked against official docs, GitHub repositories and pricing pages on October 30, 2026.

## TL;DR

- **LangSmith** is LangChain's platform for tracing, evals (offline and online) and alerts. Its runtime, Agent Server, checkpoints, interrupts and resumes LangGraph runs, and can deploy other frameworks. Self-hosting is an Enterprise option.
- **Langfuse** is open source (MIT, except its `ee` folders) for tracing, evals, prompt management, dashboards and alerts. All core features are free to self-host; production self-hosting runs Postgres, ClickHouse, Redis and S3.
- **Arize Phoenix** is free to self-host with every feature, on one container with SQLite or Postgres. It's Elastic License 2.0, and threshold monitors are in Arize AX, which has a free tier.
- **Braintrust** is an observability and evals platform with `Eval()` in code, online scoring, SQL over logs, trace clustering and CI eval gating. Self-hosting the data plane is an Enterprise option.
- **Datadog Agent Observability** (formerly LLM Observability) puts agent traces, evals, cost and prompt management next to Datadog APM, monitors and security tooling. It's billed per LLM span.
- **Trigger.dev** runs your TypeScript agent and records the whole run: attempts, retries, waits, logs and spans. Vercel AI SDK calls appear in the same trace with model, tokens, cost and latency. You can run evals as tasks, and there are example projects for it, but it has no built-in datasets, scorers or experiment comparison.
- Of these six, only Trigger.dev and LangSmith (through Agent Server) run your agent. The others record and score what it did.

## Execution observability vs LLM call observability

Execution observability shows whether a run started, whether it's queued, running or waiting on a human, which attempt failed, whether the retry succeeded, how long it waited and what the compute cost.

LLM call observability shows the prompt and the completion for each model call, its token use and cost, which tools the model called, and how the output scores against a dataset or a judge.

A trace of model calls alone can explain a bad answer, but it can't show that a deploy killed the run two seconds after the model returned. An execution trace shows the run failed on attempt 2 of 3, but it can't show that the model made up a refund amount. The tools overlap more each year: Trigger.dev captures AI SDK calls inside its run trace, LangSmith has a runtime, Datadog links LLM spans to APM traces, and most of the tracing tools accept generic OpenTelemetry spans if you send them.

We left out Helicone, Opik, W&B Weave, MLflow Tracing, Laminar and Pydantic Logfire, which also cover LLM tracing and evals. We picked the five tools most often paired with TypeScript agents, plus Trigger.dev.

## LangSmith

[LangSmith](https://docs.langchain.com/langsmith/observability-concepts) is LangChain's platform for tracing and evaluating agents, and it works without LangChain. A LangSmith trace is made of runs (its word for spans), and traces group into threads for multi-turn conversations. LangSmith has datasets, experiments, a prompt hub, [annotation queues](https://docs.langchain.com/langsmith/annotation-queues) for human review, and [online evaluators](https://docs.langchain.com/langsmith/online-evaluations) that run LLM-as-a-judge on sampled production runs or whole threads. [Insights](https://docs.langchain.com/langsmith/insights) clusters traces into behavior and failure categories (Plus and Enterprise). In LangSmith Studio you can open the LLM calls inside a LangGraph node in the Playground and [test them without running the whole graph](https://docs.langchain.com/langsmith/observability-studio).

LangSmith takes OTLP at `https://api.smith.langchain.com/otel` and [maps `gen_ai.*`, OpenInference and OpenLLMetry attributes](https://docs.langchain.com/langsmith/trace-with-opentelemetry). The TypeScript SDK is the `langsmith` package, with a [Vercel AI SDK integration](https://docs.langchain.com/langsmith/trace-with-vercel-ai-sdk): `wrapAISDK` for AI SDK 5 and 6, `LangSmithTelemetry` for AI SDK 7. LangSmith [alerts](https://docs.langchain.com/langsmith/alerts) trigger on error count or rate, latency, feedback scores, run count and cost, and go to Slack, PagerDuty, Dynatrace or a webhook.

LangSmith is the only dedicated tracing tool here that also runs agents. [Agent Server](https://docs.langchain.com/langsmith/agent-server) puts runs on a task queue, writes a checkpoint at each step, and resumes from the last checkpoint if a worker is interrupted. It runs LangGraph natively and can deploy Strands, Claude Agent SDK and Google ADK agents through the Functional API or a wrapper package. Runs can pause for [human input](https://docs.langchain.com/langsmith/add-human-in-the-loop) and resume, threads have [double-texting](https://docs.langchain.com/langsmith/double-texting) strategies, and LangGraph [time travel](https://docs.langchain.com/oss/python/langgraph/use-time-travel) replays or forks a run from any past checkpoint.

[LangSmith pricing](https://www.langchain.com/pricing): Developer is free for 1 seat and 5,000 base traces a month. Plus is $39 per seat with 10,000 base traces, then usage-based, and includes one free small serverless deployment. Base traces are kept 14 days and extended traces 180 days. Self-hosting is [an Enterprise option](https://docs.langchain.com/langsmith/self-hosted), either fully self-hosted on Kubernetes with ClickHouse, Postgres and Redis, or hybrid with your own data plane. A [standalone Agent Server](https://docs.langchain.com/langsmith/deploy-standalone-server) can run on Docker with a license key.

**Best fit:** teams who want tracing, online and offline evals and a managed agent runtime from one vendor, especially on LangGraph.

**Limits:** LangSmith isn't open source, and self-hosting needs Enterprise. Agent Server's checkpointing and time travel are LangGraph features, so other frameworks get them only as far as the wrapper maps to LangGraph's model.

## Langfuse

[Langfuse](https://github.com/langfuse/langfuse) is open-source tracing, evals and prompt management, and has been part of ClickHouse since January 2026. Langfuse has 10 [observation types](https://langfuse.com/docs/observability/features/observation-types), including agent, tool, generation and guardrail, plus sessions, datasets, experiments, annotation queues and a playground. [LLM-as-a-judge](https://langfuse.com/docs/evaluation/evaluation-methods/llm-as-a-judge) and code evaluators can score live production traffic with sampling. Langfuse [agent graphs](https://langfuse.com/docs/observability/features/agent-graphs) draw the steps an agent took, and you can filter traces by tool name or tool call count. Custom dashboards and a [Metrics API](https://langfuse.com/docs/metrics/overview) break down cost, latency and scores by user, model or prompt version.

Langfuse takes [OpenTelemetry over HTTP](https://langfuse.com/docs/opentelemetry/get-started) at `/api/public/otel` in EU, US, Japan and HIPAA regions, and aims to follow the GenAI semantic conventions. The TypeScript SDK is `@langfuse/tracing`, and the [Vercel AI SDK integration](https://langfuse.com/integrations/frameworks/vercel-ai-sdk) uses `experimental_telemetry` on AI SDK 6 or `@langfuse/vercel-ai-sdk` on AI SDK 7. You can open any Langfuse generation in the playground and re-run that one call. Langfuse [alerts](https://langfuse.com/docs/observability/features/alerts) fire on thresholds for latency, counts, cost or numeric, categorical and boolean scores, and go to Slack, a signed webhook or GitHub Actions.

Langfuse is MIT licensed except the `ee` folders, and all core features are [free to self-host](https://langfuse.com/self-hosting) without limits. Add-ons like audit logs, SCIM and project-level RBAC need a license key. Production self-hosting runs web and worker containers with Postgres, Redis, ClickHouse and S3-compatible storage, with Helm and AWS, Azure and GCP guides; Docker Compose works for a single-VM test setup.

[Langfuse Cloud](https://langfuse.com/pricing) Hobby is free for 50,000 units a month (a unit is any trace, observation or score), 2 users and 30 days of data, with no overage. Core is $29 a month with 100,000 units, unlimited users and 90 days. Pro is $199 with 3 years of data, SOC 2 Type II and ISO 27001 reports and HIPAA support. Extra usage starts at $8 per 100,000 units and falls to $6 at higher volume.

**Best fit:** teams who want an open-source tool covering tracing, online and offline evals, prompt management and dashboards, with the option to run it themselves.

**Limits:** production self-hosting means operating ClickHouse and three other stateful services. On Cloud, the free plan is a hard cap and compliance reports start at Pro.

## Langfuse vs LangSmith

Langfuse is open source and its core is free to self-host. LangSmith is proprietary, and self-hosting it needs Enterprise. Both trace apps built without LangChain, through OpenTelemetry and the Vercel AI SDK, and both run online LLM-as-a-judge evals on production traffic. LangSmith also runs agents through Agent Server and adds LangGraph time travel, while Langfuse only records and evaluates them. If you use LangGraph and want the runtime from the same vendor, LangSmith fits better. If you want to own your data on an open-source stack, Langfuse fits better.

## Arize Phoenix

[Arize Phoenix](https://github.com/Arize-ai/phoenix) is Arize's tool for tracing, evals and experiments. Phoenix has OTel tracing, sessions, versioned datasets, experiments, human annotations, LLM evals in Python and TypeScript, a prompt playground, prompt management, a remote MCP server and a debugging agent. It draws a [graph view of multi-agent systems](https://arize.com/docs/phoenix/cookbook/tracing/openinference-best-practices) from graph attributes that LangGraph, CrewAI, AutoGen and the OpenAI Agents SDK set automatically, and shows tool calls as their own spans. Phoenix [Span Replay](https://arize.com/docs/phoenix/prompt-engineering/overview-prompts/span-replay) loads a stored LLM span into the playground so you can edit and re-run that one call.

Phoenix is built on [OpenInference](https://github.com/Arize-ai/openinference), an Apache 2.0 tracing convention with auto-instrumentation for 30+ Python libraries and about 10 TypeScript ones, plus Java and Go, that works with any OTel backend. Since 15.10.0 Phoenix also [converts `gen_ai.*` attributes automatically](https://arize.com/docs/phoenix/release-notes/05-2026/05-15-2026-otel-semconv-conversion). TypeScript support is `@arizeai/phoenix-otel`, with a [Vercel AI SDK integration](https://arize.com/docs/phoenix/integrations/typescript/vercel/vercel-ai-sdk-tracing-js) through OpenInference.

Phoenix is [free to self-host](https://arize.com/docs/phoenix/self-hosting) with no feature limits, and runs as one container on [SQLite or Postgres](https://arize.com/docs/phoenix/self-hosting/architecture), with no ClickHouse. It supports OAuth2, LDAP and role-based access. The [Phoenix license](https://arize.com/docs/phoenix/self-hosting/license) is Elastic License 2.0: self-hosting is fully permitted, but you can't offer Phoenix to others as a hosted service. Threshold [monitors](https://arize.com/docs/ax/observe/production-monitoring) with Slack, PagerDuty and webhook alerts are in Arize AX, which has a free tier of 25,000 spans a month with 15 days of retention, and a $50 Pro tier with 50,000 spans and 30 days ([pricing](https://arize.com/pricing/)).

**Best fit:** teams who want to self-host tracing and evals with the least infrastructure, or who want broad auto-instrumentation through OpenInference.

**Limits:** open-source Phoenix has no threshold alerts on latency, errors, cost or eval scores; those are in Arize AX. ELv2 isn't an OSI-approved license.

## Braintrust

[Braintrust](https://docs.braintrust.dev/) is an observability and evals platform for agents. You write `Eval()` in TypeScript or Python with a dataset, a task and [scorers](https://docs.braintrust.dev/evaluate/llm-as-a-judge) (autoevals, your own code, or LLM-as-a-judge), and run it with `bt eval` or in CI with the [eval GitHub Action](https://github.com/braintrustdata/eval-action), which comments results on the pull request. Each eval run is an immutable experiment you can compare against the last. The same scorers can [score production traffic online](https://docs.braintrust.dev/evaluate/score-online).

Braintrust [traces](https://docs.braintrust.dev/observe/examine-traces) show a span tree with typed LLM and tool spans, a timeline view and a thread view for whole sessions. [Logs](https://docs.braintrust.dev/observe/view-logs) can be queried with SQL, and Loop, Braintrust's built-in assistant, answers questions about them in plain language. [Topics](https://www.braintrust.dev/docs/observe/topics) clusters production traces by task and issue. Braintrust also has prompt management (environments on Pro and above), playgrounds, human review, custom dashboards, and an LLM gateway with caching and provider fallbacks. Braintrust [alerts](https://docs.braintrust.dev/observe/alerts) fire on any SQL filter or aggregate over your logs, including score thresholds, or when a prompt or dataset environment changes, and go to Slack, PagerDuty or a webhook. The TypeScript and Python SDKs are Apache 2.0, with SDKs for Go, Ruby, Java and C# too. The TypeScript SDK [wraps the Vercel AI SDK](https://docs.braintrust.dev/integrations/sdk-integrations/vercel), and Braintrust takes [OTLP traces and logs](https://docs.braintrust.dev/integrations/sdk-integrations/opentelemetry) over HTTP.

[Braintrust pricing](https://www.braintrust.dev/pricing): Starter is free for unlimited users with 1 GB of processed data, 10,000 scores and 14 days of retention, then $4 per GB and $2.50 per 1,000 scores. Pro is $249 a month for unlimited users with 5 GB, 50,000 scores and 30 days, which you can extend. [Self-hosting](https://docs.braintrust.dev/guides/self-hosting) is Enterprise: the data plane runs in your AWS, GCP or Azure account while Braintrust hosts the control plane, or Braintrust runs it in your cloud (BYOC).

**Best fit:** teams where evals decide what ships, so every prompt or model change runs against a dataset in CI before it goes out.

**Limits:** self-hosting the data plane needs Enterprise, and free-tier data is kept 14 days.

## Datadog Agent Observability

[Datadog Agent Observability](https://docs.datadoghq.com/llm_observability/), formerly LLM Observability, records traces with span kinds for LLM calls, workflows, agents, tools, tasks, embeddings and retrieval, and shows agent runs as an execution graph. Prebuilt [agent monitoring](https://docs.datadoghq.com/llm_observability/monitoring/agent_monitoring/) views cover the OpenAI Agents SDK, LangGraph and CrewAI in Python. In Node.js, `dd-trace` has [agent instrumentation](https://docs.datadoghq.com/llm_observability/instrument/agentic/nodejs/) and a [Vercel AI SDK integration](https://docs.datadoghq.com/integrations/vercel-ai-sdk/).

Datadog has managed and custom LLM-as-a-judge [evaluations](https://docs.datadoghq.com/llm_observability/evaluations/), with [templates](https://docs.datadoghq.com/llm_observability/investigate/evaluations/llm_as_a_judge_evaluations/template_evaluations/) for prompt injection, toxicity, hallucination and tool selection, plus [datasets and experiments](https://docs.datadoghq.com/llm_observability/experiments/), a playground and Sensitive Data Scanner for redaction. [Prompt management](https://docs.datadoghq.com/llm_observability/configure/prompt_management/) is a registry with versions, environments and A/B tests (fetching prompts at runtime is Python-only). A built-in dashboard covers cost, latency and usage, Patterns clusters production traffic by topic, and Datadog's [MCP server](https://docs.datadoghq.com/llm_observability/mcp_server/) lets a coding agent search traces, find error spans and review experiments.

Datadog's advantage is the rest of the Datadog platform. The SDK is built on `dd-trace`, so you can [move between LLM spans and APM spans](https://docs.datadoghq.com/llm_observability/instrument/agent_observability_and_apm/) of the same service, and its [metrics](https://docs.datadoghq.com/llm_observability/monitoring/metrics/) drive the same dashboards and monitors as the rest of your stack. SDKs cover Python, Node.js and Java, with Go experimental. Datadog also [accepts OTLP over HTTP](https://docs.datadoghq.com/llm_observability/instrumentation/otel_instrumentation/) directly, with no Datadog Agent, mapping GenAI semantic conventions 1.37+, OpenInference and OpenLLMetry.

[Datadog pricing](https://www.datadoghq.com/pricing/?product=llm-observability) counts LLM spans only, not tool, workflow or retrieval spans. The free tier covers 40,000 LLM spans a month with 15 days of retention. Pro includes 100,000 LLM spans for $160 a month on an annual plan ($200 month to month, $240 on demand), with overage from $3.50 per 10,000. Retention can be extended to 30, 60 or 90 days as a paid add-on.

**Best fit:** teams already on Datadog who want agent traces, evals and cost next to their service traces, logs and monitors.

**Limits:** data lives in Datadog's SaaS, with no self-hosted option. Prebuilt framework views are Python-only, and rebuilding an exact prompt from a span in the playground needs prompt tracking set up.

## Trigger.dev

[Trigger.dev](https://trigger.dev) runs TypeScript tasks on managed machines, with retries, queues and checkpointed waits. Because Trigger.dev executes the code, it records every run.

**Run traces.** Each Trigger.dev run has a trace built on [OpenTelemetry](https://trigger.dev/docs/logging), with automatic spans for triggers, attempts and HTTP requests. `console.*` and the structured `logger` both land in it, and you can add your own spans with `logger.trace()`. You can add OpenTelemetry instrumentations and export traces, logs and metrics to any OTLP backend from [`trigger.config.ts`](https://trigger.dev/docs/config/config-file).

**LLM calls.** Vercel AI SDK calls with telemetry enabled are captured [as spans in the run trace](https://trigger.dev/docs/ai/observability), with model, token usage, cost and latency, and Messages and Tools tabs in the span inspector. `chat.agent()` turns this on for you. Each call is also written to an `llm_metrics` table you can query. Trigger.dev [prompts](https://trigger.dev/docs/ai/prompts) are versioned on deploy, you can override text or model from the dashboard without redeploying, and each prompt version shows its generations, tokens, cost and latency.

**Querying, dashboards and alerts.** [TRQL](https://trigger.dev/docs/observability/query) is a SQL-style query language over runs, metrics and LLM metrics, and custom [dashboards](https://trigger.dev/docs/observability/dashboards) are built from those queries. Trigger.dev [alerts](https://trigger.dev/docs/troubleshooting-alerts) fire on run failures, deployment failures and successes, and new or regressed error groups, by email, Slack or webhook. Alerts on approaching limits are in private beta, and [billing alerts](https://trigger.dev/docs/billing-limits) email you when spend crosses a threshold. [Realtime](https://trigger.dev/docs/realtime/overview) streams run status, metadata and AI tokens to your frontend. The [MCP server](https://trigger.dev/docs/mcp-tools) lets a coding agent list runs, read full traces, run TRQL queries and replay runs.

**Evals.** Evals run as ordinary tasks, so a batch of model calls or an LLM judge gets the same retries, parallelism and traces as everything else. The [Batch LLM Evaluator](https://trigger.dev/docs/guides/example-projects/batch-llm-evaluator) example runs several models in parallel and streams the results to a frontend, and the [evaluator-optimizer guide](https://trigger.dev/docs/guides/ai-agents/translate-and-refine) has one LLM judge another's output in a loop. You write the scoring and store the results yourself.

**Retries and waits.** When a Trigger.dev run retries, [`run()` executes again from the top](https://trigger.dev/docs/idempotency). `wait.for` and `wait.until` [don't count toward compute](https://trigger.dev/docs/wait) once they're over 5 seconds, and waits on child tasks or tokens aren't billed. Waits over 60 seconds checkpoint the process and free the machine. The [Replay](https://trigger.dev/docs/replaying) button starts a new run with the same payload, or an edited one, against the latest deployed version. There's no replay from a past step.

**Hosting and pricing.** Trigger.dev is [Apache 2.0](https://github.com/triggerdotdev/trigger.dev) and you can [self-host](https://trigger.dev/docs/self-hosting/overview) it with Docker or Kubernetes, but checkpoints are Cloud-only. [Trigger.dev pricing](https://trigger.dev/pricing) is per second of compute by machine size plus $0.25 per 10,000 runs. The free plan includes $5 of compute a month and 1 day of log retention. Hobby is $10 a month with $10 of compute and 7 days. Pro is $50 with $50 of compute and 30 days.

**Best fit:** TypeScript teams who need to know whether a long-running or waiting agent actually finished, and want Vercel AI SDK calls, tokens and cost in the same trace.

**Limits:** Trigger.dev has no built-in datasets, scorers, experiment comparison or online evals; you can build evals as tasks, but you write the scoring and storage. It can't re-run a single logged model call. Automatic LLM capture only covers the Vercel AI SDK, so raw `fetch` calls or a provider's own SDK need your own spans. There are no alerts on latency or LLM cost thresholds. Tasks are TypeScript; Python scripts run through the [Python build extension](https://trigger.dev/docs/config/extensions/pythonExtension), not a Python SDK.

## Comparison matrix

### Observing model calls

| | LangSmith | Langfuse | Arize Phoenix | Braintrust | Datadog | Trigger.dev |
| --- | --- | --- | --- | --- | --- | --- |
| LLM call spans with tokens and cost | Yes | Yes | Yes | Yes | Yes | Yes, opt-in telemetry for Vercel AI SDK calls |
| Agent and tool views | Threads, Studio graph | Agent graphs, tool filters | Agent graph, sessions | Span tree with typed tool spans, threads | Execution graph; framework views in Python | Tools tab per span |
| Datasets and experiments | Yes | Yes | Yes | Yes | Yes | No |
| LLM-as-a-judge, offline | Yes | Yes | Yes | Yes | Yes | Build it as a task (examples, no built-in scorers) |
| Online evals on production traffic | Yes | Yes | Evals run over traced spans | Yes | Yes | No |
| Re-run a logged model call | Yes (Playground) | Yes (Playground) | Yes (Span Replay) | Yes (Playground) | Yes, with prompt tracking | No |
| Prompt management | Yes (prompt hub) | Yes | Yes | Yes (environments on Pro) | Yes (registry, environments) | Yes (versions, dashboard overrides) |
| Threshold alerts | Errors, latency, scores, run count, cost | Latency, counts, cost, scores | Arize AX monitors | SQL filters and aggregates | Datadog monitors on LLM metrics | No (run, deploy and error alerts) |
| OpenTelemetry | Ingests OTLP | Ingests OTLP over HTTP | Ingests OTLP | Ingests OTLP over HTTP | Ingests OTLP over HTTP | Built on it; exports OTLP |
| TypeScript SDK | `langsmith` | `@langfuse/tracing` | `@arizeai/phoenix-otel` | `braintrust` | `dd-trace` | `@trigger.dev/sdk` |
| Python SDK | Yes | Yes | Yes | Yes (also Go, Ruby, Java, C#) | Yes (also Java, Go) | No (Python scripts via extension) |

### Running the agent

| | LangSmith | Langfuse | Arize Phoenix | Braintrust | Datadog | Trigger.dev |
| --- | --- | --- | --- | --- | --- | --- |
| Runs and retries your agent | Yes, Agent Server (LangGraph native, other frameworks wrapped) | No | No | No | No | Yes, TypeScript tasks |
| Durable waits and human input | Checkpoint per step, interrupts | No | No | No | No | Checkpointed waits on Cloud, wait tokens |
| Replay from a past step | Yes (LangGraph time travel) | No | No | No | No | No (retries restart `run()`) |
| Run status, attempts, queues | On Agent Server | No | No | No | Via APM, if the runtime is instrumented | Yes |

### Hosting and price

| | LangSmith | Langfuse | Arize Phoenix | Braintrust | Datadog | Trigger.dev |
| --- | --- | --- | --- | --- | --- | --- |
| License | Proprietary | MIT (except `ee`) | Elastic License 2.0 | Proprietary (SDKs Apache 2.0) | Proprietary | Apache 2.0 |
| Self-hosting | Enterprise: full or hybrid | Free; you run Postgres, ClickHouse, Redis, S3 | Free; SQLite or Postgres | Enterprise: data plane in your cloud, or BYOC | No | Docker or Kubernetes, without checkpoints |
| Free tier | 5k traces, 1 seat | 50k units (hard cap), 2 users | Self-host free; AX 25k spans | 1 GB, 10k scores, unlimited users | 40k LLM spans | $5 compute credit; waits over 5s aren't billed |
| Free tier retention | 14 days | 30 days | Self-host: you set it; AX: 15 days | 14 days | 15 days | 1 day of logs |

## Using Trigger.dev with an eval tool

If you want Trigger.dev's run view and another tool's evals, keep one ID in both places. The task below turns on AI SDK telemetry, so Trigger.dev records the call in the run trace, and puts the run ID in the telemetry metadata so you can search for it in the other tool.

```ts
import { task } from "@trigger.dev/sdk";
import { generateText } from "ai";
import { anthropic } from "@ai-sdk/anthropic";

export const triageTicket = task({
  id: "triage-ticket",
  retry: { maxAttempts: 3 },
  run: async (payload: { ticketId: string; body: string }, { ctx }) => {
    const { text } = await generateText({
      model: anthropic("claude-sonnet-4-5"),
      prompt: `Classify this support ticket:\n\n${payload.body}`,
      experimental_telemetry: {
        isEnabled: true,
        functionId: "triage-ticket",
        metadata: { triggerRunId: ctx.run.id, ticketId: payload.ticketId },
      },
    });

    return { category: text };
  },
});
```

This is AI SDK 6. On AI SDK 7, register telemetry with `@ai-sdk/otel` first, as the [AI observability docs](https://trigger.dev/docs/ai/observability) show. How each tool stores AI SDK metadata differs (Langfuse, for example, documents `propagateAttributes()` for trace-level metadata), so check that the field shows up where you expect it.

To send Trigger.dev's spans to another backend, add an OTLP exporter in `trigger.config.ts`. This one points at Langfuse's EU endpoint (US, Japan and HIPAA regions have their own hosts), with Basic auth from your public and secret keys:

```ts
import { defineConfig } from "@trigger.dev/sdk";
import { OTLPTraceExporter } from "@opentelemetry/exporter-trace-otlp-http";

const auth = Buffer.from(
  `${process.env.LANGFUSE_PUBLIC_KEY}:${process.env.LANGFUSE_SECRET_KEY}`
).toString("base64");

export default defineConfig({
  project: "<your-project-ref>",
  maxDuration: 3600,
  telemetry: {
    exporters: [
      new OTLPTraceExporter({
        url: "https://cloud.langfuse.com/api/public/otel/v1/traces",
        headers: {
          Authorization: `Basic ${auth}`,
          "x-langfuse-ingestion-version": "4",
        },
      }),
    ],
  },
});
```

Exporters have to be set in the config file, not through `OTEL_*` environment variables. This exporter sends every span in the run, not only model calls, and Langfuse bills each one as a unit. If you only want model calls in your eval tool, use that tool's own integration instead: Langfuse's `LangfuseSpanProcessor` exports only GenAI spans by default, Braintrust has `wrapAISDK`, and LangSmith has `wrapAISDK` (AI SDK 5 and 6) or `LangSmithTelemetry` (AI SDK 7). Neither Trigger.dev nor Langfuse documents this pairing yet, so test it before you rely on it.

A retry runs `run()` again from the top, so a model call can happen twice and show up twice in your LLM tool. The `triggerRunId` metadata groups both attempts. For side effects like charging a card or sending an email, pass the provider's own idempotency key on the API call, for example one built from `ctx.run.id`. Moving the call into a child task with a Trigger.dev [idempotency key](https://trigger.dev/docs/idempotency) stops the child from being triggered twice, but the child's own `run()` can still retry after the provider has accepted the request.

## Choosing a setup

**Evals first, Python or TypeScript.** LangSmith, Langfuse or Braintrust for tracing, online scoring and datasets, plus whatever runs your agent today.

**LangGraph.** LangSmith for tracing and evals, and Agent Server if you want LangChain to run it, including time travel from past checkpoints.

**Already on Datadog.** Datadog Agent Observability for agent traces and evals next to APM, plus whatever runs your agent.

**You must self-host everything.** Langfuse (MIT core) or Phoenix (ELv2, lighter to run) for tracing and evals. If you also need a runtime, Trigger.dev is Apache 2.0, but self-hosted Trigger.dev doesn't checkpoint, so long waits hold a machine.

**TypeScript agent on the Vercel AI SDK, no eval program yet.** Trigger.dev covers runs, retries, waits, model calls, tokens and cost in one trace. Add an eval tool when you start testing prompt changes against a dataset.

**TypeScript agent with evals that gate releases.** Trigger.dev for execution, plus Braintrust, Langfuse or LangSmith for datasets and scoring, joined on the run ID.

## FAQs

- **What's the best open-source LLM observability tool?** Langfuse is MIT licensed (except its `ee` folders), its core features are free to self-host, and it covers tracing, online and offline evals, prompt management, dashboards and alerts; production self-hosting needs Postgres, ClickHouse, Redis and S3. Arize Phoenix is lighter to run, on SQLite or Postgres, and free to self-host with every feature under Elastic License 2.0, but threshold alerts are in Arize AX.

- **What are the main LangSmith alternatives?** Langfuse if you want open source and self-hosting, Braintrust if evals and CI gating drive your releases, Arize Phoenix for a lightweight self-hosted tracer, and Datadog Agent Observability if you already run Datadog APM.

- **Is Trigger.dev a replacement for LangSmith, Langfuse or Phoenix?** Only partly. If you call models through the Vercel AI SDK with telemetry turned on, Trigger.dev records those calls, tokens and cost inside the run trace. To link each call to its prompt version, define the prompt with AI Prompts and pass `toAISDKTelemetry()` into the call. You can run evals as Trigger.dev tasks, but there are no built-in datasets, scorers, experiment comparison, online evals or session views, so if you grade outputs regularly you'll want one of them alongside it.

- **Does Trigger.dev's checkpointing mean deterministic replay?** No. Trigger.dev snapshots the process during a wait and restores it afterward, so code before the wait doesn't run again. A retry starts `run()` from the top, and the Replay button starts a new run. If you need to replay or fork a run from a past step, LangGraph's time travel does that on LangSmith Agent Server.

- **What happens to an in-flight LLM call when a Trigger.dev run retries?** It runs again, because `run()` starts from the top. Model calls are usually safe to repeat. For payments, emails and other side effects that must not repeat, pass the provider's own idempotency key on the request.

- **Can Datadog alone cover agent observability?** Datadog doesn't run your agent. If the runtime is instrumented with APM, its retries and errors appear next to the LLM spans; durable waits and checkpoints show up only as far as that runtime emits them.

- **Which tools can I self-host for free?** Trigger.dev (Apache 2.0), Langfuse (MIT core) and Arize Phoenix (ELv2) can all be self-hosted for free. LangSmith and Braintrust self-hosting needs an Enterprise plan, and Datadog is SaaS only.
