Next.js Batch LLM Evaluator

​

Overview

This demo is a full stack example that uses the following:

• A Next.js app with Prisma for the database.
• Trigger.dev Realtime to stream updates to the frontend.
• Work with multiple LLM models using the Vercel AI SDK. (OpenAI, Anthropic, XAI)
• Distribute tasks across multiple tasks using the new batch.triggerByTaskAndWait method.

​

GitHub repo

​

Video

​

Relevant code

• View the Trigger.dev task code in the src/trigger/batch.ts file.
• The evaluateModels task uses the batch.triggerByTaskAndWait method to distribute the task to the different LLM models.
• It then passes the results through to a summarizeEvals task that calculates some dummy “tags” for each LLM response.
• We use a useRealtimeRunsWithTag hook to subscribe to the different evaluation tasks runs in the src/components/llm-evaluator.tsx file.
• We then pass the relevant run down into three different components for the different models:
  • The AnthropicEval component: src/components/evals/Anthropic.tsx
  • The XAIEval component: src/components/evals/XAI.tsx
  • The OpenAIEval component: src/components/evals/OpenAI.tsx
• Each of these components then uses useRealtimeRunWithStreams to subscribe to the different LLM responses.

​

Learn more about Trigger.dev Realtime

To learn more, take a look at the following resources:

• Realtime docs - learn more about Realtime.
• Batch Trigger docs - learn more about Batch Triggers.
• React hooks - learn more about React hooks.