> ## Documentation Index
> Fetch the complete documentation index at: https://trigger.dev/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# Crawl a URL using Firecrawl

> This example demonstrates how to crawl a URL using Firecrawl with Trigger.dev.

## Overview

Firecrawl is a tool for crawling websites and extracting clean markdown that's structured in an LLM-ready format.

Here are two examples of how to use Firecrawl with Trigger.dev:

## Prerequisites

* A project with [Trigger.dev initialized](/quick-start)
* A [Firecrawl](https://firecrawl.dev/) account

## Example 1: crawl an entire website with Firecrawl

This task crawls a website and returns the `crawlResult` object. You can set the `limit` parameter to control the number of URLs that are crawled.

```ts trigger/firecrawl-url-crawl.ts theme={"theme":"css-variables"}
import Firecrawl from "@mendable/firecrawl-js";
import { task } from "@trigger.dev/sdk";

// Initialize the Firecrawl client with your API key
const firecrawlClient = new Firecrawl({
  apiKey: process.env.FIRECRAWL_API_KEY, // Get this from your Firecrawl dashboard
});

export const firecrawlCrawl = task({
  id: "firecrawl-crawl",
  run: async (payload: { url: string }) => {
    const { url } = payload;

    // Crawl: scrapes all the URLs of a web page and return content in LLM-ready format
    const crawlResult = await firecrawlClient.crawl(url, {
      limit: 100, // Limit the number of URLs to crawl
      scrapeOptions: {
        formats: ["markdown", "html"],
      },
    });

    if (crawlResult.status === "failed") {
      throw new Error(`Failed to crawl: ${url}`);
    }

    return {
      data: crawlResult,
    };
  },
});
```

### Testing your task

You can test your task by triggering it from the Trigger.dev dashboard.

```json theme={"theme":"css-variables"}
"url": "<url-to-crawl>" // Replace with the URL you want to crawl
```

## Example 2: scrape a single URL with Firecrawl

This task scrapes a single URL and returns the `scrapeResult` object.

```ts trigger/firecrawl-url-scrape.ts theme={"theme":"css-variables"}
import Firecrawl from "@mendable/firecrawl-js";
import { task } from "@trigger.dev/sdk";

// Initialize the Firecrawl client with your API key
const firecrawlClient = new Firecrawl({
  apiKey: process.env.FIRECRAWL_API_KEY, // Get this from your Firecrawl dashboard
});

export const firecrawlScrape = task({
  id: "firecrawl-scrape",
  run: async (payload: { url: string }) => {
    const { url } = payload;

    // Scrape: scrapes a URL and get its content in LLM-ready format (markdown, structured data via LLM Extract, screenshot, html)
    const scrapeResult = await firecrawlClient.scrape(url, {
      formats: ["markdown", "html"],
    });

    return {
      data: scrapeResult,
    };
  },
});
```

### Testing your task

You can test your task by triggering it from the Trigger.dev dashboard.

```json theme={"theme":"css-variables"}
"url": "<url-to-scrape>" // Replace with the URL you want to scrape
```