• web_search sends a query to your configured provider (Brave by default) and returns results: title, URL, and snippet per result
• web_fetch takes a specific URL, does an HTTP fetch, and extracts readable content from the HTML as markdown or plain text
• Both tools are enabled together under group:web but can be allowlisted individually (web_search / web_fetch)
• web_fetch does not execute JavaScript, so JS-rendered pages return empty or incomplete content without a fallback
• Adding your Firecrawl API key gives web_fetch a real-browser fallback for pages Readability can't extract
• Installing the Firecrawl CLI skill adds a firecrawl search command that returns search results and full page content in a single step
• The Firecrawl /interact endpoint lets your agent act on a page after scraping it: click buttons, fill forms, and navigate to reach content that only appears after an interaction — something no search provider can do on its own

Send your OpenClaw agent a research task and the failure mode is predictable: web_search returns URLs, web_fetch tries to read them, and a lot of the modern web doesn't cooperate with plain HTTP requests.

This guide explains how the pipeline works, what breaks it, and how Firecrawl fixes it.

For a broader look at the full Firecrawl integration with OpenClaw including browser automation, see the OpenClaw + Firecrawl guide.

OpenClaw ships two distinct web tools: web_search and web_fetch. They serve different purposes and are configured separately. Both are enabled together under group:web but can be allowlisted individually.

web_search sends a search query to your configured provider and returns a list of results. With Brave (the default), each result is a structured object: title, URL, and a short snippet. Returns 5 results by default (configurable up to 10), cached for 15 minutes. The tool won't run without an API key. If none is configured, it returns a setup error rather than silently failing.

web_fetch takes a specific URL, makes a plain HTTP GET request, and extracts readable content from the HTML response as markdown or plain text. It does not execute JavaScript.

In practice, these two tools run in sequence. The agent searches for URLs, then fetches each one to read the content. But that handoff is where things break. Brave gives the agent URLs. web_fetch tries to read them. Many modern sites return JavaScript shells to plain HTTP requests: the HTML loads, but the meaningful content renders later in the browser. Others serve 403 errors to anything that doesn't look like an active browser session. web_fetch gets back an empty page or nothing, and the agent proceeds with whatever it has.

The internal extraction order for web_fetch is:

1. Readability: local main-content extraction from the raw HTML
2. Firecrawl: if an API key is configured, routes through Firecrawl's API with real browser rendering and bot circumvention
3. Basic HTML cleanup: strips tags and returns whatever text remains

If Readability fails and Firecrawl isn't configured, the agent falls through to basic cleanup, which often returns navigation links, cookie banners, and other noise instead of article content.

OpenClaw supports three built-in providers for web_search. If no provider is explicitly set, OpenClaw auto-detects based on which API keys are present, checking in order: Brave → Gemini → Perplexity → Grok. For a full comparison of OpenClaw search providers — including Firecrawl, Tavily, SearXNG, and pricing for each — see the dedicated guide. For a broader view of web search for AI agents — covering Exa, Tavily, and Perplexity's standalone APIs used outside OpenClaw — see the search engine comparison.

Firecrawl is not a web_search provider in the configuration sense above. It doesn't plug into the web_search tool. Instead it connects to the pipeline in two other ways: as a web_fetch fallback (via the API key config), and as the Firecrawl CLI skill, which gives your agent a firecrawl search command that runs search independently of the web_search tool entirely. More on both below.

Firecrawl connects to the OpenClaw web pipeline in two places. It's worth being precise about which is which, because they solve different problems.

Adding your Firecrawl API key to the web_fetch config gives it a second extraction attempt for pages where Readability fails. Instead of falling through to basic HTML cleanup, it routes the request through Firecrawl's API, which uses real browser rendering and bot circumvention automatically.

{
  "tools": {
    "web": {
      "fetch": {
        "firecrawl": {
          "apiKey": "fc-YOUR-API-KEY",
          "onlyMainContent": true,
          "maxAgeMs": 172800000
        }
      }
    }
  }
}

maxAgeMs controls how fresh cached results need to be (in milliseconds). The default is 2 days, fine for content that doesn't change often. For time-sensitive pages like pricing or release notes, lower this to force fresher fetches.

This configuration doesn't change how web_search works. The agent still searches via Brave and still calls web_fetch as a second step. But when web_fetch would otherwise fail on a JS-heavy or bot-protected page, Firecrawl catches it and returns actual content. See the OpenClaw Firecrawl docs for the full config reference.

The Firecrawl CLI skill changes the search step itself. Instead of web_search returning a list of URLs that the agent must then fetch individually, your agent runs firecrawl search, which returns search results and the scraped content of each result in a single call.

Install the skill with:

npx -y firecrawl-cli@latest init --all

Or install everything separately:

npm install -g firecrawl-cli
firecrawl init skills
export FIRECRAWL_API_KEY="fc-YOUR-API-KEY"

Verify the setup:

firecrawl --status

Once installed, your agent can run:

# Search and return top results
firecrawl search "OpenClaw release notes February 2026" --limit 10
 
# Search and return results with full scraped content
firecrawl search "OpenClaw release notes February 2026" --scrape --scrape-formats markdown --limit 5

Each result in the --scrape response includes the URL, title, description, and the full markdown content of the page. No separate web_fetch call needed, and no 403 errors, because Firecrawl handles the actual extraction. For a deeper look at what the search endpoint returns, see Mastering the Firecrawl Search Endpoint.

These two integrations are independent and serve different purposes:

You can run both at the same time. Use the API key config to harden web_fetch, and use the CLI skill when the task calls for search-first workflows with full page content.

The CLI skill also gives your agent scraping, crawling, and map capabilities for when search isn't the right tool.

# Scrape a single page
firecrawl https://example.com --only-main-content
 
# Scrape with specific formats
firecrawl https://example.com --format markdown,links --pretty

This is useful when you need to pull structured data from a known URL rather than find it first, or when you want to crawl an entire docs site and process the output.

Scraping stops at the page. Most of the web data agents actually care about sits behind a search form, a "load more" button, a login, or a filter dropdown. Static scraping — whether via web_fetch or firecrawl scrape — returns what the page renders on first load, and nothing else.

No search provider solves this. Brave, Perplexity, and Gemini all return links or synthesized answers. None of them let your agent take an action inside a live page and extract what comes back. web_fetch doesn't execute JavaScript. firecrawl scrape returns initial render. The problem isn't finding the page — it's what happens after you land on it.

Firecrawl's /interact endpoint addresses this directly. After scraping a page with firecrawl scrape, you stay in that browser session and tell it what to do next — in plain English or Playwright code:

# Scrape a page to open it
firecrawl scrape https://example.com/search
 
# Then interact with it
firecrawl interact "Search for 'quarterly earnings' and click the first result"
firecrawl interact "Extract the table on this page"
firecrawl interact stop

Sessions stay live for up to 10 minutes, and you can chain as many interaction calls as needed within that window.

For OpenClaw agents doing research that hits paywalls, paginated results, or login-gated content, /interact is the part of the pipeline that scraping alone can't cover.

OpenClaw's default is to drive a local browser. That works for simple workflows but the costs show up quickly: the agent runs in the same environment as your real browsing state, parallel sessions spike RAM, and runs get flaky under load. Local browsers behave like dev tooling, not infrastructure. Running agents in an isolated agent sandbox removes those risks — no shared state, no local resource pressure.

Firecrawl Browser Sandbox moves that work into a secure, remote, disposable environment. No local Chromium install, no driver setup. agent-browser and Playwright are pre-installed. Your OpenClaw agent can run on a free-tier EC2 instance or a Raspberry Pi while the actual browsing happens elsewhere.

Your agent just issues intent-level commands (open, click, fill, snapshot, scrape) through the firecrawl browser shorthand. Playwright is still available if you need it.

firecrawl browser "open https://news.ycombinator.com"
firecrawl browser "snapshot"
firecrawl browser "scrape"
firecrawl browser close

A few mechanics worth knowing:

• Shorthand auto-session: the shorthand form (firecrawl browser "...") auto-launches a sandbox session if one isn't active, so your agent doesn't need to manage session lifecycle up front
• Token efficiency: the agent gets back clean artifacts (snapshot, extracted content) instead of raw DOM or driver logs in the context window
• Context offloading: fetched pages and interactions are saved to the file system and queried only when needed

You can give your agent a prompt like: "Use Firecrawl Browser Sandbox to open Hacker News and get the top 5 stories and the first 10 comments on each." The agent figures out the rest.

See the Browser Sandbox docs for the full command reference.

A few patterns that get consistent results from the OpenClaw web pipeline:

Targeted queries over broad ones. Specific search terms outperform general ones. "OpenClaw changelog entries February 2026" gives the agent something actionable; "OpenClaw news" surfaces a mixed bag. When your agent's task depends on finding accurate, specific information, guide it toward narrow queries.

Multiple queries instead of one. A single broad query returns mixed results. Running two or three targeted queries in sequence, like "OpenClaw memory tool documentation" and "OpenClaw memory tool community issues" as separate calls, and then combining the results gives the agent better raw material than one catch-all query.

Use --scrape for content-heavy tasks. When the task requires reading actual page content rather than just titles and snippets, firecrawl search ... --scrape returns full markdown in one call and skips the web_fetch round-trip entirely.

Prompt examples that work well:

Search for the three most recent Firecrawl changelog entries and summarize what changed in each.

Find the pricing page for [product] and extract plan names, monthly prices, and any seat or usage limits.

Use Firecrawl to search for "OpenClaw memory tool site:github.com" and read the top result in full.

Find community discussion about OpenClaw search providers from the past week and summarize the most common complaints.

The web_search to web_fetch pipeline is the right mental model for understanding OpenClaw's web access. Each tool has a distinct role, and the failure modes are specific: the search provider delivers links, web_fetch tries to read them, and a lot of the modern web doesn't cooperate with plain HTTP requests.

Firecrawl addresses this at two levels. The API key config patches web_fetch for pages Readability can't handle, adding real browser rendering as a fallback with no change to how search works. The CLI skill goes further: firecrawl search replaces the two-step search-then-fetch pattern entirely, returning content alongside results in one call. And for pages that need an actual browser session, firecrawl browser handles interactive automation in a remote sandbox with no local Chromium required.

For the full configuration reference, the OpenClaw web tools docs cover every parameter for both web_search and web_fetch. And if you're setting up Firecrawl with OpenClaw for the first time, the OpenClaw + Firecrawl guide covers the broader integration including browser automation and deployment. To expand what your agent can do beyond web access, the best OpenClaw skills on ClawHub covers top picks across Gmail, GitHub, memory, and more.