Most AI browser agents fail in production because websites detect and block them. Here is what triggers detection and what actually works.

The Problem

Browser Use lets AI agents control a real Chromium browser -- clicking, typing, navigating, and extracting data just like a human would. The gap that most tutorials skip: modern websites detect headless browsers and bot traffic, often blocking or serving degraded content to them before your agent has a chance to do anything useful.

Even on the WebArena benchmark, state-of-the-art browser agents succeed on only about 35% of real-world web tasks. A significant portion of those failures are detection-related. This article covers the detection mechanisms, which ones Browser Use handles by default, and what you need to configure for the sites that don't cooperate.

How Websites Detect Headless Browsers

Detection method What it checks Browser Use default handling
User-Agent string Headless Chromium has a distinct UA string Partially -- configure explicitly
navigator.webdriver JS property set to true in automation contexts Handled by Browser Use by default
Missing browser fingerprint Plugins, screen resolution, hardware concurrency Partial -- needs configuration
Mouse movement patterns Bots move in straight lines, humans don't Not handled -- you add this
Request timing Bots request pages too fast Not handled -- add delays
CAPTCHA / Cloudflare Active challenge pages Not handled -- needs service

Step 1: Configure a Realistic User Agent

from browser_use import Agent, Browser, ChatAnthropic
 
# Use a real, up-to-date Chrome user agent string.
# Match the Chrome version on your system for consistency.
browser = Browser(
    user_agent=(
        "Mozilla/5.0 (Windows NT 10.0; Win64; x64) "
        "AppleWebKit/537.36 (KHTML, like Gecko) "
        "Chrome/131.0.0.0 Safari/537.36"
    ),
)
 
agent = Agent(
    task="Find the latest pricing for the Pro plan",
    llm=ChatAnthropic(model="claude-sonnet-5"),
    browser=browser,
)
 
await agent.run()

Step 2: Launch Real Chrome Instead of Bundled Chromium

Browser Use already hides the most common automation signal: it launches Chrome with --disable-blink-features=AutomationControlled, so navigator.webdriver is not exposed. The bigger fingerprint win is to run your real, installed Chrome with a persistent user-data profile instead of the bundled Chromium build -- that gives the agent a genuine, consistent fingerprint (fonts, plugins, GPU, timezone) that headless Chromium cannot fake. Browser Use does not ship a JavaScript stealth-patcher in the open-source package; for JS-fingerprint-level evasion on hard sites, use a managed cloud browser or stealth backend (Step 5).

from browser_use import Agent, Browser, ChatAnthropic
 
# Run your real, installed Chrome with a persistent profile instead of the
# bundled headless Chromium. A genuine profile carries a consistent
# fingerprint (fonts, plugins, GPU, timezone) that headless Chromium lacks.
browser = Browser(
    channel="chrome",   # use installed Google Chrome, not bundled Chromium
    user_data_dir="~/.config/browseruse/profiles/stealth",
    headless=False,     # headful is far less detectable than headless
    viewport={"width": 1366, "height": 768},  # common real-world size
)
 
agent = Agent(
    task="Extract product data from the catalogue",
    llm=ChatAnthropic(model="claude-sonnet-5"),
    browser=browser,
)
 
await agent.run()

Step 3: Add Human-Like Timing

Bots access pages at machine speed -- instantly. Humans pause, scroll, and take time between actions. Adding small randomised delays between agent actions significantly reduces detection on timing-sensitive sites.

import asyncio
import random
from browser_use import Agent, Browser, ChatAnthropic
 
# Option A: a fixed pause between every action (built-in).
browser = Browser(wait_between_actions=1.5)
 
# Option B: randomised, human-like pauses via a step hook. Random timing is
# harder to fingerprint than a fixed interval.
async def human_pause(agent):
    await asyncio.sleep(random.uniform(0.5, 2.5))
 
agent = Agent(
    task="Log in and check account balance",
    llm=ChatAnthropic(model="claude-sonnet-5"),
    browser=browser,
)
 
await agent.run(on_step_start=human_pause)
For sites that do timing analysis, combine random delays with occasional longer pauses (3-8 seconds) to simulate reading or thinking time. Pure random is better than fixed delays, which can themselves be detected as a pattern.

Step 4: Handle CAPTCHAs

Browser Use cannot solve CAPTCHAs by itself. For sites that serve CAPTCHAs, you have three options:

Option How it works Cost/complexity
2captcha / Anti-Captcha API Human solvers answer CAPTCHAs via API in 10-30 seconds Low cost (~$1-3 per 1000), easy to integrate
Capsolver / NopeCHA AI-based CAPTCHA solver, faster than human services Low cost, very fast, good reCAPTCHA/hCaptcha coverage
Managed browser service (BrowserBase, Steel) Pre-warmed browsers with built-in CAPTCHA handling and residential proxies Higher cost, zero config 
# Integrating a CAPTCHA solver callback
from browser_use import Agent
import httpx
 
async def solve_captcha_callback(page):
    # Detect and solve reCAPTCHA if present
    captcha_present = await page.query_selector(".g-recaptcha")
    if captcha_present:
        # Get site key
        site_key = await page.get_attribute(".g-recaptcha", "data-sitekey")
        page_url = page.url
 
        # Submit to 2captcha (example)
        async with httpx.AsyncClient() as client:
            response = await client.post(
                "http://2captcha.com/in.php",
                data={"key": "YOUR_2CAPTCHA_KEY", "method": "userrecaptcha",
                      "googlekey": site_key, "pageurl": page_url}
            )
            task_id = response.text.split("|")[1]
 
            # Poll for result (2captcha is async)
            for _ in range(30):
                await asyncio.sleep(5)
                result = await client.get(
                    f"http://2captcha.com/res.php?key=YOUR_2CAPTCHA_KEY&action=get&id={task_id}"
                )
                if result.text.startswith("OK|"):
                    token = result.text.split("|")[1]
                    # Inject the token into the page
                    await page.evaluate(
                        f'document.getElementById("g-recaptcha-response").value = "{token}"'
                    )
                    break

Step 5: Use a Cloud Browser Service for Hard Sites

For sites with aggressive bot protection (Cloudflare Enterprise, Akamai Bot Manager, PerimeterX), no amount of local stealth configuration will reliably work. These services use hundreds of signals including TLS fingerprinting, IP reputation, and behavioral analytics that are impossible to spoof from a local Playwright instance.

For these sites, route your Browser Use agent through a managed cloud browser service:

  • BrowserBase: managed cloud Chromium, residential proxies, CAPTCHA handling, SOC 2 compliant
  • Steel.dev: open-source self-hosted alternative, CDP-compatible, works with Browser Use directly
  • Browserless: managed Chrome-as-a-service, drops in as a Playwright replacement
from browser_use import Agent, Browser, ChatAnthropic
 
# Connect Browser Use to a managed cloud Chrome over CDP.
browser = Browser(
    cdp_url="wss://connect.browserbase.com?apiKey=YOUR_BROWSERBASE_KEY",
)
 
agent = Agent(
    task="Extract data from protected site",
    llm=ChatAnthropic(model="claude-sonnet-5"),
    browser=browser,
)
 
await agent.run()

Quick Reference

  • Set a realistic Chrome user-agent string via Browser(user_agent=...)
  • Run real Chrome with a persistent profile -- Browser(channel='chrome', user_data_dir=...) -- instead of bundled Chromium
  • Add delays with Browser(wait_between_actions=...) or randomise them with an on_step_start hook
  • For CAPTCHAs: use 2captcha, Capsolver, or NopeCHA API integration
  • For heavily protected sites (Cloudflare Enterprise): use BrowserBase or Steel.dev as the browser backend
  • Never run agents against sites that explicitly prohibit automated access -- check robots.txt and ToS