Get started

Scrapfly for AI agents

Scrapfly is a web scraping API with JavaScript rendering and anti-bot bypass, letting developers pull page content and screenshots without running their own browser infrastructure. Connect it once through Arc0, and your agent, or Claude, ChatGPT and Cursor, can use it through one MCP endpoint, limited to what each user approved.

Connect ScrapflySee the 12 actions ↓
Web data and scrapingAPI keyMCP + RESTscrapfly.io
AUDIT LOG · SCRAPFLYPOLICY: acme-support
09:41:07 · claude · u_8f2read
scrapfly.get_account_info
Get Scrapfly Account Information✓ allowed · 212ms
09:41:08 · claude · u_8f2read
scrapfly.scrape
Scrapfly Scrape✓ allowed · 164ms
09:41:09 · claude · u_8f2write
scrapfly.create_crawler
Create Scrapfly Crawler✓ approved · approved by user
EVERY SCRAPFLY CALL, ON THE RECORD
01 · USE CASES

What agents do in Scrapfly.

01

Capture a website screenshot

Renders and captures a screenshot of a target page for a report or visual record.

02

Check a crawler's status

Looks up whether a multi-page crawler job has finished before pulling its results.

03

Extract structured data from a page

Pulls structured fields, like prices or listings, out of a rendered webpage in one call.

02 · ACTIONS

12 Scrapfly actions, graded by risk.

Every Scrapfly action is tagged read, write or destructive, so one policy covers the whole app and new actions inherit the right default.

read

9

Look things up. Allowed by default.

  • scrapfly.get_account_info
    Get Scrapfly Account Information
  • scrapfly.get_crawler_urls
    Get Crawler URLs
  • scrapfly.get_crawler_status
    Get Crawler Status
  • scrapfly.get_crawler_artifact
    Get Crawler Artifact
  • scrapfly.get_crawler_contents
    Get Crawler Contents
  • scrapfly.scrape
    Scrapfly Scrape
  • scrapfly.scrape_post
    Scrapfly Scrape POST
  • scrapfly.extract_data
    Extract Structured Data
  • scrapfly.scrape_with_put
    Scrape With PUT

write

3

Create and change things. Allow, or ask the user first.

  • scrapfly.create_crawler
    Create Scrapfly Crawler
  • scrapfly.capture_screenshot
    Capture Website Screenshot
  • scrapfly.capture_screenshot_head
    Capture Screenshot Metadata (HEAD)

destructive

0

Delete, cancel or archive. Ask first, or deny outright.

  • No destructive actions.
03 · HOW IT WORKS

Scrapfly in three steps.

  1. 01Your users connect ScrapflyThey add their Scrapfly api key on Arc0 Connect, under your brand. It goes straight into the vault.
  2. 02You set the rulesReads run, and writes like “create Scrapfly Crawler” can wait for the user to approve.
  3. 03Any agent can actYour agent calls Scrapfly through the Arc0 SDK or MCP, and so can Claude, ChatGPT and Cursor. Every call lands on the audit log.
POLICY.TS
await arc0.policies.set('scrapfly', {
  read: 'allow',
  write: 'ask',        // create_crawler
  destructive: 'deny',  
})

# Claude Code: the same connection, one URL
$ claude mcp add --transport http arc0 \
    https://mcp.arc0.ai/u/u_8f2
04 · AUTH AND DATA

How Scrapfly connects.

Users add their Scrapfly api key on Arc0 Connect. It is encrypted in the vault, never shown to the model, and each user can rotate or revoke it at any time.

The same Scrapfly connection serves your agent over MCP and your own backend over REST and the proxy, so a user connects once. How Arc0 handles credentials →

AUTH
API key
CREDENTIALS
Per-tenant encrypted vault
MODEL SEES
Results only, never credentials
AUDIT LOG
Every call, on every plan
05 · WORKS WITH

Use Scrapfly from any agent.

Claude
ChatGPT
Cursor
Codex
VS Code
OpenAI Agents SDK
Claude Agent SDK
Vercel AI SDK
Mastra
LangGraph
06 · RELATED

More web data and scraping apps.

Firecrawl30 actionsTavily5 actionsExa20 actionsAgentql3 actionsAgenty79 actionsAirtop10 actionsJust One API288 actionsApify183 actions
ALL WEB DATA AND SCRAPING APPS →
07 · FAQ

Scrapfly and Arc0, answered.

Q01

Can I use Scrapfly with Claude, ChatGPT or Cursor?

Yes. Connect Scrapfly to Arc0 once, then add your Arc0 MCP URL to Claude, ChatGPT, Cursor, Claude Code or any other remote-MCP client. Each assistant only gets the Scrapfly actions you allow.

Q02

How do users connect Scrapfly?

Users add their Scrapfly api key on Arc0 Connect. It is encrypted in the vault, never shown to the model, and each user can rotate or revoke it at any time.

Q03

Which Scrapfly actions can my agent take?

12 in total: 9 read, 3 write and 0 destructive, such as “create Scrapfly Crawler”. Your policies decide which of them each agent may call.

Q04

Can I make my agent read-only in Scrapfly?

Yes. Allow read actions and deny writes in the Scrapfly policy. Your agent can still look things up, and any write it attempts is blocked and logged.

Q05

Can my own backend call Scrapfly too?

Yes. The same Scrapfly connection is available over REST and through the Arc0 proxy, so your product and your agent share one connection per user.

Get started

Plug Scrapfly into your agent.

Your users connect Scrapfly once, under your brand. Your agent gets 12 actions behind your policies, with every call on the record.

Connect ScrapflyBrowse all apps →

Free to build · MCP + REST · Audit log on every plan