Get started

Webscraper io for AI agents

Web Scraper.io runs cloud-based scraping jobs from configured sitemaps, used by data teams that extract structured data from websites on a schedule. Connect it once through Arc0, and your agent, or Claude, ChatGPT and Cursor, can use it through one MCP endpoint, limited to what each user approved.

Connect Webscraper ioSee the 10 actions ↓
Developer toolsAPI keyMCP + RESTwebscraper.io
AUDIT LOG · WEBSCRAPER IOPOLICY: acme-support
09:41:07 · claude · u_8f2read
webscraper_io.get_sitemap
Get Sitemap✓ allowed · 212ms
09:41:08 · claude · u_8f2read
webscraper_io.get_sitemaps
Get Sitemaps✓ allowed · 164ms
09:41:09 · claude · u_8f2write
webscraper_io.create_sitemap
Create Sitemap✓ approved · approved by user
09:41:12 · claude · u_8f2destructive
webscraper_io.delete_sitemap
Delete Sitemap✕ blocked · policy: deny
EVERY WEBSCRAPER IO CALL, ON THE RECORD
01 · USE CASES

What agents do in Webscraper io.

01

Get scraping jobs

Pull the status and results of scraping jobs as a read-only check.

02

Create a sitemap

Define a new sitemap describing which pages and fields to scrape.

03

Confirm before deleting a sitemap

Check with the user before deleting a sitemap, since its scheduled jobs stop running.

02 · ACTIONS

10 Webscraper io actions, graded by risk.

Every Webscraper io action is tagged read, write or destructive, so one policy covers the whole app and new actions inherit the right default.

read

5

Look things up. Allowed by default.

  • webscraper_io.get_sitemap
    Get Sitemap
  • webscraper_io.get_sitemaps
    Get Sitemaps
  • webscraper_io.get_account_info
    Get Account Info
  • webscraper_io.get_scraping_jobs
    Get Scraping Jobs
  • webscraper_io.get_sitemap_scheduler
    Get Sitemap Scheduler

write

3

Create and change things. Allow, or ask the user first.

  • webscraper_io.create_sitemap
    Create Sitemap
  • webscraper_io.update_sitemap
    Update Sitemap
  • webscraper_io.enable_sitemap_scheduler
    Enable Sitemap Scheduler

destructive

2

Delete, cancel or archive. Ask first, or deny outright.

  • webscraper_io.delete_sitemap
    Delete Sitemap
  • webscraper_io.disable_sitemap_scheduler
    Disable Sitemap Scheduler
03 · HOW IT WORKS

Webscraper io in three steps.

  1. 01Your users connect Webscraper ioThey add their Webscraper io api key on Arc0 Connect, under your brand. It goes straight into the vault.
  2. 02You set the rulesReads run, writes like “create Sitemap” can wait for the user, and “delete Sitemap” can be denied outright.
  3. 03Any agent can actYour agent calls Webscraper io through the Arc0 SDK or MCP, and so can Claude, ChatGPT and Cursor. Every call lands on the audit log.
POLICY.TS
await arc0.policies.set('webscraper_io', {
  read: 'allow',
  write: 'ask',        // create_sitemap
  destructive: 'deny',  // delete_sitemap
})

# Claude Code: the same connection, one URL
$ claude mcp add --transport http arc0 \
    https://mcp.arc0.ai/u/u_8f2
04 · AUTH AND DATA

How Webscraper io connects.

Users add their Webscraper io api key on Arc0 Connect. It is encrypted in the vault, never shown to the model, and each user can rotate or revoke it at any time.

The same Webscraper io connection serves your agent over MCP and your own backend over REST and the proxy, so a user connects once. How Arc0 handles credentials →

AUTH
API key
CREDENTIALS
Per-tenant encrypted vault
MODEL SEES
Results only, never credentials
AUDIT LOG
Every call, on every plan
05 · WORKS WITH

Use Webscraper io from any agent.

Claude
ChatGPT
Cursor
Codex
VS Code
OpenAI Agents SDK
Claude Agent SDK
Vercel AI SDK
Mastra
LangGraph
06 · RELATED

More developer tools apps.

GitHub896 actionsSupabase129 actions0CodeKit8 actionsAbly25 actionsAbstract3 actionsAddressfinder17 actionsAddresszen2 actionsAlchemy36 actions
ALL DEVELOPER TOOLS APPS →
07 · FAQ

Webscraper io and Arc0, answered.

Q01

Can I use Webscraper io with Claude, ChatGPT or Cursor?

Yes. Connect Webscraper io to Arc0 once, then add your Arc0 MCP URL to Claude, ChatGPT, Cursor, Claude Code or any other remote-MCP client. Each assistant only gets the Webscraper io actions you allow.

Q02

How do users connect Webscraper io?

Users add their Webscraper io api key on Arc0 Connect. It is encrypted in the vault, never shown to the model, and each user can rotate or revoke it at any time.

Q03

Which Webscraper io actions can my agent take?

10 in total: 5 read, 3 write and 2 destructive, such as “create Sitemap”. Your policies decide which of them each agent may call.

Q04

Can I stop my agent from deleting things in Webscraper io?

Yes. Actions like “delete Sitemap” are graded destructive. Set destructive actions to deny, or to ask so the user approves each one, and blocked calls still show up on the audit log.

Q05

Can my own backend call Webscraper io too?

Yes. The same Webscraper io connection is available over REST and through the Arc0 proxy, so your product and your agent share one connection per user.

Get started

Plug Webscraper io into your agent.

Your users connect Webscraper io once, under your brand. Your agent gets 10 actions behind your policies, with every call on the record.

Connect Webscraper ioBrowse all apps →

Free to build · MCP + REST · Audit log on every plan