olostep_web_crawl $0.25 per call

Crawl a website. Start a bounded asynchronous website crawl and pay only for completed pages. (Olostep · Open web & SEO data) Async task: answers 202 with a job id; poll GET /v1/jobs/<id>. Price depends on the options ($0.0025 to $0.25); refunded if the task fails. Prepaid credits only.

For AI agents: call olostep_web_crawl
POST https://toll402.dev/v1/p/olostep.web.crawl
{
 "start_url": "https://example.com",
 "max_pages": 1,
 "max_depth": 0,
 "follow_robots_txt": true
}
Answers 402 with x402 payment requirements; pay with any x402 client (@x402/fetch, x402 for Python, toll402-client, toll402-mcp) or send a prepaid-credits key as x-toll402-key (https://toll402.dev/credits). Paid per call via x402 (USDC on Base) or with a prepaid-credits key (x-toll402-key, card at /credits); charged only on success. Start at /llms.txt or connect the MCP server: npx -y toll402-mcp · remote https://toll402.dev/mcp.

Input schema

{
 "type": "object",
 "properties": {
  "start_url": {
   "type": "string",
   "examples": [
    "https://example.com"
   ]
  },
  "max_pages": {
   "type": "integer",
   "description": "treg reserves this many one-credit pages before submission; shared-key crawls are capped at 100 pages ($0.20 at the platform rate).",
   "examples": [
    10
   ]
  },
  "max_depth": {
   "type": "integer",
   "examples": [
    2
   ]
  },
  "follow_robots_txt": {
   "type": "boolean",
   "description": "Catalog crawls always respect robots.txt.",
   "enum": [
    true
   ]
  },
  "include_urls": {
   "type": "string"
  },
  "exclude_urls": {
   "type": "string"
  },
  "include_external": {
   "type": "boolean",
   "default": false
  },
  "include_subdomain": {
   "type": "boolean",
   "default": false
  },
  "search_query": {
   "type": "string"
  },
  "top_n": {
   "type": "integer"
  },
  "timeout": {
   "type": "integer"
  },
  "scrape_options": {
   "type": "object",
   "properties": {
    "formats": {
     "type": "string",
     "enum": [
      "html",
      "markdown",
      "text",
      "screenshot"
     ]
    }
   }
  }
 },
 "additionalProperties": true,
 "required": [
  "start_url",
  "max_pages",
  "follow_robots_txt"
 ],
 "description": "Webhooks, follow_robots_txt=false and structured parsers are deliberately excluded. Use olostep.web.crawl.results with formats=markdown or html to read completed content."
}

MCP

{"mcpServers":{"toll402":{"command":"npx","args":["-y","toll402-mcp"],"env":{"TOLL402_WALLET_KEY":"0x..."}}}}

All tools · OpenAPI