webclaw: Web Scraping API for LLMs and AI Agents

# webclaw Documentation > webclaw is a Rust-based web extraction toolkit that turns any website into LLM-ready markdown, JSON, plain text, or token-optimized output. - Source: [github.com/0xMassi/webclaw](https://github.com/0xMassi/webclaw) - Cloud API: [api.webclaw.io](https://api.webclaw.io) - License: AGPL-3.0 ## Table of Contents - [Introduction](#introduction) - [Getting Started](#getting-started) - [CLI Reference](#cli-reference) - [REST API](#rest-api) - [Overview](#rest-api-overview) - [POST /v1/scrape](#post-v1scrape) - [POST /v1/crawl](#post-v1crawl) - [GET /v1/crawl/{id}](#get-v1crawlid) - [GET /v1/crawl/{id}/stream](#get-v1crawlidstream) - [GET /v1/crawl/history](#get-v1crawlhistory) - [POST /v1/crawl/{id}/retry](#post-v1crawlidretry) - [POST /v1/batch](#post-v1batch) - [POST /v1/map](#post-v1map) - [POST /v1/search](#post-v1search) - [POST /v1/extract](#post-v1extract) - [POST /v1/lead](#post-v1lead) - [POST /v1/lead/batch](#post-v1leadbatch) - [POST /v1/summarize](#post-v1summarize) - [POST /v1/diff](#post-v1diff) - [POST /v1/brand](#post-v1brand) - [Webhooks](#webhooks) - [Firecrawl v2 Compatibility](#firecrawl-v2-compatibility) - [MCP Server](#mcp-server) - [Self-Hosting](#self-hosting) - [Cloud API](#cloud-api) - [SDKs](#sdks) --- ## Introduction webclaw is a web extraction toolkit built in Rust. It turns any website into LLM-ready markdown, JSON, plain text, or token-optimized output. Extraction is fast, lightweight, and deployable anywhere, with static pages returned in around 118ms. ### Three binaries, one engine webclaw ships as three standalone binaries, all powered by the same extraction core: - **webclaw** -- The CLI. Extract, crawl, summarize, and track changes from the terminal. Pipe output to files, chain with other tools, or use interactively. - **webclaw-server** -- The REST API. An axum-based HTTP server with authentication, CORS, gzip compression, and async job management. Every extraction

{ "openapi": "3.1.0", "info": { "title": "webclaw API", "version": "1.0.0", "description": "Web extraction API for AI agents. Scrape, crawl, search, map, batch, extract, summarize, diff, research, and inspect brand signals from public web pages." }, "servers": [ { "url": "https://api.webclaw.io" } ], "externalDocs": { "description": "webclaw API documentation", "url": "https://webclaw.io/docs/api" }, "x-service-info": { "categories": [