webclaw — Web Scraping API for LLMs and AI Agents

# webclaw Documentation > webclaw is a Rust-based web extraction toolkit that turns any website into LLM-ready markdown, JSON, plain text, or token-optimized output -- without a headless browser. - Source: [github.com/0xMassi/webclaw](https://github.com/0xMassi/webclaw) - Cloud API: [api.webclaw.io](https://api.webclaw.io) - License: AGPL-3.0 ## Table of Contents - [Introduction](#introduction) - [Getting Started](#getting-started) - [CLI Reference](#cli-reference) - [REST API](#rest-api) - [Overview](#rest-api-overview) - [POST /v1/scrape](#post-v1scrape) - [POST /v1/crawl](#post-v1crawl) - [GET /v1/crawl/{id}](#get-v1crawlid) - [GET /v1/crawl/{id}/stream](#get-v1crawlidstream) - [GET /v1/crawl/history](#get-v1crawlhistory) - [POST /v1/crawl/{id}/retry](#post-v1crawlidretry) - [POST /v1/batch](#post-v1batch) - [POST /v1/map](#post-v1map) - [POST /v1/search](#post-v1search) - [POST /v1/agent-scrape](#post-v1agent-scrape) - [POST /v1/extract](#post-v1extract) - [POST /v1/summarize](#post-v1summarize) - [POST /v1/diff](#post-v1diff) - [POST /v1/brand](#post-v1brand) - [Webhooks](#webhooks) - [Firecrawl v2 Compatibility](#firecrawl-v2-compatibility) - [MCP Server](#mcp-server) - [Self-Hosting](#self-hosting) - [Cloud API](#cloud-api) - [SDKs](#sdks) --- ## Introduction webclaw is a web extraction toolkit built in Rust. It turns any website into LLM-ready markdown, JSON, plain text, or token-optimized output -- without a headless browser. All extraction happens over raw HTTP using browser-grade TLS impersonation, making it fast, lightweight, and deployable anywhere. ### Three binaries, one engine webclaw ships as three standalone binaries, all powered by the same extraction core: - **webclaw** -- The CLI. Extract, crawl, summarize, and track changes from the terminal. Pipe output to files, chain with other tools, or use interactively. - **webclaw-server** -- The REST API. An axum-based HTTP server with authentication, CORS, gz