llms.txt Content
# Crawleo
> Privacy-first API to search and crawl any website. Get clean, AI-ready data from protected sites with zero data retention.
Crawleo provides a Search API and Crawl API designed for LLMs, RAG pipelines, and automation workflows. It supports the Model Context Protocol (MCP) for direct integration with AI assistants like Claude, Cursor, and GitHub Copilot.
## Key Features
- **Search API**: Real-time web search with geo, language, and device targeting. Returns clean, structured results optimized for AI consumption.
- **Crawl API**: Extract content from any website as Markdown or HTML. Handles JavaScript rendering, anti-bot protections, and Cloudflare challenges.
- **MCP Server**: Native Model Context Protocol support. Connect directly to Claude Desktop, Cursor, GitHub Copilot, and other MCP-compatible tools.
- **Zero Data Retention**: No queries or crawled content are stored, sold, or used for training — on every plan including Free.
- **Proxy Support**: Standard and premium proxies for geo-restricted or heavily protected sites.
## Pricing
- **Free**: 500 credits/month (50 searches or 500 crawls)
- **Starter**: $20/month — 100,000 credits
- **Growth**: $50/month — 300,000 credits
- **Business**: $100/month — 1,000,000 credits
- **Enterprise**: Custom pricing
Search costs 10 credits per page of results. Crawling costs 1 credit (no proxy), 2 credits (standard proxy), or 10 credits (premium proxy) per page.
## Integrations
- [LangChain](https://www.crawleo.dev/integrations/langchain): Official Python package `langchain-crawleo` with CrawleoSearch and CrawleoCrawler tools.
- [n8n](https://www.crawleo.dev/integrations/n8n): HTTP Request node integration for workflow automation.
- [MCP](https://www.crawleo.dev/integrations/mcp): Available on MCP.so, Smithery, Glama, and MCP Market.
- Make and Zapier integrations coming soon.
## Links
- [Homepage](https://www.crawleo.dev/)
- [Documentation](https://docs.crawleo.dev/)
- [Pricing](https://www.crawle