← Back to search
40
Partial
Agentic Readiness Score
developer llms-txtmcpapiaisearch

Agentic Signals

📄
Found
🤖
ai-plugin.json
Not found
📖
OpenAPI Spec
Not found
🔗
Structured API
Not found
🛡
Not specified
🏷
Schema.org Markup
Found
MCP Server
Found

Embed this badge

Show off your agentic readiness — the badge auto-updates when your score changes.

Agentic Ready 40/100

            

llms.txt Content

# Crawleo > Privacy-first API to search and crawl any website. Get clean, AI-ready data from protected sites with zero data retention. Crawleo provides a Search API and Crawl API designed for LLMs, RAG pipelines, and automation workflows. It supports the Model Context Protocol (MCP) for direct integration with AI assistants like Claude, Cursor, and GitHub Copilot. ## Key Features - **Search API**: Real-time web search with geo, language, and device targeting. Returns clean, structured results optimized for AI consumption. - **Crawl API**: Extract content from any website as Markdown or HTML. Handles JavaScript rendering, anti-bot protections, and Cloudflare challenges. - **MCP Server**: Native Model Context Protocol support. Connect directly to Claude Desktop, Cursor, GitHub Copilot, and other MCP-compatible tools. - **Zero Data Retention**: No queries or crawled content are stored, sold, or used for training — on every plan including Free. - **Proxy Support**: Standard and premium proxies for geo-restricted or heavily protected sites. ## Pricing - **Free**: 500 credits/month (50 searches or 500 crawls) - **Starter**: $20/month — 100,000 credits - **Growth**: $50/month — 300,000 credits - **Business**: $100/month — 1,000,000 credits - **Enterprise**: Custom pricing Search costs 10 credits per page of results. Crawling costs 1 credit (no proxy), 2 credits (standard proxy), or 10 credits (premium proxy) per page. ## Integrations - [LangChain](https://www.crawleo.dev/integrations/langchain): Official Python package `langchain-crawleo` with CrawleoSearch and CrawleoCrawler tools. - [n8n](https://www.crawleo.dev/integrations/n8n): HTTP Request node integration for workflow automation. - [MCP](https://www.crawleo.dev/integrations/mcp): Available on MCP.so, Smithery, Glama, and MCP Market. - Make and Zapier integrations coming soon. ## Links - [Homepage](https://www.crawleo.dev/) - [Documentation](https://docs.crawleo.dev/) - [Pricing](https://www.crawle