State of the Agentic Web

Live data on how well the web serves AI agents. Updated with every recrawl.

9042
Sites Indexed
17.5
Average Score
66
Score 70+
519
Live MCP Servers

Signal Adoption

How many of the 9042 indexed sites expose each agentic signal. llms.txt leads adoption, but most sites stop there — only 4% pair it with a structured API.

llms.txt
4530 sites (50.1%)
Schema.org
3584 sites (39.6%)
robots.txt AI rules
1171 sites (13.0%)
MCP Server
519 sites (5.7%)
OpenAPI Spec
372 sites (4.1%)
Structured API
345 sites (3.8%)
ai-plugin.json
158 sites (1.7%)
Key finding: 50% of sites publish llms.txt — but only 4% expose an OpenAPI spec. The agentic web is still mostly content, not APIs. Sites that pair llms.txt with an MCP server (412 sites) score dramatically higher than content-only sites.

Categories

CategorySitesAvg Score
other 4274 15.7
developer 2048 16.7
ai-tools 1104 22.0
data 623 13.9
finance 268 23.1
productivity 188 24.9
security 165 23.8
ecommerce 155 23.5
health 79 23.8
communication 73 20.5
education 31 30.2
jobs 26 26.3

Top Scoring Sites

DomainScoreCategory
feedoracle.io 100 security
nothumansearch.ai 100 ai-tools
passdown.arflow.io 100 productivity
prereason.com 100 finance
agentndx.ai 100 developer
aidevboard.com 100 jobs
8bitconcepts.com 100 ai-tools
borealhost.ai 100 ai-tools
chartlibrary.io 95 ai-tools
archtools.dev 95 ai-tools
dchub.cloud 95 data
headlessoracle.com 95 ai-tools
agoragentic.com 90 developer
blackveilsecurity.com 90 developer
tomba.io 85 ai-tools
rendoc.dev 85 developer
agentgrade.com 85 ai-tools
node-core-firstlineaiis.replit.app 85 ai-tools
0latency.ai 85 ai-tools
clawaimail.com 85 developer

Methodology

Every site is scored on 7 signals: llms.txt (25 pts), ai-plugin.json (20), OpenAPI spec (20), structured API (15), MCP server (10), robots.txt AI rules (5), Schema.org (5). All detections are content-verified — the crawler probes MCP endpoints with JSON-RPC, parses OpenAPI specs, and validates ai-plugin.json manifests.

The index grows via auto-discovery from the official MCP registry, PulseMCP, awesome-mcp-servers lists, and llms.txt directories. Sites are recrawled daily.

Full scoring methodology: Guide

Check your site's agentic readiness

Score any URL instantly and see which signals you're missing.

Score Your Site