State of the Agentic Web

Live data on how well agent-first sites serve AI agents. Updated with every recrawl.

4311
Agent-First Sites
37.4
Average Score
352
Score 70+
486
Live MCP Servers

Signal Adoption

How many of the 4311 agent-first indexed sites expose each agentic signal. A site must expose a hard signal such as a structured API, OpenAPI spec, ai-plugin manifest, or MCP server to appear in this report.

llms.txt
2477 sites (57.5%)
Schema.org
2065 sites (47.9%)
robots.txt AI rules
964 sites (22.4%)
MCP Server
486 sites (11.3%)
OpenAPI Spec
603 sites (14.0%)
Structured API
4012 sites (93.1%)
ai-plugin.json
351 sites (8.1%)
Key finding: 57% of agent-first sites publish llms.txt, while 14% expose an OpenAPI spec. Sites that pair llms.txt with an MCP server (384 sites) score dramatically higher than passive content-only sites, which are excluded from this report.

Categories

CategorySitesAvg Score
developer 1313 37.8
ai-tools 927 42.5
other 805 28.4
data 394 33.5
finance 198 43.4
ecommerce 162 40.8
productivity 160 40.1
communication 122 39.0
security 114 39.4
health 60 43.3
jobs 26 42.3
education 20 50.0

Top Scoring Sites

DomainScoreCategory
nothumansearch.com 100 ai-tools
rettfrabonden.com 100 ecommerce
childanxiety.ai 100 ai-tools
tooloracle.io 100 developer
xquik.com 100 developer
prereason.com 100 finance
terminalfeed.io 100 finance
aurelianflo.com 100 ai-tools
chartlibrary.io 100 finance
flowzap.xyz 100 developer
claudereviews.com 100 ai-tools
gptr.dev 100 developer
lobehub.com 100 other
voidly.ai 100 ai-tools
asterpay.io 100 ai-tools
blooio.com 100 productivity
agentprobe.fly.dev 100 developer
chainray.online 100 ai-tools
teenadhd.ai 100 ai-tools
zgts.in 100 health

Methodology

Every site is scored on 7 signals: llms.txt (25 pts), ai-plugin.json (20), OpenAPI spec (20), structured API (15), MCP server (10), robots.txt AI rules (5), Schema.org (5). All detections are content-verified — the crawler probes MCP endpoints with JSON-RPC, parses OpenAPI specs, and validates ai-plugin.json manifests.

The index grows via auto-discovery from the official MCP registry, PulseMCP, awesome-mcp-servers lists, and llms.txt directories. Sites are recrawled daily.

Full scoring methodology: Guide

Check your site's agentic readiness

Score any URL instantly and see which signals you're missing.

Score Your Site