State of the Agentic Web

Live data on how well agent-first sites serve AI agents. Updated with every recrawl.

5237
Agent-First Sites
38.3
Average Score
495
Score 70+
683
Live MCP Servers

Signal Adoption

How many of the 5237 agent-first indexed sites expose each agentic signal. A site must expose a hard signal such as a structured API, OpenAPI spec, ai-plugin manifest, or MCP server to appear in this report.

llms.txt
3069 sites (58.6%)
Schema.org
2548 sites (48.7%)
robots.txt AI rules
1259 sites (24.0%)
MCP Server
683 sites (13.0%)
OpenAPI Spec
841 sites (16.1%)
Structured API
4857 sites (92.7%)
ai-plugin.json
414 sites (7.9%)
Key finding: 59% of agent-first sites publish llms.txt, while 16% expose an OpenAPI spec. Sites that pair llms.txt with an MCP server (544 sites) score dramatically higher than passive content-only sites, which are excluded from this report.

Categories

CategorySitesAvg Score
developer 1601 38.0
ai-tools 1118 44.1
other 1035 31.1
data 491 33.2
finance 229 45.0
ecommerce 178 42.0
productivity 176 41.2
communication 153 40.4
security 132 40.4
health 62 42.1
jobs 31 41.5
education 20 50.0

Top Scoring Sites

DomainScoreCategory
bibigpt.co 100 ai-tools
asset-manager-jaymarrero2.replit.app 100 other
headlessoracle.com 100 ai-tools
tryspook.com 100 communication
api.headlessoracle.com 100 data
370.ai 100 ai-tools
childanxiety.ai 100 ai-tools
bbwbelles.com 100 other
workprotocol.ai 100 ai-tools
x711.io 100 ai-tools
lobehub.com 100 other
nothumansearch.com 100 ai-tools
budgetfitter.uk 100 ecommerce
childadhd.ai 100 ai-tools
teenadhd.ai 100 ai-tools
tooloracle.io 100 developer
zgts.in 100 health
terminalfeed.io 100 finance
api.contrastcyber.com 100 data
gptr.dev 100 developer

Methodology

Every site is scored on 7 signals: llms.txt (25 pts), ai-plugin.json (20), OpenAPI spec (20), structured API (15), MCP server (10), robots.txt AI rules (5), Schema.org (5). All detections are content-verified — the crawler probes MCP endpoints with JSON-RPC, parses OpenAPI specs, and validates ai-plugin.json manifests.

The index grows via auto-discovery from the official MCP registry, PulseMCP, awesome-mcp-servers lists, and llms.txt directories. Sites are recrawled daily.

Full scoring methodology: Guide

Check your site's agentic readiness

Score any URL instantly and see which signals you're missing.

Score Your Site