Here's exactly what we'd ship for scrapeninja.net:
# /llms.txt for scrapeninja.net > One-line description of scrapeninja.net for AI agents. ## Key pages - /: homepage - /api: API docs (if any)
{
"schema_version": "v1",
"name_for_model": "scrapeninja.net",
"description_for_model": "What scrapeninja.net does, for an AI agent.",
"api": { "type": "openapi", "url": "https://scrapeninja.net/openapi.yaml" }
}# robots.txt for scrapeninja.net — allow AI agents User-agent: GPTBot Allow: / User-agent: ClaudeBot Allow: / User-agent: PerplexityBot Allow: / Sitemap: https://scrapeninja.net/sitemap.xml
Two ways to fix it: get the $29 report instantly on-screen — the exact files & fixes to apply yourself, no email or repo needed — or have us do it for $199, delivered as a pull request within 72 hours. $199 tier: full refund if your score doesn't hit 90+.
← Back to score report