AI Bot Directory / Claude-SearchBot
What is Claude-SearchBot?
Claude-SearchBot builds the index behind Anthropic's AI search and answer products. Pages it crawls can be retrieved and cited in AI answers — this is the crawler that most directly turns your content into AI visibility.
Last 7 days: 5 hits across 1 reporting sites. Live aggregate data on the AI Crawler Traffic Report.
How to spot Claude-SearchBot in your logs
Look for this token in the request User-Agent (case-insensitive):
claude-searchbot
Should you allow or block Claude-SearchBot?
Blocking it removes your pages from AI answer retrieval — if you want to be cited by AI assistants, this is the last crawler you should block.
To block it anyway, add this to your robots.txt:
User-agent: Claude-SearchBot Disallow: /
Note: robots.txt is a request, not a lock — well-behaved crawlers honor it; verify with your own traffic data.
Is Claude-SearchBot visiting your site?
NeverDrafts tracks every AI crawler hit on your site (one snippet, live in minutes) and monitors whether ChatGPT and Claude actually recommend you when buyers ask.
Start monitoring