About the AgentReadyChecker bot
If you saw AgentReadyChecker in your server logs, someone asked the Agent Ready Checker plugin to check your website.
What it sends
Requests use this User-Agent:
Mozilla/5.0 (compatible; AgentReadyChecker/1.0; +https://agentready.ferasallaou.com/bot)
What it fetches
Only when a person asks for a check, and only:
- the page they asked about, and the
http://version of your homepage (to see if it redirects to HTTPS) /robots.txt,/llms.txt,/llms-full.txtand your sitemap- up to 5 links listed in your llms.txt, to check they work
- the same page once more, asking for Markdown, to see if you offer it
It doesn't crawl, doesn't follow other links, doesn't run JavaScript, and doesn't pretend to be a browser or another company's bot. Results are cached for 10 minutes, and each site can be checked at most 20 times a minute.
robots.txt
The checker follows robots.txt. If your rules for AgentReadyChecker, or for all bots (User-agent: *), disallow a page, it won't fetch that page or anything else from your site, and only reports what your robots.txt says. To opt out of this checker only, add:
User-agent: AgentReadyChecker Disallow: /
Questions: me@ferasallaou.com.