🤖
robots.txt Validatorcoming soon
Paste a robots.txt and see parse warnings, user-agent grouping, and a "would this URL be crawled?" test against any given path + bot.
/web & seo
In progress
robots.txt Validator — coming soon.
Paste a robots.txt and see parse warnings, user-agent grouping, and a "would this URL be crawled?" test against any given path + bot.
Pinned to the roadmap. Want me to prioritise this one? ← Browse the rest