Free tool
Fetch robots.txt and the sitemap for a domain, count URLs, and spot common blocking mistakes.
More tools