CRAWL DIAGNOSTICS

Robots & Sitemap

Inspect crawler rules for your URL and validate the site’s sitemap files.

Measured from this VPS · Public HTTP / HTTPS

Start a checkEnter a domain or URL

Robots rules control crawling, not authentication or search indexing. Sitemap discovery is bounded; nested sitemap files are listed without crawling the whole site.