Before publishing a change
Look for crawling restrictions left over from an earlier stage.
Robots.txt checker
Does your website guide
search crawlers?
Read robots.txt, review published rules and simulate a path for Googlebot. Understand the file before editing.
- Public file and HTTP response
- Organized rules and sitemaps
Free, no signup
Your result
Read the rules before changing them.
A specific page
Enter a path and compare it with applicable Googlebot rules.
Where is the sitemap?
See declared sitemap URLs without confusing a declaration with crawling or indexing.
Your website deserves room
for its next step.
Explore hosting plans and compare their resources with your project’s needs.
Explore hosting plans
A few things worth knowing.
Does blocking crawling prevent indexing?
Not necessarily. A blocked URL may still appear through external links. Robots.txt does not replace noindex, authentication or access control.
Is the simulation Google’s decision?
It is a local interpretation of Allow and Disallow for Googlebot using the most specific rule. It does not query Search Console or reproduce crawler cache or internal decisions.
What does a file retrieval error mean?
We show the HTTP response or retrieval failure. Unavailability is not treated as a blanket permission. Reading is capped at 512 KB; larger files need a full review.