robots.txt
Builds the file that tells AI crawlers they're welcome
Scan a site first
This tool fills itself in from your scan.
Uses 1 token · competitors free
Your settings
AI crawlersfrom your site
All crawlers
Keep privatefrom your site
File to check
We read the live file at your site/robots.txt.
Your file
What we found
| Line | Rule | Meaning |
|---|
Your fixed file
Pick a site and we'll read its live robots.txt.
Put it on your site
WordPress
Send to your web person
Free. Checks the live file after you install it.