Robots.txt Generator
Create robots.txt files visually to control how search engines crawl your website. Choose a preset or build custom rules for any crawler.
Quick Presets
User-Agent Rules
UA*
No disallow paths — everything is accessible
Sitemaps
No sitemaps added. Click "Add" to include a sitemap URL.
Generated robots.txt
User-agent: * Allow: /
Tip: Place the robots.txt file in the root directory of your website (e.g., https://example.com/robots.txt). Search engines check this file before crawling your site.