Robots.txt & Sitemap Generator
Generate crawler-friendly robots.txt files and XML sitemaps with crawl-delay and disallow directives.
Crawler Directives Configuration
About Robots.txt & Sitemap Generator
Guide search engine bots (Googlebot, Bingbot) cleanly through your website. Configure allow/disallow directives for admin folders, declare XML sitemap locations, and export valid robots.txt and XML sitemap files.
How to Use Robots.txt & Sitemap Generator
Configure default access (Allow / Disallow) for all user agents or specific bots.
Add paths you wish to block from indexation (e.g. /admin/, /private/).
Enter your sitemap URL.
Download or copy your production-ready `robots.txt`.
Supported Formats & Limits
- Supported Inputs: TEXT
- Guest Maximum Size: 25 MB
- Logged-in Max Size: 100 MB
- System Compatibility: Windows, macOS, Linux, iOS & Android
Privacy & Security Guarantee
This tool processes your files entirely inside your web browser using client-side JavaScript, WebAssembly, and Canvas APIs. Your document is never uploaded to any external server, guaranteeing total confidentiality.
Frequently Asked Questions
Where does the robots.txt file belong on my server?
It must be uploaded directly to the root directory of your domain (e.g., https://example.com/robots.txt).