Meta Title & Description Preview
Check title and description length alongside a search-style preview.
SEO & websitesSEO & websites tools
Robots.txt Generator
Create crawl directives for your website and sitemap.
Your workspaceProcessed in your browser
Loading workspace…
How to use robots.txt generator
- Set User agent to the intended crawler group.
- Enter disallowed paths beginning with / and a complete Sitemap URL.
- Choose Generate result and review the full robots.txt before deployment.
Work through an example
Create one user-agent group and a sitemap reference while distinguishing crawl guidance from access protection.
Generate a small robots.txt file with explicit disallowed paths →Formats, privacy, and practical limits
This tool processes the values you enter in your browser. No account or upload to ERITHRA is required.
Files and generated results exist only for this tab session. Download any result you want to keep before leaving the page. Browsers may use temporary memory or disk cache while processing.
Frequently asked questions
Does robots.txt protect private pages?
No. It is public crawl guidance, not access control. Private content needs appropriate authentication.
Does this merge with my existing robots file?
No. It creates one user-agent group and a sitemap reference from the supplied values without fetching or modifying the existing file.