robots.txt Builder
Create a clean robots.txt with presets, per-bot rules, crawl-delay and sitemaps. Live preview + copy/download.
Optional – used to suggest
sitemap.xml URL.
Sitemaps
User-agent Rules
Tips: Use * as wildcard and $ to match end (e.g., /*.pdf$). Put more specific rules after the User-agent line they belong to.