All Tools
1049+ free tools

robots.txt Generator

Build a robots.txt file with allow/disallow rules per user-agent, crawl-delay, sitemap reference and host directive.

Block 1

* matches all bots

Optional. Ignored by Googlebot.

Sitemap & host

Absolute URL to your XML sitemap

Used by Yandex and some other crawlers.

robots.txt
User-agent: *
Disallow: /admin/
Disallow: /private/
Allow: /
Sitemap: https://example.com/sitemap.xml

Related seo tools

Open Graph Preview
Preview & generate OG meta tags for social sharing
Meta Tag Generator
Generate HTML meta tags for SEO
Open Graph Tag Generator
Generate Open Graph (OG) meta tags
XML Sitemap Generator
Generate XML sitemap from URL list