# robots.txt generator — no upload

> Generate a robots.txt to allow or disallow crawlers per path, with sitemap directives, in your browser. Nothing is uploaded.

Generate a robots.txt that allows or disallows crawlers per user-agent and path, and points to your sitemap. It's assembled in the browser, so nothing is uploaded.

robots.txt guides search-engine crawlers on what to index, which matters for SEO and for keeping low-value paths out of results. Generating it correctly avoids the classic mistake of accidentally disallowing your whole site.

Pair it with the meta-tag generator for on-page SEO. Remember it's guidance, not security.

## How to

1. **Set rules.** Choose user-agents and allow/disallow paths.
2. **Add a sitemap.** Optionally include your sitemap URL.
3. **Copy or download.** Save robots.txt to your site root.

## FAQ

### Does robots.txt keep a page private?

No. It requests that compliant crawlers skip paths, but it's not access control — anyone can read robots.txt, and it doesn't stop direct access. Use real auth for private content.

### Should I include my sitemap?

Yes. A Sitemap directive helps search engines discover your URLs, so including it is good practice.

### Is anything uploaded?

No. The file is generated locally in the tab.


## Related tools

- [Meta / OG tags](https://www.safepaper.app/dev/meta-tags)
- [README badge](https://www.safepaper.app/dev/badge-gen)
- [Security headers](https://www.safepaper.app/dev/security-headers)
- [URL parser](https://www.safepaper.app/dev/url-parser)

---

Canonical HTML: https://www.safepaper.app/dev/robots-txt
Markdown: https://www.safepaper.app/dev/robots-txt.md

There is no upload endpoint — your files are processed in this browser tab. Open your network tab and check.