llms.txt Guide
How to Write One for Your Website
The llms.txt file is the most important thing you can add to your website for AI agent discoverability. Think of it as a README for LLMs — a curated, Markdown-formatted summary of what your site offers.
What is llms.txt?
llms.txt is a Markdown file served at your site root (e.g., https://example.com/llms.txt) that provides a curated summary of your website for Large Language Models. It was proposed by Jeremy Howard at Answer.AI and has been adopted by Stripe, Cloudflare, Anthropic, Vercel, and hundreds of other companies.
Unlike a sitemap (which lists URLs for crawlers) or robots.txt (which sets permissions), llms.txt provides context. It tells an LLM: "Here is who we are, what we offer, and where to find the important stuff." An agent that reads your llms.txt before interacting with your site will make fewer mistakes, hallucinate less, and complete tasks more reliably.
The file should be concise — ideally under 2,000 tokens. For sites with extensive documentation, a companion llms-full.txt file can provide the complete reference.
The llms.txt Format Specification
The format uses standard Markdown with a specific structure. Here are the required and recommended sections:
# Your Company Name > Brief one-line description of what your company does. ## Docs - [Getting Started](https://example.com/docs/getting-started): Quick start guide - [API Reference](https://example.com/docs/api): Complete API documentation - [Authentication](https://example.com/docs/auth): Auth flows and API keys ## API - [OpenAPI Spec](https://example.com/openapi.json): Machine-readable API spec - [MCP Server](https://example.com/.well-known/mcp.json): MCP discovery ## Products - [Product A](https://example.com/products/a): Description of Product A - [Pricing](https://example.com/pricing): Plans and pricing ## Optional - [Blog](https://example.com/blog): Latest updates - [Changelog](https://example.com/changelog): Version history
Key rules: The H1 heading must be your company or product name. The blockquote is a one-line summary. Each section uses H2 headings with bulleted lists of links. Every link should include a brief description after the colon.
The format is intentionally simple. LLMs parse Markdown natively — no special tooling required. The goal is not to replicate your entire site but to give an agent a map of the most important resources.
Best Practices
Keep it under 2,000 tokens
Agents often have limited context windows or use llms.txt as a pre-flight check. Brevity is essential. Use llms-full.txt for comprehensive content.
Link to machine-readable resources
Prioritize links to OpenAPI specs, JSON-LD endpoints, Markdown docs, and API references over marketing pages.
Update it regularly
If you launch a new product or API, update llms.txt. Stale content causes agents to hallucinate about features that no longer exist or miss new ones.
Use absolute URLs
Always use full URLs (https://...) in your links. Agents may not resolve relative paths correctly.
Serve with correct content type
Return text/markdown or text/plain content type. Some agents check the MIME type before parsing.
Add llms-full.txt for large sites
If your site has extensive docs, create an llms-full.txt with the complete reference. Link to it from your llms.txt.
Who Already Uses llms.txt?
As of April 2026, llms.txt has been adopted by major platforms including Stripe (links to all API docs and SDKs), Cloudflare (developer platform documentation), Anthropic (Claude API reference and guides), Vercel (Next.js docs and deployment guides), and Supabase (database and auth documentation).
The pattern is clear: developer-facing companies with APIs lead adoption because their users are already building with AI tools. But llms.txt is not limited to dev tools. E-commerce sites, SaaS platforms, and content publishers all benefit from telling agents what they offer.
The llms.txt directory at llmstxt.site tracks over 1,200 sites that publish the file. If your competitors are on that list and you are not, agents are learning about them and ignoring you.
llms.txt and Your AX Score
In our AX Audit, llms.txt contributes to the Discoverability dimension (20% of total score). Presence of llms.txt alone can add 3-5 points to your overall score. But quality matters too — an llms.txt with broken links or outdated information scores lower than one with accurate, well-structured content.
llms.txt also impacts Token Efficiency (10% of total score) because it gives agents a condensed entry point instead of forcing them to crawl and parse your entire site. Agents that read llms.txt first use fewer tokens to accomplish the same tasks.
Generate Your llms.txt
You can write llms.txt by hand — it is just Markdown. But if you want a starting point, our AX Audit tool will analyze your site and suggest an llms.txt based on what it finds: your key pages, API docs, product listings, and documentation structure.
The generated file is a starting point, not a final product. Review it, remove irrelevant links, add missing resources, and make sure every link works. Then deploy it at your site root and re-run the audit to see the score improvement.
Check If Your Site Has llms.txt
The AX Audit checks for llms.txt, validates its format, tests every link, and scores its quality. Find out where you stand in under 60 seconds.
Check Your llms.txt With AX AuditFree audit. No signup required.