
What Is llms.txt? Format, Example, and Whether You Need One
llms.txt is a plain Markdown file at the root of your site (/llms.txt) that tells large language models what your site is and where its most useful pages are. Think of it as a short, curated table of contents written for AI assistants rather than for search engine crawlers. It was proposed in September 2024 by Jeremy Howard of Answer.AI and has spread quickly through developer documentation sites.
This guide explains what llms.txt is, the exact llms.txt format, a complete llms.txt example you can adapt, and the honest answer to the question everyone asks: does anything actually read it?
| llms.txt quick facts | |
|---|---|
| Location | https://yoursite.com/llms.txt |
| Format | Markdown: an H1 title, a short summary in a blockquote, then sections of links |
| Companion file | llms-full.txt, the full text of your key pages in one file |
| Standard? | A community proposal, not an IETF or W3C standard |
| Used by Google Search? | No. Google has said it does not use llms.txt for ranking or AI Overviews |
| Effort | Low: an hour for most sites |
What problem llms.txt tries to solve
When an AI assistant fetches your site to answer a question, it gets HTML full of navigation, cookie banners, scripts and layout. The useful content is buried, and the model's context window is limited. llms.txt gives the model a shortcut: a clean summary of the site and a short list of the pages worth reading, in a format models handle well. The idea is the same one behind good documentation: tell the reader where to start.
llms.txt vs robots.txt vs sitemap.xml
The three files are often confused, but they do different jobs:
| File | Audience | Job |
|---|---|---|
robots.txt | Crawlers, including AI bots | Says what bots may and may not fetch. This is where you allow or block GPTBot, ClaudeBot or PerplexityBot. |
sitemap.xml | Search engines | Lists every URL you want indexed, with last-modified dates. |
llms.txt | Language models and AI agents | A curated, human-readable summary and the handful of pages that best explain your site. |
llms.txt does not grant or block access. If robots.txt blocks an AI crawler, an llms.txt file will not help it. Get robots.txt right for AI crawlers first; our robots.txt and sitemap guide covers both files in depth.
The llms.txt format
The llms.txt format is deliberately simple, and the order of sections matters:
- An H1 with the site or project name. This is the only required element.
- A blockquote with a one or two sentence summary of what the site is.
- Optional paragraphs with anything a model should know: who the product is for, key terms, what not to assume.
- H2 sections containing Markdown lists of links, each in the form
[Title](URL): short description. - An optional section named "Optional" for secondary links a model can skip when its context is tight.
An llms.txt example
Here is a complete llms.txt example for a fictional SaaS product. Copy it, replace the content, and keep it short: the point of llms.txt is curation, not completeness.
Serve it as text/plain or text/markdown, not as an HTML page. A common mistake is a framework catch-all route that returns your 404 page or homepage HTML at /llms.txt with a 200 status. That looks like an llms.txt file to a quick check but is useless to a model.
What about llms-full.txt?
Some sites also publish llms-full.txt, which contains the full text of the key pages in a single Markdown file, so an agent can load everything in one request. It is popular with developer documentation (it is handy to paste into a coding assistant). It can get large; if you publish one, generate it from your docs at build time so it never goes stale.
Does anything actually read llms.txt?
Here is the honest picture as of late 2026:
- Google Search does not use llms.txt. Google's search team has said publicly that it is not used for ranking or for AI Overviews. Do not expect a traffic bump in Google from adding one.
- The major AI assistants have not committed to it. No large AI company has documented that its crawler reads llms.txt as a ranking or citation signal. Server logs on sites that publish one show only occasional fetches.
- Developer tools do use it. Coding assistants, documentation tools and MCP-based doc servers read llms.txt and llms-full.txt to pull accurate docs into an agent's context. If you run a developer product, this is the strongest reason to publish one.
So should you add llms.txt? It costs an hour and does no harm, and for documentation-heavy and developer-facing sites it already helps. For a local business or an ecommerce store, the things that move AI search visibility are still crawl access, clear content and structured data. Treat llms.txt as a low-cost extra, not a strategy.
How to create an llms.txt file
- List the 5 to 20 pages that best explain your product or site. Start with what a new customer or a new developer would read first.
- Write the H1 and a two-sentence summary. Be plain and specific; this is the text a model is most likely to quote.
- Group the links under a few H2 headings, each with a one-line description.
- Save it as
llms.txtin your public root (for examplepublic/llms.txtin Next.js, Astro or Vite projects). - Check that
https://yoursite.com/llms.txtreturns status 200, a non-HTML content type, and starts with a Markdown heading. - Review it whenever your main pages change. A stale llms.txt that points at removed pages is worse than none.
How Scanverra checks llms.txt
The Scanverra SEO Optimizer checks whether your site serves an llms.txt file at the root, and it is strict about it: the file must return a successful status, must not be an HTML page, and must start with a Markdown heading, so a framework fallback page does not count. Because no search engine uses llms.txt, Scanverra reports it as information only and never lowers your score for a missing one. The same scan checks which AI crawlers your robots.txt allows (search bots such as OAI-SearchBot, Claude-SearchBot and PerplexityBot versus training bots such as GPTBot and Google-Extended), along with structured data and the rest of your on-page SEO. Those crawler settings matter far more for AI visibility than llms.txt does today.
Key takeaways
- llms.txt is a curated Markdown map of your site for AI models, served at
/llms.txt. - The llms.txt format is an H1, a blockquote summary and sections of links.
- It does not replace robots.txt or sitemap.xml, and Google does not use it.
- It is worth publishing for developer and documentation sites, and harmless elsewhere.
Frequently asked questions
llms.txt is a Markdown file served at the root of a website that gives large language models a short summary of the site and a curated list of its most useful pages. It was proposed in September 2024 by Jeremy Howard of Answer.AI.
Not in Google. Google has said it does not use llms.txt for ranking or AI Overviews. It can help AI coding assistants and documentation tools find the right pages, which is why developer-focused sites publish one.
robots.txt controls which bots may crawl which URLs. llms.txt grants or blocks nothing; it is a curated guide for models that can already read your site. If robots.txt blocks an AI crawler, llms.txt cannot override that.
llms-full.txt is an optional companion file containing the full Markdown text of your key pages in one file, so an AI agent can load everything in a single request. It is mostly used by documentation sites.
At the root of your domain, so it loads at https://yoursite.com/llms.txt. In most frameworks that means the public or static folder. Make sure it returns status 200 with a plain text or Markdown content type, not an HTML page.
Related reading
Structured Data and AI Search: What JSON-LD Actually Does for You in 2026
What JSON-LD structured data is, why it matters for both Google rich results and AI crawlers, and a practical checklist for auditing what's actually on your pages.
How to Fix robots.txt Blocking AI Crawlers
Why a blanket Disallow rule can silently block GPTBot, ClaudeBot, and other AI crawlers from your site, and how to unblock them without opening the site to everything.
A Practical Guide to robots.txt and Sitemap.xml for SEO and AI Crawlers
What each file actually controls, the mistakes that accidentally block crawlers (human search engines and AI ones alike), and how to verify both are configured correctly.
AI Readiness Assessment: A Practical Guide to Measuring AI Readiness
What AI readiness means, the six areas every AI readiness assessment should cover, a simple scoring model, a checklist, and the security and website checks most teams skip.
SEO Optimizer
Checks for llms.txt, AI crawler access in robots.txt and structured data in one scan.
robots.txt Generator
Build a robots.txt that allows AI search bots and blocks training bots if you choose.
See your full audit score right now
Run a free instant audit and get an AI-prioritized fix list for performance, SEO, and accessibility.
Run a free audit