What Is llms.txt and Do You Need One? We Analyzed 6 Reddit Threads

llms.txt is a proposed Markdown file at the root of a website (example.com/llms.txt) that gives AI tools a short, hand-picked list of the site's most useful pages. Jeremy Howard of Answer.AI proposed it in September 2024 so that language models could find the right pages when someone asks them about a site. The file is optional. Google says Search doesn't use it, and no major AI search engine has confirmed reading it. Today it is mostly used by AI coding assistants and agents that work with documentation.

It takes half an hour to add, so the question is less whether you can and more whether it does anything for your site. Below: the format, where the file goes, who reads it, and what practitioners on Reddit report. We analyzed six threads, and most people there think it does nothing.

What does an llms.txt file look like?

Here is a shortened version of ours, at searchlane.app/llms.txt:

# SearchLane

> SearchLane is an SEO and GEO (AI search) platform for small teams: site audits,
> keyword research, rank tracking and visibility in ChatGPT and Google AI answers.

## Product

- [Pricing](https://www.searchlane.app/pricing): plans, per-lookup prices and a usage calculator
- [AI agents (MCP)](https://www.searchlane.app/agents): connect Claude, ChatGPT or Cursor

## Free tools

- [Website audit](https://www.searchlane.app/website-seo-checker/audit): up to 25 pages checked for SEO and AI search problems
- [llms.txt validator](https://www.searchlane.app/tools/llms-txt-validator): checks a site's llms.txt against the format

## Optional

- [Privacy policy](https://www.searchlane.app/website-seo-checker/privacy)

The parts, in order:

1. An H1 with the name of the site or project. This is the only required line.
2. A blockquote with a one- or two-sentence summary of what the site is.
3. Optional notes in plain paragraphs or lists, without headings, for anything a model should know before it starts.
4. H2 sections, each a list of Markdown links, with a short note after a colon.
5. A section called ## Optional for links a tool can skip when it runs short of context.

The proposal also suggests serving a Markdown copy of each page at the same address with .md added, so /pricing.md next to /pricing. A model can read that without the surrounding HTML.

llms.txt vs robots.txt vs sitemap.xml

These three files all live at the root of a site and do different jobs:

Crawlers obey robots.txt. llms.txt is a suggestion that some AI tools may read, and it can't allow or block anything. Access is still decided by robots.txt, which we cover in AI crawlers and robots.txt.

Are llms.txt files worth it?

That depends on who you want to reach. As of October 2026:

Google Search doesn't use it. Google's May 2026 guide to generative AI features says you don't need AI text files, special markup or Markdown copies of pages to appear in Search, AI Overviews and AI Mode included. Search ignores them, and they don't help or hurt rankings.

Chrome does check for it. Lighthouse in Chrome 150 and later has an "Agentic browsing" category with an llms.txt audit. If the file is missing, the audit is marked "not applicable"; if the server returns an error, it fails.

The AI search engines haven't said anything. OpenAI, Anthropic and Perplexity document robots.txt rules for their crawlers, but none of them has said its search crawler reads llms.txt.

Developer platforms use it. Documentation sites such as Stripe's and Cloudflare's publish llms.txt files so coding assistants can find the right pages and their plain-text versions.

If you run documentation, an API, or a product that developers use with AI coding tools, add one. That is the case the format was designed for. On a typical business site it does no harm and takes half an hour, but don't expect it to bring traffic. Crawler access, indexable HTML and clear answers matter more, and our GEO guide covers them.

What does Reddit say about llms.txt? We analyzed 6 threads

We read six Reddit threads about llms.txt, from November 2024 to May 2026, in r/SEO, r/TechSEO, r/seogrowth, r/ProWordPress and r/LLMDevs: 138 visible comments from 94 people. Of those, 46 took a clear position on whether the file helps a site get found or cited by AI:

Ten people said they had tried it on their own sites. Seven saw no change. Three reported more AI mentions or some referrals from ChatGPT soon after adding the file, without comparing against a site that didn't have one.

• The skeptics' main point is that nothing reads the file. "LLMS.txt does nothing, isn't used by LLMs, and you do not need it. But you can do it if you want," was the top answer in r/SEO. In r/TechSEO, a self-described former "fanboy" installed an AI crawler tracker and found that llms.txt "doesn't even get crawled by LLMs". In two days of tracking, the files AI crawlers requested most were the home page, sitemap.xml and robots.txt.
• People who tested it for months mostly saw nothing: "I've tested it on two of my sites for six or seven months, but nothing has changed either" (r/SEO).
• The positive reports are anecdotes. One company wrote that it "saw real referral sessions from ChatGPT appear soon after publishing the file, not huge, but meaningful enough to keep investing" (r/LLMDevs).
• The strongest case for the file came from someone who builds AI apps. For an app that searches the web, they argued, it is "WAY cheaper and simpler to process a reasonable llms.txt file" than full HTML pages (r/SEO). That is the job the format was designed for.
• A thread in r/ProWordPress asked how to fix a plumber's old phone number in ChatGPT. Most replies said llms.txt won't fix it: correct the number on the site, in its schema, in Google Business Profile and in directories, because that is where AI tools pick it up.
• The biggest thread in the sample started in December 2025, when SEOs spotted an llms.txt file on one of Google's documentation sites. Commenters later said it had been removed, and Google's May 2026 guide says Search doesn't use such files.

How we counted

We found the threads through Google (site:reddit.com searches on the topic) and saved them on October 10, 2026, with every visible comment. We counted people, not comments: each person once per thread, by the position they took. We didn't count the thread's author, replies that only asked a question or made a joke, or three replies that promoted a product or service. An AI model sorted the replies into positions, and we checked each label against the reply itself. Removed and deleted comments aren't included. People who answer on Reddit aren't a random sample of site owners, so read the shares as the mood among practitioners, not as a survey.

The threads:

• What is llms.txt and why does it matter for your content?, r/SEO, August 2025
• Do llms.txt files actually help websites appear in LLMs and AI agents? Any one tried?, r/seogrowth, March 2026
• How Do You Guys Manage llms.txt and llms-full.txt on wordpress?, r/ProWordPress, January 2026
• /llms.txt directory – A growing list of sites adopting the emerging llms.txt standard!, r/LLMDevs, November 2024
• Tell meDifference in Robots.txt and LLM file, r/TechSEO, May 2026
• Google added the LLMS.TXT file to their docs - any thoughts?, r/SEO, December 2025

Our analysis: what the documentation and data say

What we recommend: add llms.txt if you publish documentation or a product that developers use with AI tools, and check it with the llms.txt validator. On any other site, the same half hour is better spent on robots.txt access for AI crawlers and on pages that answer questions clearly.

Is llms.txt mandatory?

No. It is a community proposal rather than a web standard, and nothing breaks without it. Search engines and AI assistants crawl and cite sites without an llms.txt every day. Our own website audit lists a missing llms.txt only as a minor note.

How to create an llms.txt file

1. Start with the H1 and the summary: the site's name, then one or two sentences on what it is and who it's for. Write the sentence you'd like an AI to repeat about you.
2. Choose 10 to 30 pages that matter, such as product and pricing pages, docs, main guides and policies. Skip tag pages, archives and thin pages.
3. Group them under H2 headings like ## Product, ## Docs or ## Guides.
4. After each link, add a colon and a few words on what the page answers.
5. Move secondary links into ## Optional.
6. Save the file as plain text at the root of your domain, and update it when pages move.

Several documentation platforms and SEO plugins generate the file for you. If you write it by hand, a small build script that reads page titles and descriptions will keep it current. The blog section of our own file is maintained that way.

Where to put the llms.txt file

At the root of the domain: https://example.com/llms.txt. A few things go wrong often:

• Each host needs its own file, so docs.example.com gets a separate llms.txt.
• Serve it as plain text with a 200 status, as text/plain or text/markdown.
• Single-page apps often answer every address with the same HTML shell. In that case /llms.txt returns a web page instead of the file, so look at the response itself, not just whether the URL opens.
• Don't let /llms.txt redirect to an HTML page such as your home page, because tools will read that page instead.

How to generate llms-full.txt

llms-full.txt is a companion file: the full text of your main pages in a single Markdown document, so an AI tool can load all of it with one request. It is useful for documentation, where a coding assistant may want the whole reference at once.

To build one, join the Markdown sources of your docs in reading order, with each page under its own heading and its URL. Keep it a reasonable size. A file of several megabytes won't fit in a model's context, so split it by product if you have to. A marketing site doesn't need one; llms.txt is enough.

How to check your llms.txt file

1. Open it in a browser. You should see plain Markdown text; if you see your site's design, the server is returning HTML.
2. Run curl -I https://example.com/llms.txt and check for status 200 and a text content type.
3. Run the free llms.txt validator, which checks that the file exists and follows the format: H1 title, summary, sections and Markdown links.
4. Run Lighthouse in Chrome and look at the Agentic browsing category.

FAQ

What does "LLM text" mean?

Usually it means llms.txt, a text file written for large language models (LLMs) rather than for people or search engines. LLM is the type of model behind ChatGPT, Claude and Gemini.

Does ChatGPT read llms.txt?

OpenAI hasn't said that ChatGPT or its crawlers use llms.txt. ChatGPT finds pages through search and its own crawler, OAI-SearchBot. An agent browsing a site for a user might open the file, but it won't affect whether ChatGPT cites you.

Does llms.txt replace robots.txt?

No. robots.txt controls which crawlers may access which pages, and crawlers follow it. llms.txt only suggests which pages are most useful and can't allow or block anything.

Should llms.txt link to Markdown versions of pages?

If you have them, yes. The proposal recommends .md copies of pages because models read them more easily. If you don't, link to the normal HTML pages; the file is still valid.

Is llms.txt worth it, according to Reddit?

Mostly no. In six Reddit threads we analyzed, 34 of the 46 people who took a side said AI systems don't use the file. Five called it a harmless experiment, five said it helped them, and two said it's useful for documentation and AI apps rather than for visibility. Of the ten who said they had tried it, seven saw no change.