Does llms.txt Matter? What We Publish and What We Can Verify

Does llms.txt Matter? What We Publish and What We Can Verify
📌

Executive Summary & Key Takeaways

We publish both /llms.txt and /llms-full.txt on this site. Whether that meaningfully affects how AI systems index or cite Zovintra content is a different question - and one the evidence does not yet clearly answer.

Table of Contents
  1. What llms.txt Is
  2. What It Is Not
  3. Why We Publish It Anyway
  4. What Actually Affects AI Search Visibility
  5. How to Create a Useful llms.txt
  6. The Honest Verdict
  7. llms-full.txt: Is the Extended Version Worth It?
  8. What to Monitor After Publishing
  9. Common questions
  10. Sources

We publish both /llms.txt and /llms-full.txt on this site. Whether that meaningfully affects how AI systems index or cite Zovintra content is a different question - and one the evidence does not yet clearly answer.

What llms.txt Is

llms.txt is a proposed convention, not a W3C standard or an official specification. It was proposed by Jeremy Howard and published on 3 September 2024, and the site itself describes it as a proposal open for community input. The idea: put a Markdown-formatted file named /llms.txt at the root of your domain describing your site, its key pages, and its content structure. The goal is to give large language models a concise, human-readable summary they can use when processing your site - instead of having to parse HTML, JavaScript and navigation menus.

Two versions exist in practice. The /llms.txt file is a short summary covering what the site is and links to the most important pages (the spec asks for an H1 name, an optional summary, and sections of links). The /llms-full.txt file is a longer concatenation of key page content that some sites add; the llmstxt.org page does not mention it, so treat it as a community habit rather than part of the proposal.

What It Is Not

llms.txt is not a replacement for robots.txt and it does not control crawl access. It does not tell search engine crawlers whether to index a page - that is still the job of robots.txt, noindex meta tags, and canonical URLs. It does not guarantee that any AI system will read it, use it, or weight it in any particular way.

Google's own guide to AI features in Search says you "don't need to create new machine readable files, AI text files, or markup to appear in these features" and does not mention llms.txt. For the other major AI search and retrieval systems, we found no public documentation that they consume llms.txt as an input to their indexing or citation decisions. Perplexity, ChatGPT Browse, Claude and Gemini all have their own crawlers and summarisation pipelines. Whether any of them prioritise llms.txt content over the actual pages is not publicly confirmed by the providers.

Why We Publish It Anyway

We publish both files for two reasons. First, the cost is low: maintaining a concise description of our site and services is useful independently of AI crawlers - it doubles as a structured reference for anyone trying to understand what we do. Second, if some AI systems do start consuming it, having it in place is better than retrofitting. We make no claim that this is happening today.

The /llms.txt convention also aligns with the broader practice of structured content: sitemap.xml, robots.txt and schema.org markup are all ways of expressing machine-readable intent about your content. Adding one more machine-readable file has marginal cost and potential upside.

What Actually Affects AI Search Visibility

If your goal is to appear in AI-generated answers - sometimes called Generative Engine Optimisation or GEO - the factors that matter most are the same ones that matter for traditional search: clear, specific, factually grounded content that directly answers a question; structured headings that allow a crawling system to extract key claims; and links from credible sources that signal authority.

AI overview systems like Google's AI Overviews appear to prioritise content that already ranks well in organic search, content with clear authorship signals, and content that matches the specific question format well. How to show up in AI overviews covers this in more depth. Adding llms.txt does not substitute for this underlying content quality.

One factor that is plausibly helpful and measurable: ensuring your site's important content is accessible without JavaScript rendering. Google's JavaScript SEO guide says server-side or pre-rendering is still worthwhile because "not all bots can run JavaScript". If your core pages depend on client-side rendering to display their main content, a static or server-rendered version is more crawlable. This is a technical SEO issue that llms.txt does not solve.

How to Create a Useful llms.txt

If you decide to add llms.txt to your site, the most useful structure is:

  • A one-paragraph description of what the site is and who it is for.
  • A short list of the most important pages with their URLs and one-line descriptions.
  • Optionally, a note about what content you want AI systems to use (and what you would prefer they did not cite).
  • Keep the /llms.txt version concise - a long file defeats its purpose as a summary.
  • For /llms-full.txt, concatenate your most important pages in Markdown format, prioritising the content that best represents your expertise.

The Honest Verdict

llms.txt is a low-cost, low-risk addition to a site. Publish it because it is easy and harmless. Do not publish it expecting a measurable traffic change in the next six months. The factors that drive AI search visibility are content quality, site authority and structured data - not the presence of a text file.

If you are thinking about AI search visibility as part of a broader SEO strategy, that conversation is worth having with someone who can look at your actual traffic data, your content structure and your keyword coverage. A technical SEO review will give you a prioritised list of what to fix first.

llms-full.txt: Is the Extended Version Worth It?

The extended version, /llms-full.txt, is a larger document that concatenates the full content of your most important pages. The theory is that an AI system processing your site as part of a longer context window will find a clean Markdown version easier to work with than HTML. This is plausible - HTML is noisy, full of navigation boilerplate, script tags and CSS class names that consume context tokens without contributing meaning.

The practical question is how often an AI system would need to process your entire site in one request. For single-page lookups - a user asking a chatbot about a specific topic and the system retrieving a relevant page - the full-text file adds little. For a system building a comprehensive understanding of your business to answer complex questions, a well-structured /llms-full.txt could in theory be more useful than crawling dozens of individual pages.

Maintaining /llms-full.txt requires more effort than /llms.txt. The file needs to stay in sync with your content. Automating the generation from your existing pages or CMS is worth doing if you intend to keep the file accurate. A stale llms-full.txt that includes outdated pricing, discontinued services or old product descriptions is worse than no file at all, because an AI system may confidently cite outdated information from it.

What to Monitor After Publishing

Once you have published both files, there is no immediate feedback loop. Unlike a sitemap, which Google Search Console can validate and report on, there is no official tool that tells you whether an AI crawler fetched your llms.txt and used it. You can check server access logs for requests to /llms.txt from known AI crawler user agents, but the absence of a request is not evidence that the file is not working - it may simply mean the crawler fetched it once and cached it.

The better signal is whether your site appears in AI-generated answers for relevant queries, and whether those answers are accurate. If an AI system is citing outdated or incorrect information about your services, that is a content quality and freshness problem that llms.txt maintenance may help with. If the answers are accurate, the file is doing its job - whether or not you can attribute it directly to llms.txt versus good page content.

Common questions

Does llms.txt help me show up in ChatGPT or AI Overviews?

The evidence does not clearly say so. Google's guide to AI features says you don't need new machine readable files or AI text files to appear, and for other systems we found no public documentation that they use llms.txt. Publish it because it is cheap, not for traffic.

Is llms.txt an official standard?

No. It is a proposed convention published by Jeremy Howard on 3 September 2024, and its own site describes it as a proposal open for community input. It does not control crawl access; robots.txt, noindex tags and canonical URLs still do that job.

Who can review my site's AI search visibility?

The factors that matter are clear, factually grounded content that answers a question directly, structured headings, links from credible sources, and content accessible without JavaScript. A technical SEO review from Zovintra will give you a prioritised list of what to fix first.

Sources

TECHNICAL CONSULTATION

Building something similar?

Talk to our engineering team about your architecture, roadmap, and delivery timeline.

Request a Technical Review →

Related Reading

← Back to all posts