There’s a lot of noise right now about adding an llms.txt file to your website. Google’s John Mueller just weighed in — and most people are reading his response wrong.
Mueller called it “purely speculative for now.” And I want to focus on those last two words for a second: for now. That’s not someone saying it’ll never matter. That’s someone leaving the door open.
What llms.txt Actually Is
It’s a plain text file that lives in your root directory. The idea is simple: instead of an AI system having to crawl your entire website to understand what’s on it, you give it a tidy summary it can read directly.
Think of it like a table of contents, but for AI.
What Mueller Actually Said
According to him, none of the major AI systems are actively using the file right now. So at this exact moment, it does nothing.
He also made the point that if you’re using AI to generate the file, couldn’t an AI system just create it for you itself anyway? Fair enough on the surface. But I have a problem with that logic.
The volume of content being generated online is increasing at a rate that’s hard to overstate. At some point, the compute required to crawl and process all of it becomes a real constraint. A tidy summary file that an AI can grab in one hit, instead of wading through an entire site, starts to look a lot like an efficiency gain for their systems — not just yours.
So “AI can do it itself” doesn’t quite hold up when you scale it.
So Should You Bother?
If you already have the file: leave it. There’s no downside to it sitting in your root directory.
If you don’t have one: it takes about two minutes to generate with AI — the same kind of repeatable AI task that most business owners are sleeping on. Drop it in. The worst case is it does nothing. The best case is it’s already in place when these systems start using it — and “for now” has a way of changing quickly in this space.
What Mueller Said Actually Matters Right Now
This is the part most people glossed over. He pointed to two things worth your attention:
WebMCP is a Google-backed standard that lets AI agents properly interact with your site. It’s early, but it’s the direction things are heading — AI not just reading your content, but being able to navigate and interact with it.
Don’t block AI agents. This is probably the most important thing to take away from this whole conversation. If your robots.txt or other config is set up to block AI crawlers, you’re cutting yourself off from how search and discovery is increasingly going to work. These agents are crawling sites now, and they’re going to play a real part in how businesses get found online.
The Short Version
The llms.txt file isn’t doing anything today. But dismissing it entirely based on a “for now” comment from one person at Google isn’t the move either. It costs you almost nothing to have it. And if you’re not sure whether your site is blocking AI agents, that’s worth checking today.
If you want help getting your site set up correctly for how AI systems are going to interact with it, that’s exactly what our AI consulting work covers — get in touch.
