The best free llms.txt validator — checks your file against the llmstxt.org v2 spec, HEAD-checks every link, validates Content-Type headers, cross-references robots.txt for AI crawler conflicts, and simulates access from 5 crawlers including GPTBot, ClaudeBot and PerplexityBot. Aligned with Chrome Lighthouse 13.3 Agentic Browsing.
We append /llms.txt automatically if you enter just a domain. Spec: llmstxt.org v2. Up to 100 links checked per audit.
uploaded over TLS · deleted within one hour
Every tool on ToolsVale is scoped to one job. Here is exactly what this one does, and what it does not — so you know whether it fits before you start.
Notes from other visitors. Sign in with Google to add yours, or send a private suggestion straight to the developer.
Sign in to leave a comment. We only use Google — no passwords, no email spam.
Sign in with GoogleNo comments yet. Be the first.
Paste your site URL or the direct /llms.txt link — we append /llms.txt automatically if you enter just a domain. Or switch to paste mode and drop in the raw file content.
Your file is parsed against the llmstxt.org v2 spec, every link is HEAD-checked (with GET fallback), robots.txt is parsed for AI crawler conflicts, and we simulate access as 5 different crawlers to catch WAF blocks and cloaking.
Spec Compliance, Delivery & Accessibility, Link Integrity, and Agent Readiness — each scored 0–100 with severity caps. Every finding carries a rule ID, line number, evidence snippet, and a Lighthouse prediction badge.
Deterministic fixes come as ready-to-paste code blocks with platform tabs (Nginx, Apache, Cloudflare, Vercel, Netlify). Review-required fixes flag human judgment. Share the audit via a 48-hour link — no signup for you or your team.
| Input | Output | Typical saving | Best for |
|---|---|---|---|
| site URL (auto-appends /llms.txt) | 4-score audit with findings | full audit including link health, robots.txt, and crawler simulation | |
| direct /llms.txt URL | 4-score audit with findings | testing a specific llms.txt at a subpath like /docs/llms.txt | |
| pasted raw content | spec compliance score + findings | validating a file before deploying — no link or crawler checks |
We ran the same files through the popular alternatives first. These are the gaps we found, and what this tool does instead.
We HEAD-check every link in your file (up to 100) with GET fallback for servers that reject HEAD. Each result includes the HTTP status, redirect chain, response time, and exact line number in your file. A dead link in llms.txt breaks every agent that follows it.
We fetch your file as 5 different user agents (Chrome, Googlebot, GPTBot, ClaudeBot, PerplexityBot), normalize the response body, and compare content hashes. If any crawler is blocked or sees different content, the crawler access matrix shows exactly which one and why.
We fetch and parse your robots.txt using the standard robots protocol, check Allow/Disallow rules for each AI crawler against your /llms.txt path, and flag contradictions with the specific robots.txt line that causes the block — plus a copy-paste fix to add an Allow rule.
We fetch your llms.txt as five different user agents — Chrome Desktop, Googlebot, GPTBot (OpenAI), ClaudeBot (Anthropic), and PerplexityBot — and compare the normalized response hashes. Cloudflare Bot Management and AWS WAF routinely challenge AI crawlers with 403 or 503 while serving 200 to Chrome. A standard browser-based test misses this entirely. Our crawler access matrix shows each UA's HTTP status, content parity, and response time side by side.
Your llms.txt might be perfectly formatted but useless if your robots.txt blocks the crawlers that need to read it. We fetch your robots.txt, parse it using the standard protocol's exact Allow/Disallow semantics (not naive string matching), and test each AI crawler's access to /llms.txt. When we find a conflict — like GPTBot blocked site-wide — we show the specific robots.txt line that causes it and offer a copy-paste Allow rule for the exact path.
Every link in your file gets a HEAD request, with automatic GET fallback for servers that return 405 or 403 to HEAD (common on Cloudflare and Fastly). We report the HTTP status, redirect chain, response time, and the exact line number in your file. The llms.txt spec is a curated map of your important pages — a single dead link breaks the agent that follows it. Across 523+ real-world llms.txt files analyzed by Mdream, the median file contains 83 links. We check up to 100 per audit.
The spec requires your file to be served as text/plain or text/markdown, not text/html. Webflow, Framer, and Shopify serve /llms.txt as HTML by default — the most common deployment mistake. We check the Content-Type header, charset, HTTP status, redirect chains, and scan the response body for WAF challenge signatures from Cloudflare, Akamai, and Sucuri. Every fix comes with platform-specific configuration for Nginx, Apache, Cloudflare, Vercel, and Netlify.
Since May 2026, Chrome Lighthouse 13.3 includes llms.txt in its Agentic Browsing audit — every PageSpeed Insights and Chrome DevTools report now checks whether your file exists and follows the spec. Our validator maps to the same checks Lighthouse runs and predicts your llms.txt audit result: PASS, FAIL, or N/A (file absent — Lighthouse correctly treats a missing file as not applicable, not a failure). Fix your issues here first, then confirm with the real Lighthouse audit.
An llms.txt file is the difference between a site AI assistants can navigate and one they have to guess about. It’s a plain Markdown file at your site root — a curated index of your most important pages, written for machines that need to find the right content without wading through HTML, ads, and JavaScript. The ToolsVale llms.txt Validator is the free llms.txt validator built for how the file actually gets consumed in 2026: it checks your file against the llmstxt.org v2 specification (August 2026), HEAD-checks every link, validates your Content-Type headers, cross-references your robots.txt for AI crawler conflicts, and simulates access from five different crawlers — Chrome, Googlebot, GPTBot, ClaudeBot, and PerplexityBot — to catch the WAF blocks and cloaking that browser testing misses entirely.
This page explains what an llms.txt validator does, why most existing validators catch less than half of the real problems, and how this tool closes the gaps that actually matter for AI agent discoverability. If you’re in a hurry, scroll up and paste your URL — the audit runs in under 15 seconds and streams results as they arrive.
llms.txt is a proposed standard created by Jeremy Howard (co-founder of Answer.AI and fast.ai) in September 2024. The idea is simple: web pages are built for people, with navigation menus, ads, cookie banners, and JavaScript that make it hard for AI agents to find the content that matters. An llms.txt file gives AI assistants a clean, curated map — a Markdown document listing your most important pages with one-line descriptions, organized into sections.
The specification defines a strict format: exactly one H1 with your site name (the only required element), an optional blockquote summary that AI models often quote verbatim, optional body text, and H2 sections containing curated link lists. The file must be served as text/plain or text/markdown at your site root (/llms.txt) or any subpath.
The spec sounds simple, but the real world is messy. Webflow, Framer, and Shopify serve the file as text/html by default. WAFs silently block AI crawlers. robots.txt rules ban GPTBot from the very file it’s supposed to read. Links go stale, descriptions get skipped, and the file drifts out of sync with the site. That’s why validation matters — and why a validator needs to check far more than just Markdown syntax.
Three things changed between 2024 and 2026 that made shallow validation useless:
Chrome Lighthouse 13.3 added llms.txt to Agentic Browsing audits. Since May 7, 2026, every PageSpeed Insights run and Chrome DevTools audit checks whether your llms.txt exists and follows the spec. The audit sits in the new “Agentic Browsing” category alongside WebMCP, accessibility tree, and CLS checks. A broken llms.txt now shows up as a visible failure in every developer’s workflow — and the search queries for “llms.txt validator” have surged accordingly. But Lighthouse’s check is deliberately limited: it flags server errors (5xx = FAIL) and marks a missing file as N/A (not a failure). It doesn’t check your links, your robots.txt, your Content-Type, or whether AI crawlers can actually reach the file. That’s the gap a serious validator fills.
Adoption accelerated, but most implementations are broken. An Ahrefs study of 137,000 sites in June 2026 found roughly 38,000 with a valid llms.txt — but 97% of those files received zero requests. Among the top 1,000 Tranco sites, 186 served a file and only 64 were valid. The other 122 had problems. Most sites that try still get it wrong. The issues aren’t exotic edge cases — they’re wrong Content-Type headers, dead links, robots.txt contradictions, and missing blockquotes.
The spec evolved to v2. The August 2026 revision added subpath support (/docs/llms.txt covering everything under /docs/), BOM tolerance, and link relation headers (rel="alternate" for Markdown versions, rel="describedby" pointing to the covering llms.txt). Validators built against v1 miss subpath scoping violations and don’t know about the new conventions.
Every existing free validator we tested — llms-txt.io, llmstxtvalidator.dev, Rankability, MRS Digital, llmstxtchecker.net, and others — focuses on Markdown structure: does your H1 exist, is there a blockquote, are links in the right format. That’s the floor. None of them HEAD-check every link. None validate Content-Type headers. None cross-reference robots.txt. None simulate access from AI crawlers. The ToolsVale llms.txt Validator was built specifically to cover everything above the floor.
Every link in your file gets a HEAD request. If the server returns 405 or 403 to HEAD (common on Cloudflare and Fastly CDNs), we automatically retry with a controlled GET request — fetching headers only, closing the body stream immediately. For each link we report the HTTP status code, any redirect chain, response time, and the exact line number in your file. Dead links get flagged as LLMS-013, redirects as LLMS-014 with the final URL offered as a copy-paste deterministic fix, and DNS failures as LLMS-015. We check up to 100 links per audit with concurrent requests and per-host throttling to avoid rate-limiting the target site.
Across 523+ real-world llms.txt files analyzed by Mdream, the median file contains 83 links. A single dead link breaks every agent that follows it — and in a file that’s designed to be a curated index, a broken link is worse than no link at all.
We fetch your llms.txt as five different user agents: Chrome Desktop (baseline), Googlebot (search), GPTBot (ChatGPT/OpenAI), ClaudeBot (Anthropic), and PerplexityBot (Perplexity). The responses are normalized — whitespace-insensitive content hashes, not raw byte counts — and compared. If any crawler gets a different HTTP status, different content, or is blocked entirely, the crawler access matrix shows exactly which one and what happened.
This catches the silent failures that browser testing can never reveal. Cloudflare Bot Management and AWS WAF routinely serve 403 or challenge pages to AI crawlers while returning a clean 200 to Chrome. Your llms.txt could be perfectly formatted and completely invisible to GPTBot. We label this capability “user-agent access simulation” because we’re testing server behavior from our infrastructure, not claiming to be the actual crawlers fetching from their IP ranges.
Your robots.txt and your llms.txt live on the same domain but serve different purposes. robots.txt tells crawlers what they may access; llms.txt tells them what’s important. If your robots.txt contains User-agent: GPTBot / Disallow: /, GPTBot is banned from your entire site — including /llms.txt. We parse your robots.txt using Python’s standard RobotFileParser (correct Allow/Disallow semantics, not naive string matching), test each AI crawler’s access rules against your llms.txt path, and flag contradictions with the specific robots.txt line and a copy-paste Allow: /llms.txt fix for each affected crawler.
The spec requires text/plain or text/markdown. Webflow, Framer, and Shopify serve llms.txt as text/html by default — the single most common deployment mistake. We check the Content-Type header, charset, HTTP status code, redirect chains, and scan the response body for WAF challenge signatures from Cloudflare, Akamai, and Sucuri. Every Content-Type fix comes with platform-specific configuration tabs: Nginx, Apache, Cloudflare Rules, Vercel vercel.json, and Netlify _headers.
Our validator maps to Lighthouse 13.3’s exact llms.txt audit behavior. Lighthouse marks a missing file (404) as “Not Applicable” — not a failure. It marks server errors (5xx) and spec violations as “FAIL”. A valid, accessible file is “PASS”. We predict your result so you can fix issues before running the real Lighthouse audit in Chrome DevTools or PageSpeed Insights. We’re explicit about scope: this prediction covers only the llms.txt portion of Lighthouse’s Agentic Browsing category — it doesn’t predict the WebMCP, accessibility tree, or CLS audits.
Every audit produces four scores on a 0–100 scale, each with a breakdown. Severity caps prevent a passing average from hiding a critical failure: one critical issue caps its category at 25, and three or more errors cap at 50.
Spec Compliance checks your file’s structural conformance to the llmstxt.org v2 specification. H1 presence and uniqueness, blockquote placement, heading hierarchy (no headings between H1 and first H2), link syntax, URL absoluteness, duplicate detection, subpath scoping, and valid CommonMark Markdown. This score is deterministic — the same file produces the same score every time.
Delivery & Accessibility validates the HTTP response. Content-Type (must be text/plain or text/markdown), charset, status code, redirect chains, and WAF challenge detection. A file that parses correctly but is served as text/html still fails here.
Link Integrity reports the health of every URL in the file. Dead links, redirect chains, DNS failures, timeouts, and slow responses. This score reflects how reliably agents can follow the links you’ve curated.
Agent Readiness covers the ecosystem around the file: robots.txt permissions for each AI crawler, multi-UA access parity, llms-full.txt companion status, per-link description coverage, and token estimation for context-window efficiency.
After validating production llms.txt files across hundreds of sites, these are the mistakes we see most often — and that no other free validator catches:
Content-Type served as text/html. The #1 deployment mistake. Webflow, Framer, Shopify, and many static-site hosts serve /llms.txt as HTML by default because the file extension is .txt, not .html, but their routing treats all files as web pages. LLMS-020 catches this with platform-specific fix instructions.
robots.txt blocks AI crawlers from /llms.txt. Sites that added Disallow: / for GPTBot or ClaudeBot to prevent training-data scraping inadvertently blocked these crawlers from reading their llms.txt — the inference-time file that’s specifically designed for them. LLMS-030 and LLMS-031 flag this with an Allow: /llms.txt patch.
Dead links from site redesigns. A site restructures its URLs, the marketing pages get redirected, but the llms.txt file still points to the old paths. LLMS-013 catches dead links; LLMS-014 catches redirects and offers the final URL as a deterministic fix.
Missing blockquote summary. Only 50% of real-world llms.txt files include the blockquote, according to Mdream’s analysis. AI models use this as a one-line site definition — skipping it means the model has to guess. LLMS-004 flags this as a recommendation (not a spec failure, since the blockquote is technically optional).
Bare links without descriptions. The spec supports - [Name](url): description but many generators omit the description. Without it, an agent has to fetch every page to decide which one answers its query. LLMS-011 counts coverage and flags low-description files.
Subpath scope violations. The v2 spec says /docs/llms.txt covers only URLs under /docs/. If a subpath file contains links to /blog/ or /pricing/, those links are technically out of scope. LLMS-072 catches this.
WAF challenge pages served as the llms.txt content. The server returns HTTP 200, but the body is a Cloudflare or Akamai JavaScript challenge page instead of your actual Markdown. LLMS-024 scans for known WAF challenge signatures in the response body.
These three files serve different purposes and complement each other:
robots.txt tells crawlers what they may access. It’s a permission layer — “you can crawl this, you can’t crawl that.” It predates AI agents by decades and was designed for search engine indexing bots.
sitemap.xml lists every indexable page on your site. It’s comprehensive and flat — a URL inventory for search crawlers, not a curated guide.
llms.txt tells AI agents what matters. It’s a curated map — not every page, but the 10–30 that genuinely answer the questions agents ask. It includes descriptions, groupings, and context that sitemaps don’t provide.
A well-configured site has all three. robots.txt controls access, the sitemap ensures discovery, and llms.txt ensures understanding. Our validator checks the llms.txt file itself and cross-references robots.txt to catch contradictions between the permission layer and the discovery layer.
Since May 7, 2026, Chrome Lighthouse 13.3 includes an “Agentic Browsing” audit category that ships on by default. It evaluates four areas: llms.txt presence and validity, WebMCP tool registration, accessibility tree well-formedness, and Cumulative Layout Shift. The category produces a pass-ratio score, not a traditional 0–100 weighted score — Google is explicit that this is early and data-gathering.
For llms.txt specifically, Lighthouse checks whether the file exists, returns a valid HTTP response, and follows the spec. A 404 is marked N/A (not applicable) — Lighthouse correctly treats a missing file as optional, not a failure. A 5xx server error is a FAIL. A 200 with spec violations is a FAIL.
Our validator predicts this specific audit result. But we’re careful about scope: this prediction covers only the llms.txt portion. Lighthouse’s other three Agentic Browsing checks (WebMCP, accessibility tree, CLS) are separate evaluations that require different tools. We never claim to predict the full Agentic Browsing score.
One important nuance: Google’s official AI Optimization Guide (May 2026) explicitly states that llms.txt is not needed for Google Search rankings or AI Overviews. Lighthouse checking for it is an “agent readiness” signal, not a ranking signal. The real use case is AI coding assistants (Cursor, GitHub Copilot, Claude Code) and AI answer engines (ChatGPT, Perplexity, Claude) that fetch the file at inference time to understand site structure.
This tool is built for three audiences. SEO teams and GEO practitioners who need to verify their llms.txt implementations pass Lighthouse’s Agentic Browsing audit and work correctly for AI answer engines. Developers shipping documentation sites where coding assistants like Cursor and Claude Code rely on llms.txt to find API references and tutorials. And agencies auditing client sites — the 48-hour shareable link and JSON export make it easy to hand off findings to a development team without requiring anyone to sign up.
If you need to create an llms.txt file, start with our llms.txt Generator — it discovers your pages, helps you curate the important ones, and exports a spec-compliant file. Then come here to validate it against the full audit: spec compliance, link health, Content-Type, robots.txt, and crawler access.
Scroll to the tool above, paste your URL, and click Validate. The audit runs in under 15 seconds and streams results as they arrive. The tool is free, no signup, no watermark. For related audits, see our Robots.txt Generator, Sitemap Validator, AI Readability Checker, and Schema Validator — together they cover the full AI discoverability checklist for 2026.
Last reviewed September 2026 · this tool runs server.
An llms.txt validator checks whether your /llms.txt file follows the specification at llmstxt.org, is accessible to AI crawlers, and contains working links. The file is a plain Markdown document at your site root that tells AI assistants which pages on your site matter and what they're about. A validator ensures the file is correctly structured, properly served, and reachable by the agents it's designed for.
Three things no other free tool does. First, we HEAD-check every link in your file (up to 100) with GET fallback — not sampling, not paid-only. Second, we simulate access from five different crawlers (Chrome, Googlebot, GPTBot, ClaudeBot, PerplexityBot) and detect blocks and cloaking. Third, we cross-reference your robots.txt to catch contradictions where AI crawlers are banned from the file they're meant to read. Every finding includes a rule ID, line number, evidence snippet, and platform-specific copy-paste fix.
llms-txt.io is the closest competitor — it checks structure and agent readiness, samples a few links, and detects the agent-instructions pattern. But link checking and shareable results require login or a paid plan. llmstxtvalidator.dev validates structure and supports ai.txt, but does not check link health, Content-Type headers, robots.txt, or crawler access. Neither tool does multi-UA simulation, robots.txt cross-referencing, or Lighthouse 13.3 alignment — those are exclusive to our validator.
The llmstxt.org v2 specification requires exactly one H1 heading with your site or project name. Beyond that, the spec recommends a blockquote summary (AI models often quote this verbatim), optional body text without headings, and H2 sections containing Markdown lists of links with optional descriptions. The file must be valid CommonMark Markdown served as text/plain or text/markdown, not text/html. An "Optional" section is a convention for links agents can skip in tight context windows.
Serving a valid llms.txt is only half the job — if your CDN or WAF blocks GPTBot with a 403 while Chrome gets a 200, AI agents can never read it. Cloudflare Bot Management and AWS WAF routinely challenge AI crawlers silently. Our multi-UA simulation fetches your file as five different crawlers and shows exactly which ones are blocked, so you can add WAF exceptions for the /llms.txt path specifically.
Your robots.txt and your llms.txt live on the same domain but serve different purposes. robots.txt tells crawlers what they may access; llms.txt tells them what's important. If your robots.txt contains "User-agent: GPTBot / Disallow: /", GPTBot is banned from your entire site — including /llms.txt. Our cross-reference engine parses your robots.txt using the standard protocol semantics and tests each AI crawler's Allow/Disallow rules against your llms.txt path. When a conflict exists, we show the exact line and give you a copy-paste fix.
No — we never auto-apply changes to your files. Every fixable issue includes a copy-paste code block classified by safety tier: deterministic fixes are safe to paste directly (like correcting a Content-Type header), safe-with-review fixes are likely correct but need a quick human check (like updating a redirected URL to its final destination), and human-required fixes explain what needs writing (like adding a meaningful blockquote summary). Platform-specific tabs show the exact configuration for Nginx, Apache, Cloudflare, Vercel, and Netlify.
Yes. Switch to paste mode and drop in your raw llms.txt content. The validator runs all spec compliance checks (structure, links syntax, Markdown validity) without needing a live URL. Link health, robots.txt, and crawler simulation are URL-only features because they require fetching live servers.
No. Every feature is free for anonymous visitors on first click: the full 4-score audit, link health checking, robots.txt cross-reference, multi-UA simulation, shareable links, and JSON export. No signup, no email, no credit card, no watermark, no "upgrade for full report" upsell. Rate limit is 30 requests per minute per IP, which is more than enough for any normal use.
Spec Compliance checks your file's structural conformance to the llmstxt.org v2 specification — H1 presence and uniqueness, blockquote, heading hierarchy, link syntax, and URL absoluteness. Delivery & Accessibility validates the HTTP response: Content-Type, charset, status code, redirects, and WAF detection. Link Integrity reports the health of every URL in the file — dead links, redirects, DNS failures, and timeouts. Agent Readiness covers robots.txt permissions, multi-UA access, llms-full.txt companion status, and link description coverage. Severity caps prevent one critical failure from being diluted by many passing checks.
Yes. Click "Share audit" and we generate a 48-hour shareable link — no signup required for you or anyone viewing it. You can also export the full audit as a JSON file for CI/CD integration or archival. The JSON includes all findings, scores, link results, crawler results, and robots.txt analysis.
Yes. The v2 spec supports subpath files — /docs/llms.txt covers URLs under /docs/. Enter the full URL and we validate accordingly, including a check that links in the file stay within the declared scope (LLMS-072). If a subpath file contains links outside its scope, we flag it.
Since May 2026, Chrome Lighthouse 13.3 includes llms.txt in its Agentic Browsing audit. Our prediction maps to Lighthouse's exact behavior: PASS means your file exists and follows the spec; FAIL means the file is present but has structural errors or a server error; N/A means the file is absent (Lighthouse correctly treats a missing file as not applicable, not a failure). This prediction covers only the llms.txt portion of Lighthouse's Agentic Browsing category, not WebMCP, accessibility tree, or CLS.
Build a clean, spec-compliant llms.txt — the curated file that tells ChatGPT, Claude and Perplexity which of your pages matter. Discover pages, keep the best, describe them, export
Build a valid robots.txt with the right allow/disallow rules, crawl-delay and sitemap line — with warnings for the mistakes that accidentally block your whole site from Google.
Free sitemap validator that actually fetches every URL. Checks XML syntax, live URL health, robots.txt, and hreflang across Google, GPTBot, ClaudeBot & PerplexityBot. No signup.
