From de30a88d3e3e83ad147527b883d70e2e740ec9b9 Mon Sep 17 00:00:00 2001 From: Bloxster Date: Mon, 17 Aug 2026 14:53:27 +0200 Subject: [PATCH 1/2] =?UTF-8?q?docs(site):=20address=20review=20=E2=80=94?= =?UTF-8?q?=20describedby=20relation,=20guard=20hole,=20corpus=20claims?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Review feedback from @yperbasis and Copilot on #23336. Five fixes. 1. Advertise llms.txt with rel="describedby", not rel="alternate". The llmstxt.org proposal defines describedby for the llms.txt file that covers a page, and reserves alternate + text/markdown for a *per-page* Markdown representation. A site-wide index is not an alternate representation of every page, and v2-aware agents look for describedby. llms-full.txt is no longer advertised in head at all: it describes no single page. It stays discoverable through llms.txt, the sitemap, and the MCP docs page. 2. Close a hole in the card-count guard. `if not cards: return None` ran before the count was taken, so a grid where *every* card failed to parse was indistinguishable from an ordinary prose page: the caller fell back to strip_mdx and the guard never ran — silently degrading the exact case it exists to catch. Count first, parse second, and return None only when the page has no cards at all. 3. Stop claiming llms-full.txt holds "every documentation page, in full". It does not: synthesize_landing replaces the whole body of a card-grid page with its card list, so why-using-erigon loses its introduction and prose. Describe the corpus as cleaned rather than verbatim, and say what is dropped. 4. Stop grouping llms.txt with llms-full.txt as "the whole documentation as one plain-text file" on why-using-erigon. llms.txt is only an index. 5. Drop "nothing else links to them" from the config comment — this PR adds the MCP page links, which makes it false. Also refresh the stated file size, 420 KB -> 430 KB. --- docs/site/docs/fundamentals/mcp.mdx | 6 ++-- .../docs/get-started/why-using-erigon.mdx | 2 +- docs/site/docusaurus.config.ts | 28 ++++++++----------- docs/site/scripts/generate-llms.py | 12 ++++++-- docs/site/scripts/test_generate_llms.py | 20 +++++++++++++ docs/site/static/llms-full.txt | 6 ++-- llms-full.txt | 6 ++-- 7 files changed, 53 insertions(+), 27 deletions(-) diff --git a/docs/site/docs/fundamentals/mcp.mdx b/docs/site/docs/fundamentals/mcp.mdx index a31888e6122..8ddd30d6769 100644 --- a/docs/site/docs/fundamentals/mcp.mdx +++ b/docs/site/docs/fundamentals/mcp.mdx @@ -317,13 +317,15 @@ MCP connects a model to your **running node**. To let a model answer questions a | File | Contents | Use it when | |------|----------|-------------| | [llms.txt](https://docs.erigon.tech/llms.txt) | Page index with titles and descriptions | The model fetches pages on demand, or its context window is small | -| [llms-full.txt](https://docs.erigon.tech/llms-full.txt) | Every documentation page, in full, as one plain-text file | You want the whole corpus in context, offline, with nothing scraped | +| [llms-full.txt](https://docs.erigon.tech/llms-full.txt) | The text of every documentation page, as one plain-text file | You want the whole corpus in context, offline, with nothing scraped | Both are generated from the same Markdown source as this site by a script in the repository, and CI fails when a documentation change lands without regenerating them, so they cannot quietly go stale. +The corpus is cleaned rather than copied verbatim: MDX components are stripped, and section index pages built as card grids are reduced to the list of links they present. Read it as the documentation's prose, not as a byte-for-byte mirror of the rendered site. + ```bash # Give a local model the whole documentation, with no network access curl -sO https://docs.erigon.tech/llms-full.txt ``` -The full file is around 420 KB. If your model's context cannot hold it, use `llms.txt` to pick the pages you need, or slice the sections you want out of the full file — each page begins with a `# Title` heading. +The full file is around 430 KB. If your model's context cannot hold it, use `llms.txt` to pick the pages you need, or slice the sections you want out of the full file — each page begins with a `# Title` heading. diff --git a/docs/site/docs/get-started/why-using-erigon.mdx b/docs/site/docs/get-started/why-using-erigon.mdx index d0d61a4e09a..66b91ca6c14 100644 --- a/docs/site/docs/get-started/why-using-erigon.mdx +++ b/docs/site/docs/get-started/why-using-erigon.mdx @@ -180,4 +180,4 @@ The interface is **read-only** and localhost-bound — no risk of unintended sta claude mcp add --transport sse erigon http://127.0.0.1:8553/sse ``` -Where MCP lets a model query your *node*, [llms.txt and llms-full.txt](../fundamentals/mcp#these-docs-as-a-single-file) let it read these *docs* — the whole documentation as one plain-text file, for offline use. +Where MCP lets a model query your *node*, [llms.txt and llms-full.txt](../fundamentals/mcp#these-docs-as-a-single-file) let it read these *docs*: an index of every page, and the documentation text as one plain-text file for offline use. diff --git a/docs/site/docusaurus.config.ts b/docs/site/docusaurus.config.ts index d9623d9770b..67d82496de4 100644 --- a/docs/site/docusaurus.config.ts +++ b/docs/site/docusaurus.config.ts @@ -86,27 +86,21 @@ export default async function createConfig(): Promise { attributes: {}, innerHTML: 'window.plausible=window.plausible||function(){(plausible.q=plausible.q||[]).push(arguments)},plausible.init=plausible.init||function(i){plausible.o=i||{}};plausible.init()', }, - // Machine-readable copies of this site (llmstxt.org convention). Advertised - // on every page so crawlers and agents can find them without guessing the - // path — they are static files, so nothing else links to them. + // Point every page at the llms.txt index that covers it. `describedby` is + // the relation the llmstxt.org proposal defines for exactly this; its + // `alternate` + `text/markdown` pairing is reserved for a *per-page* + // Markdown representation, which this site-wide index is not. + // llms-full.txt is deliberately not advertised here: it is not a + // description of any single page. It stays discoverable through llms.txt, + // the sitemap, and the MCP docs page. { tagName: 'link', attributes: { - rel: 'alternate', - type: 'text/plain', + rel: 'describedby', href: 'https://docs.erigon.tech/llms.txt', title: 'Erigon Documentation — page index for LLMs', }, }, - { - tagName: 'link', - attributes: { - rel: 'alternate', - type: 'text/plain', - href: 'https://docs.erigon.tech/llms-full.txt', - title: 'Erigon Documentation — full text for LLMs', - }, - }, ], plugins: [ @@ -172,9 +166,9 @@ export default async function createConfig(): Promise { priority: 0.5, ignorePatterns: ['/search'], // The llms.txt artifacts live in static/, so Docusaurus never routes - // them and the default sitemap omits them. Nothing else on the web - // links to them either, which leaves them unindexable and unreachable - // by search — append them explicitly. + // them and the default sitemap omits them — which leaves them + // unindexable by search. Append them explicitly. This is also how + // llms-full.txt stays reachable, since it is not advertised in head. createSitemapItems: async ({defaultCreateSitemapItems, ...rest}) => [ ...(await defaultCreateSitemapItems(rest)), {url: 'https://docs.erigon.tech/llms.txt', changefreq: 'weekly', priority: 0.5}, diff --git a/docs/site/scripts/generate-llms.py b/docs/site/scripts/generate-llms.py index e7a115c78e9..cc028272ea8 100644 --- a/docs/site/scripts/generate-llms.py +++ b/docs/site/scripts/generate-llms.py @@ -321,6 +321,15 @@ def synthesize_landing(raw_body): Card hrefs that start with `/` are rewritten to absolute BASE_URL form so bullets remain useful when the body is read out of context. """ + # Count first, parse second. Deciding "this is not a card grid" on an empty + # parse result would make total parser failure indistinguishable from an + # ordinary prose page: the caller would fall back to strip_mdx and the guard + # below would never run — silently degrading exactly the case it exists to + # catch. + expected = raw_body.count('lp-card-title') + if not expected: + return None + cards = [] for link in _LANDING_LINK_RE.finditer(raw_body): href, block = link.group(1), link.group(2) @@ -330,13 +339,10 @@ def synthesize_landing(raw_body): continue cards.append((href, _card_text(title.group(1)), _card_text(desc.group(1)))) - if not cards: - return None # A card that parses to nothing is invisible in the output, and `--check` # only compares generated output against committed output — so a systematic # parser regression stays green in CI while the corpus quietly loses pages. - expected = raw_body.count('lp-card-title') if len(cards) != expected: raise SystemExit( f"synthesize_landing dropped cards: parsed {len(cards)} " diff --git a/docs/site/scripts/test_generate_llms.py b/docs/site/scripts/test_generate_llms.py index b7cce696977..db31798a339 100644 --- a/docs/site/scripts/test_generate_llms.py +++ b/docs/site/scripts/test_generate_llms.py @@ -221,6 +221,26 @@ def test_dropped_card_raises_instead_of_silently_shrinking(self): g.synthesize_landing(body) self.assertIn("dropped cards", str(ctx.exception)) + def test_all_cards_malformed_raises_rather_than_falling_back(self): + """Total parser failure must not look like an ordinary prose page. + + Returning None here would send the caller to strip_mdx, degrading the + page silently — the exact outcome the guard exists to prevent. + """ + body = """ +
+ +
A
+ + +
B
+ +
+""" + with self.assertRaises(SystemExit) as ctx: + g.synthesize_landing(body) + self.assertIn("parsed 0 of 2", str(ctx.exception)) + class LeadingH1StripTests(unittest.TestCase): """The build() body-prep step strips a leading H1 to avoid duplicate headings.""" diff --git a/docs/site/static/llms-full.txt b/docs/site/static/llms-full.txt index b875b3d5a63..5cd8769c6d4 100644 --- a/docs/site/static/llms-full.txt +++ b/docs/site/static/llms-full.txt @@ -3930,16 +3930,18 @@ MCP connects a model to your **running node**. To let a model answer questions a | File | Contents | Use it when | |------|----------|-------------| | [llms.txt](https://docs.erigon.tech/llms.txt) | Page index with titles and descriptions | The model fetches pages on demand, or its context window is small | -| [llms-full.txt](https://docs.erigon.tech/llms-full.txt) | Every documentation page, in full, as one plain-text file | You want the whole corpus in context, offline, with nothing scraped | +| [llms-full.txt](https://docs.erigon.tech/llms-full.txt) | The text of every documentation page, as one plain-text file | You want the whole corpus in context, offline, with nothing scraped | Both are generated from the same Markdown source as this site by a script in the repository, and CI fails when a documentation change lands without regenerating them, so they cannot quietly go stale. +The corpus is cleaned rather than copied verbatim: MDX components are stripped, and section index pages built as card grids are reduced to the list of links they present. Read it as the documentation's prose, not as a byte-for-byte mirror of the rendered site. + ```bash # Give a local model the whole documentation, with no network access curl -sO https://docs.erigon.tech/llms-full.txt ``` -The full file is around 420 KB. If your model's context cannot hold it, use `llms.txt` to pick the pages you need, or slice the sections you want out of the full file — each page begins with a `# Title` heading. +The full file is around 430 KB. If your model's context cannot hold it, use `llms.txt` to pick the pages you need, or slice the sections you want out of the full file — each page begins with a `# Title` heading. --- diff --git a/llms-full.txt b/llms-full.txt index b875b3d5a63..5cd8769c6d4 100644 --- a/llms-full.txt +++ b/llms-full.txt @@ -3930,16 +3930,18 @@ MCP connects a model to your **running node**. To let a model answer questions a | File | Contents | Use it when | |------|----------|-------------| | [llms.txt](https://docs.erigon.tech/llms.txt) | Page index with titles and descriptions | The model fetches pages on demand, or its context window is small | -| [llms-full.txt](https://docs.erigon.tech/llms-full.txt) | Every documentation page, in full, as one plain-text file | You want the whole corpus in context, offline, with nothing scraped | +| [llms-full.txt](https://docs.erigon.tech/llms-full.txt) | The text of every documentation page, as one plain-text file | You want the whole corpus in context, offline, with nothing scraped | Both are generated from the same Markdown source as this site by a script in the repository, and CI fails when a documentation change lands without regenerating them, so they cannot quietly go stale. +The corpus is cleaned rather than copied verbatim: MDX components are stripped, and section index pages built as card grids are reduced to the list of links they present. Read it as the documentation's prose, not as a byte-for-byte mirror of the rendered site. + ```bash # Give a local model the whole documentation, with no network access curl -sO https://docs.erigon.tech/llms-full.txt ``` -The full file is around 420 KB. If your model's context cannot hold it, use `llms.txt` to pick the pages you need, or slice the sections you want out of the full file — each page begins with a `# Title` heading. +The full file is around 430 KB. If your model's context cannot hold it, use `llms.txt` to pick the pages you need, or slice the sections you want out of the full file — each page begins with a `# Title` heading. --- From 005311be212dcc19e915e3b7aa50b481491cacd7 Mon Sep 17 00:00:00 2001 From: Bloxster Date: Mon, 17 Aug 2026 16:07:20 +0200 Subject: [PATCH 2/2] =?UTF-8?q?docs(site):=20second=20review=20round=20?= =?UTF-8?q?=E2=80=94=20independent=20card=20count,=20index=20route,=20scop?= =?UTF-8?q?e?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Follow-up review from @yperbasis on #23336. Three findings, all confirmed by reproduction before fixing. 1. The guard counted the marker it was validating. `expected` came from `lp-card-title`, so if a card lost or renamed that marker the count shrank in step with the loss it was meant to detect: verified that a two-card grid with one renamed marker emitted one bullet and raised nothing. If every marker changed, `expected` hit zero and the page fell back to strip_mdx. Count `lp-card` containers instead — the wrapper is not consumed by the parse, so the two signals stay independent. Attributes are now matched order-independently and `to=` is read separately, which the container match no longer pins down. 2. llms.txt had no route to llms-full.txt. Neither committed index contained the string at all, so once the llms-full head tag was removed, an agent following rel="describedby" reached an index with no way to find the full corpus. The generator now emits that link, and both copies are regenerated. 3. "Every documentation page" was still wrong. SECTIONS scans only docs/ and help-center/, while the site also publishes v3.3 and v3.4 from versioned_docs/ — neither artifact contains those URLs. Say current documentation, and state the exclusion outright. Two new tests: a renamed title marker must raise rather than be absorbed (verified to fail against the previous count), and card attributes must parse in either order. --- docs/site/docs/fundamentals/mcp.mdx | 6 +-- .../docs/get-started/why-using-erigon.mdx | 2 +- docs/site/scripts/generate-llms.py | 34 ++++++++++++----- docs/site/scripts/test_generate_llms.py | 37 +++++++++++++++++++ docs/site/static/llms-full.txt | 6 +-- docs/site/static/llms.txt | 2 + llms-full.txt | 6 +-- llms.txt | 2 + 8 files changed, 75 insertions(+), 20 deletions(-) diff --git a/docs/site/docs/fundamentals/mcp.mdx b/docs/site/docs/fundamentals/mcp.mdx index 8ddd30d6769..198cbe5e396 100644 --- a/docs/site/docs/fundamentals/mcp.mdx +++ b/docs/site/docs/fundamentals/mcp.mdx @@ -316,12 +316,12 @@ MCP connects a model to your **running node**. To let a model answer questions a | File | Contents | Use it when | |------|----------|-------------| -| [llms.txt](https://docs.erigon.tech/llms.txt) | Page index with titles and descriptions | The model fetches pages on demand, or its context window is small | -| [llms-full.txt](https://docs.erigon.tech/llms-full.txt) | The text of every documentation page, as one plain-text file | You want the whole corpus in context, offline, with nothing scraped | +| [llms.txt](https://docs.erigon.tech/llms.txt) | Index of the current documentation, with titles and descriptions | The model fetches pages on demand, or its context window is small | +| [llms-full.txt](https://docs.erigon.tech/llms-full.txt) | The text of those same pages, as one plain-text file | You want the whole corpus in context, offline, with nothing scraped | Both are generated from the same Markdown source as this site by a script in the repository, and CI fails when a documentation change lands without regenerating them, so they cannot quietly go stale. -The corpus is cleaned rather than copied verbatim: MDX components are stripped, and section index pages built as card grids are reduced to the list of links they present. Read it as the documentation's prose, not as a byte-for-byte mirror of the rendered site. +Two things to know about their scope. They cover the **current** documentation and the help center only — the archived versions published under `/v3.3/` and `/v3.4/` are not included. And the text is cleaned rather than copied verbatim: MDX components are stripped, and section index pages built as card grids are reduced to the list of links they present. Read the corpus as the documentation's prose, not as a byte-for-byte mirror of the rendered site. ```bash # Give a local model the whole documentation, with no network access diff --git a/docs/site/docs/get-started/why-using-erigon.mdx b/docs/site/docs/get-started/why-using-erigon.mdx index 66b91ca6c14..e2551825c83 100644 --- a/docs/site/docs/get-started/why-using-erigon.mdx +++ b/docs/site/docs/get-started/why-using-erigon.mdx @@ -180,4 +180,4 @@ The interface is **read-only** and localhost-bound — no risk of unintended sta claude mcp add --transport sse erigon http://127.0.0.1:8553/sse ``` -Where MCP lets a model query your *node*, [llms.txt and llms-full.txt](../fundamentals/mcp#these-docs-as-a-single-file) let it read these *docs*: an index of every page, and the documentation text as one plain-text file for offline use. +Where MCP lets a model query your *node*, [llms.txt and llms-full.txt](../fundamentals/mcp#these-docs-as-a-single-file) let it read these *docs*: an index of the current documentation, and the same pages' text as one plain-text file for offline use. diff --git a/docs/site/scripts/generate-llms.py b/docs/site/scripts/generate-llms.py index cc028272ea8..d2e6b970059 100644 --- a/docs/site/scripts/generate-llms.py +++ b/docs/site/scripts/generate-llms.py @@ -291,10 +291,15 @@ def strip_mdx(text): # ``) fails to match locally, and the engine then scans forward and pairs # the title with the *next* card's description — silently swallowing the card in # between. Matching each block first makes a card boundary unrepresentable. -_LANDING_LINK_RE = re.compile( - r']*\bto=[\'"]([^\'"]+)[\'"][^>]*>(.*?)', - re.DOTALL, -) +# +# The container match is also what the guard below counts. It deliberately keys +# on `lp-card` — the wrapper — and not on any marker the parse itself consumes: +# a count taken from `lp-card-title` would shrink in step with the very loss it +# is supposed to detect, so a renamed title marker would drop its card in +# silence. Attributes are matched order-independently; `to=` is read separately. +_LANDING_CARD_LINK_RE = re.compile( + r']*\blp-card\b[^>]*)>(.*?)', re.DOTALL) +_LINK_TO_RE = re.compile(r'\bto=[\'"]([^\'"]+)[\'"]') _CARD_TITLE_RE = re.compile( r']*\blp-card-title\b[^>]*>(.*?)', re.DOTALL) _CARD_DESC_RE = re.compile( @@ -326,18 +331,19 @@ def synthesize_landing(raw_body): # ordinary prose page: the caller would fall back to strip_mdx and the guard # below would never run — silently degrading exactly the case it exists to # catch. - expected = raw_body.count('lp-card-title') + containers = _LANDING_CARD_LINK_RE.findall(raw_body) + expected = len(containers) if not expected: return None cards = [] - for link in _LANDING_LINK_RE.finditer(raw_body): - href, block = link.group(1), link.group(2) + for attrs, block in containers: + href = _LINK_TO_RE.search(attrs) title = _CARD_TITLE_RE.search(block) desc = _CARD_DESC_RE.search(block) - if not title or not desc: + if not href or not title or not desc: continue - cards.append((href, _card_text(title.group(1)), + cards.append((href.group(1), _card_text(title.group(1)), _card_text(desc.group(1)))) # A card that parses to nothing is invisible in the output, and `--check` @@ -346,7 +352,7 @@ def synthesize_landing(raw_body): if len(cards) != expected: raise SystemExit( f"synthesize_landing dropped cards: parsed {len(cards)} " - f"of {expected} lp-card-title occurrences" + f"of {expected} lp-card containers" ) out = [] @@ -555,6 +561,14 @@ def display_section(p): lines.append("> modularity, and minimal disk footprint. It features an integrated consensus layer") lines.append("> (Caplin), BitTorrent-based historical data distribution, and fast node synchronization.") lines.append("") + # The only machine-readable route from this index to the full corpus. Pages + # advertise llms.txt via , not llms-full.txt, so an + # agent that arrives here would otherwise have no way to reach it. + lines.append( + f"The full text of every page listed below is also available as a single " + f"file: [llms-full.txt]({BASE_URL}/llms-full.txt)" + ) + lines.append("") current_section = None for p in all_pages: diff --git a/docs/site/scripts/test_generate_llms.py b/docs/site/scripts/test_generate_llms.py index db31798a339..f49ca8b8a35 100644 --- a/docs/site/scripts/test_generate_llms.py +++ b/docs/site/scripts/test_generate_llms.py @@ -221,6 +221,43 @@ def test_dropped_card_raises_instead_of_silently_shrinking(self): g.synthesize_landing(body) self.assertIn("dropped cards", str(ctx.exception)) + def test_renamed_title_marker_is_caught_not_absorbed(self): + """The guard must not count the marker it validates. + + Counting `lp-card-title` would shrink in step with the very loss it is + meant to detect, so a renamed marker would drop its card in silence. + Counting `lp-card` containers keeps the two signals independent. + """ + body = """ +
+ +
A
+
Desc A.
+ + +
B
+
Desc B.
+ +
+""" + with self.assertRaises(SystemExit) as ctx: + g.synthesize_landing(body) + self.assertIn("parsed 1 of 2", str(ctx.exception)) + + def test_card_attribute_order_does_not_matter(self): + """`to=` may precede or follow the class; both must parse.""" + body = """ +
+ +
A
+
Desc A.
+ +
+""" + out = g.synthesize_landing(body) + self.assertIsNotNone(out) + self.assertIn("[A](https://docs.erigon.tech/a): Desc A.", out) + def test_all_cards_malformed_raises_rather_than_falling_back(self): """Total parser failure must not look like an ordinary prose page. diff --git a/docs/site/static/llms-full.txt b/docs/site/static/llms-full.txt index 5cd8769c6d4..442bfefb595 100644 --- a/docs/site/static/llms-full.txt +++ b/docs/site/static/llms-full.txt @@ -3929,12 +3929,12 @@ MCP connects a model to your **running node**. To let a model answer questions a | File | Contents | Use it when | |------|----------|-------------| -| [llms.txt](https://docs.erigon.tech/llms.txt) | Page index with titles and descriptions | The model fetches pages on demand, or its context window is small | -| [llms-full.txt](https://docs.erigon.tech/llms-full.txt) | The text of every documentation page, as one plain-text file | You want the whole corpus in context, offline, with nothing scraped | +| [llms.txt](https://docs.erigon.tech/llms.txt) | Index of the current documentation, with titles and descriptions | The model fetches pages on demand, or its context window is small | +| [llms-full.txt](https://docs.erigon.tech/llms-full.txt) | The text of those same pages, as one plain-text file | You want the whole corpus in context, offline, with nothing scraped | Both are generated from the same Markdown source as this site by a script in the repository, and CI fails when a documentation change lands without regenerating them, so they cannot quietly go stale. -The corpus is cleaned rather than copied verbatim: MDX components are stripped, and section index pages built as card grids are reduced to the list of links they present. Read it as the documentation's prose, not as a byte-for-byte mirror of the rendered site. +Two things to know about their scope. They cover the **current** documentation and the help center only — the archived versions published under `/v3.3/` and `/v3.4/` are not included. And the text is cleaned rather than copied verbatim: MDX components are stripped, and section index pages built as card grids are reduced to the list of links they present. Read the corpus as the documentation's prose, not as a byte-for-byte mirror of the rendered site. ```bash # Give a local model the whole documentation, with no network access diff --git a/docs/site/static/llms.txt b/docs/site/static/llms.txt index d7b26ab9446..6e5927f3913 100644 --- a/docs/site/static/llms.txt +++ b/docs/site/static/llms.txt @@ -4,6 +4,8 @@ > modularity, and minimal disk footprint. It features an integrated consensus layer > (Caplin), BitTorrent-based historical data distribution, and fast node synchronization. +The full text of every page listed below is also available as a single file: [llms-full.txt](https://docs.erigon.tech/llms-full.txt) + - [Introduction](https://docs.erigon.tech/): Official documentation for Erigon — the efficient, modular Ethereum execution client. Get started, explore the JSON-RPC API, and learn how to run a full or archive node. ## Get Started diff --git a/llms-full.txt b/llms-full.txt index 5cd8769c6d4..442bfefb595 100644 --- a/llms-full.txt +++ b/llms-full.txt @@ -3929,12 +3929,12 @@ MCP connects a model to your **running node**. To let a model answer questions a | File | Contents | Use it when | |------|----------|-------------| -| [llms.txt](https://docs.erigon.tech/llms.txt) | Page index with titles and descriptions | The model fetches pages on demand, or its context window is small | -| [llms-full.txt](https://docs.erigon.tech/llms-full.txt) | The text of every documentation page, as one plain-text file | You want the whole corpus in context, offline, with nothing scraped | +| [llms.txt](https://docs.erigon.tech/llms.txt) | Index of the current documentation, with titles and descriptions | The model fetches pages on demand, or its context window is small | +| [llms-full.txt](https://docs.erigon.tech/llms-full.txt) | The text of those same pages, as one plain-text file | You want the whole corpus in context, offline, with nothing scraped | Both are generated from the same Markdown source as this site by a script in the repository, and CI fails when a documentation change lands without regenerating them, so they cannot quietly go stale. -The corpus is cleaned rather than copied verbatim: MDX components are stripped, and section index pages built as card grids are reduced to the list of links they present. Read it as the documentation's prose, not as a byte-for-byte mirror of the rendered site. +Two things to know about their scope. They cover the **current** documentation and the help center only — the archived versions published under `/v3.3/` and `/v3.4/` are not included. And the text is cleaned rather than copied verbatim: MDX components are stripped, and section index pages built as card grids are reduced to the list of links they present. Read the corpus as the documentation's prose, not as a byte-for-byte mirror of the rendered site. ```bash # Give a local model the whole documentation, with no network access diff --git a/llms.txt b/llms.txt index d7b26ab9446..6e5927f3913 100644 --- a/llms.txt +++ b/llms.txt @@ -4,6 +4,8 @@ > modularity, and minimal disk footprint. It features an integrated consensus layer > (Caplin), BitTorrent-based historical data distribution, and fast node synchronization. +The full text of every page listed below is also available as a single file: [llms-full.txt](https://docs.erigon.tech/llms-full.txt) + - [Introduction](https://docs.erigon.tech/): Official documentation for Erigon — the efficient, modular Ethereum execution client. Get started, explore the JSON-RPC API, and learn how to run a full or archive node. ## Get Started