Skip to content

Group release notes and tutorials properly in llms.txt - #1929

Open
jborlase-snowplow wants to merge 3 commits into
mainfrom
docs/llms-txt-release-notes-tutorials
Open

Group release notes and tutorials properly in llms.txt#1929
jborlase-snowplow wants to merge 3 commits into
mainfrom
docs/llms-txt-release-notes-tutorials

Conversation

@jborlase-snowplow

Copy link
Copy Markdown
Contributor

This PR was generated automatically by Claude.

What changed?

Release notes collapse from 96 individual ## headings into one dated section, newest first:

## Release notes

Changes to Snowplow components, newest first. Each note describes one release and is superseded by later ones for the same component.

- 2026-08-25 — [Snowplow iOS tracker 6.2.6](…) (Trackers): Events tracked when an iOS app…
- 2026-08-18 — [New in Signals: ML training datasets](…) (Signals, AI tools): The Signals Python SDK…

The date and components also go into each per-page .md and into llms-full.txt.

Tutorials use progressive disclosure — llms.txt lists one entry per tutorial instead of all 115 steps:

- [Build a personalized travel agent with Signals](https://docs.snowplow.io/tutorials/signals-personalize-travel.md): Set up a demo travel site, and integrate with Snowplow Signals to personalize content and chatbot responses based on user behavior. (Solution accelerator · use case: Customer-facing AI agents · uses: Signals, Local, Jupyter notebook, OpenAI, AI assistants · 8 steps)

Each links to a new combined tutorials/<slug>.md containing every step in order (~27KB average). Individual step .md files are unchanged and still addressable.

A new content-metadata.js module reads the metadata Docusaurus doesn't carry into built HTML — date and components from release-note frontmatter, position from tutorial steps, and each tutorials/*/meta.json. gray-matter is now declared in package.json; it was already in the tree via Docusaurus.

before after
sections 161 66
entries 1239 1144
llms.txt 323KB 296KB

Why?

Both content types fell through grouping logic that assumes a /docs/<section>/<subsection>/ route shape. Release notes live one per directory, so each became its own section heading with a mangled title ("Snowplow dbt unified v0 5 1 released") and no date — a model had no way to tell a 2023 note from last week's. Tutorials were the one special case in the grouping code, so all 115 steps landed in a single flat list ordered by filesystem walk: abandoned-browse-ccdp read braze-campaign, conclusion, data-modeling, introduction, …, with the introduction fourth.

Reviewer guidance

  • All 1027 docs entries are byte-identical before and after, and docs headings are unchanged apart from the removed release-note ones. Worth confirming that's what you see in the diff of a built llms.txt.
  • The combined tutorial files break the documented "swap the trailing / for .md" convention, since /tutorials/<slug>/ has no route on the site. The second commit documents this on the Working with AI page.
  • Grouping is now three branches keyed on the content directory rather than declarative config, so a fourth contentDirs entry would need the same treatment.
  • Not addressed here, and worth separate discussion: old release notes are still listed in full rather than trimmed to a recent window, and are still included in llms-full.txt.

AI reviews

Claude will automatically review this PR against the docs style guide.

If you have questions or want it to look again at something specific, tag @claude in a comment.

jborlase-snowplow and others added 2 commits August 28, 2026 13:05
Release notes and tutorials both fell through grouping logic that assumed a
docs-shaped route tree, so each of the 96 release notes became its own section
heading and all 115 tutorial steps sat in one flat list ordered by filesystem
walk rather than step order.

Release notes now collapse into a single section, newest first, with the date
and components read back from source frontmatter. Those also appear in each
per-page .md and in llms-full.txt, so a superseded release note can't read as
current documentation.

Tutorials use progressive disclosure: llms.txt lists one entry per tutorial
rather than per step, linking to a new combined .md holding every step in
order. Individual steps stay addressable at their own URLs.

Sections drop from 161 to 66 and llms.txt from 323KB to 296KB, with all 1027
docs entries unchanged.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The combined tutorial files don't follow the documented "swap the trailing
slash for .md" pattern, because tutorials have no page of their own on the
site. Without this, there is no way to discover that they exist.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@wiz-2cefcfbd9e

wiz-2cefcfbd9e Bot commented Aug 28, 2026

Copy link
Copy Markdown

Wiz Scan Summary

Scanner Findings
Vulnerability Finding Vulnerabilities -
Data Finding Sensitive Data -
Secret Finding Secrets -
IaC Misconfiguration IaC Misconfigurations -
SAST Finding SAST Findings 5 Low
Software Management Finding Software Management Findings -
Total 5 Low

View scan details in Wiz

To detect these findings earlier in the dev lifecycle, try the Wiz Code extension for VS Code, JetBrains, or Visual Studio.

@cloudflare-workers-and-pages

cloudflare-workers-and-pages Bot commented Aug 28, 2026

Copy link
Copy Markdown

Deploying with  Cloudflare Workers  Cloudflare Workers

The latest updates on your project. Learn more about integrating Git with Workers.

Status Name Latest Commit Preview URL Updated (UTC)
✅ Deployment successful!
View logs
documentation ba255c5 Commit Preview URL

Branch Preview URL
Aug 28 2026, 12:57 PM

@claude

claude Bot commented Aug 28, 2026

Copy link
Copy Markdown

Style review — scope was prose, style guide compliance, frontmatter, links, and MDX usage. The only prose in this PR is the new ### Tutorials as single pages subsection in docs/llms-support/index.md; the rest is plugin code and lockfile changes, which are out of scope.

The new section is in good shape: sentence-case H3, preceded by prose, sits correctly under an existing H2, list introduced with a colon, no marketing language, present tense, active voice. Two small things.

1. "Tutorials have no page of their own on the site" is ambiguous (docs/llms-support/index.md:47)

Tutorials have no page of their own on the site, so these files don't follow the URL pattern above.

There is a /tutorials/ landing page (src/pages/tutorials.tsx), so as written this reads as slightly wrong. The intended meaning is that an individual tutorial has no page of its own — only its steps do. Style guide, "Writing style": clarity over compression.

Suggested fix:

Each tutorial is published only as its individual steps, with no page of its own, so these files don't follow the URL pattern above.

2. Optional: the release-note metadata change isn't documented

The plugin now emits > Date: and > Components: headers for release notes in both the per-page .md files and llms-full.txt, and gives release notes their own dated, newest-first section in llms.txt. That's a user-visible change to the generated output, but the docs page only covers the tutorial change. Worth a sentence in the Documentation index in llms.txt`` section if you want the page to stay a complete description of what the files contain — your call on whether it's detail a reader needs.

Checks that passed

  • Frontmatter unchanged; no new pages created, so the index.md directory rule doesn't apply.
  • https://docs.snowplow.io/... URLs on lines 49–50 are inline code examples demonstrating the URL transformation, not links, and match the existing pattern on lines 40–41 — not a violation of the "no docs.snowplow.io links" rule.
  • https://docs.snowplow.io/tutorials/signals-quickstart/start/ resolves, and tutorials/signals-quickstart/start.md exists in the repo, so the example is accurate.
  • No release note needed: release-notes/_README.md explicitly exempts "internal repo changes, such as tooling and CI", which is what the plugin work is.

@greptile-apps

greptile-apps Bot commented Aug 28, 2026

Copy link
Copy Markdown

Greptile Summary

The PR reorganizes generated LLM documentation so release notes form one dated section and each tutorial is represented by a combined, correctly ordered Markdown document.

  • Reads release-note and tutorial metadata from source files during the post-build phase.
  • Generates combined tutorial pages and links them from llms.txt.
  • Sorts release notes newest-first and adds dates and component metadata to generated output.
  • Documents the special URL convention for combined tutorial files.
  • Declares the existing gray-matter package as a direct development dependency.

Confidence Score: 5/5

The PR appears safe to merge, with no concrete current correctness or security failures identified.

The current tutorial routes, metadata shapes, generated paths, dependency resolution, and release-note source layout satisfy the assumptions introduced by the changed generation pipeline.

Important Files Changed

Filename Overview
plugins/docusaurus-plugin-llms-txt/src/content-metadata.js Adds source metadata loading, page classification, tutorial grouping, and deterministic ordering consistent with the repository’s current content shapes.
plugins/docusaurus-plugin-llms-txt/src/generate-index.js Replaces generic grouping with dedicated documentation, tutorial, and release-note sections and generates valid links for both Markdown-enabled and route-only configurations.
plugins/docusaurus-plugin-llms-txt/src/write-pages.js Generates one ordered Markdown document per tutorial while preserving individual page output and enriching release-note headers.
plugins/docusaurus-plugin-llms-txt/src/index.js Integrates metadata enrichment, sorting, and combined tutorial generation into the existing post-build pipeline.
docs/llms-support/index.md Documents the intentionally distinct URL convention used by combined tutorial Markdown files.
package.json Makes the already resolved gray-matter package a direct dependency of the build implementation.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
  HTML[Built documentation HTML] --> Convert[Convert pages to Markdown]
  Sources[Source frontmatter and tutorial meta.json] --> Metadata[Attach source metadata]
  Convert --> Metadata
  Metadata --> Sort[Sort tutorial steps and release notes]
  Sort --> Pages[Write per-page Markdown]
  Sort --> Tutorials[Write combined tutorial Markdown]
  Sort --> Index[Generate grouped llms.txt]
  Sort --> Full[Generate llms-full.txt]
  Tutorials --> Index
Loading

Reviews (1): Last reviewed commit: "Document combined tutorial Markdown file..." | Re-trigger Greptile

The site serves .txt and .md with no charset in the Content-Type, so clients
that fall back to Latin-1 render every multi-byte character as mojibake. The
middot and em-dash separators turned into "·" and "—" on all 116 tutorial
and release-note lines.

Semicolons and a plain space read the same and are immune. This does not fix
the underlying charset problem, which still affects content-derived characters
in llms.txt and, more seriously, 702 of the per-page .md files.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

@gleb-lobov Gleb Lobov (gleb-lobov) left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can't say anything about the code, but the change is needed.

Before, changelog is just appended to the Migration Guides
Image

After, it's added as it's own section, with dates
Image

Before, tutorials have all their steps listed:
Image

After, tutorials just list the top level items:

Image

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants