Add agent-readable 404s, markdown Accept, and llms.txt

[0thernet]

Official Is Agentic report before this change:

• URL: https://is-agentic.com/scan/stripedex.com
• Score: 75/100 (Ready with a few material gaps)
• Scanned: 2026-08-21T21:57:36.147Z
• Essential: 62.9/80 (5/7)
• Recommended: 10.6/20 (3/9)
• Bonus: +1.4

This PR does not claim a new live score. Reports cache, and only a post-deploy rescan can change the published number.

Findings from the completed report

Essential

1. markdown-negotiation-vary / Markdown content negotiation (acceptmarkdown.com) — failed

   Not acceptmarkdown.com compliant: Accept: text/markdown returned text/html; charset=utf-8; Vary header missing Accept

2. agent-friendly-404 / Agent-friendly 404s — partial

   Nonexistent paths return a real HTTP 404. For full credit, include a short markdown body (site map links, where to look next) so agents can recover.

Recommended, matching existing capabilities

3. agent-instruction / Agent instruction / when-to-use — failed

   No agent instruction file with when-to-use guidance found

4. trust-anchors / Trust anchor pages — partial

   Only About page verified - missing: Contact, Privacy

5. json-ld / JSON-LD structured data — partial

   JSON-LD Organization with name + description - add url and sameAs/logo/address for full score

Recommended leftovers (no product surface invented)

6. brand-search-accuracy / Brand name discoverability — failed

   "Stripedex" search returned 10 results but domain did not appear

7. page-token-budget / Page token budget — failed

   1 of 6 measured pages exceed ~25K tokens: https://stripedex.com/ (~27K tokens)

8. org-schema-completeness / Organization schema completeness — partial

   Organization schema found but missing: contactPoint, address

Changes

• Parse Accept per acceptmarkdown.com and negotiate in Next.js 16 proxy.ts. Markdown-preferring GETs and .md siblings receive Content-Type: text/markdown; charset=utf-8 and Vary: Accept. Browser Accept still gets HTML. Clients that reject both HTML and markdown get 406. Next.js RSC navigations and YAML/research files are left alone.
• Keep real HTTP 404s and add a recovery list (history, llms.txt, sitemap, about, data, contact, privacy) on both the HTML 404 and the markdown 404.
• Publish /llms.txt with an H1, summary, and a "When to use this" section that names the jobs Stripedex is for and the Stripe product/API/MCP jobs it is not.
• Add /contact and /privacy from existing GitHub, security-reporting, Hraness, and about-page analytics copy. No invented inbox, address, or API.
• Add a Stripedex Organization JSON-LD node with name, description, url, and sameAs. Do not invent address or contactPoint.
• Leave robots.txt and the HTML timeline behavior unchanged. Sitemap and analytics now include the two trust pages.

Verification

bun run check passed (typecheck, lint, 149 tests). Production build succeeded, including Proxy (Middleware).

Live production curls that confirmed the report, before this branch:

curl -sS -o /dev/null -w "%{http_code}" https://stripedex.com/some-path-that-does-not-exist
# 404

curl -sSI -H "Accept: text/markdown" https://stripedex.com/
# content-type: text/html; charset=utf-8
# vary: rsc, next-router-state-tree, next-router-prefetch, next-router-segment-prefetch

curl -sS -o /dev/null -w "%{http_code}" https://stripedex.com/llms.txt
# 404

Local next start curls against this branch:

curl -sS -o /dev/null -w "%{http_code}" http://127.0.0.1:3000/some-path-that-does-not-exist
# 404

curl -sSI -H "Accept: text/markdown" http://127.0.0.1:3000/some-path-that-does-not-exist
# HTTP/1.1 404 Not Found
# content-type: text/markdown; charset=utf-8
# Vary: Accept, Accept-Encoding

curl -sSI -H "Accept: text/markdown" http://127.0.0.1:3000/
# HTTP/1.1 200 OK
# content-type: text/markdown; charset=utf-8
# Vary: Accept, Accept-Encoding

curl -sSI http://127.0.0.1:3000/about.md
# content-type: text/markdown; charset=utf-8
# Vary: Accept, Accept-Encoding

curl -sS -o /dev/null -w "%{http_code}" -H "Accept: application/pdf" http://127.0.0.1:3000/about
# 406

curl -sS -o /dev/null -w "%{http_code} %{content_type}" http://127.0.0.1:3000/llms.txt
# 200 text/plain; charset=utf-8

curl -sS -o /dev/null -w "%{http_code} %{content_type}" http://127.0.0.1:3000/robots.txt
# 200 text/plain

curl -sS -o /dev/null -w "%{http_code} %{content_type}" http://127.0.0.1:3000/sitemap.xml
# 200 application/xml

curl -sS -o /dev/null -w "%{http_code}" http://127.0.0.1:3000/contact
# 200

curl -sS -o /dev/null -w "%{http_code}" http://127.0.0.1:3000/privacy
# 200

Browser Accept still returns HTML. YAML downloads stay text/yaml.

Local HTML responses still show Next's RSC Vary list instead of Accept. Markdown and 406 responses include Accept. That is enough for the failed essential check, which probed Accept: text/markdown and got HTML.

Leftovers that need a product decision

• Brand search: indexing and off-site mentions, not a page we can fabricate.
• Homepage token budget: the root page is the complete timeline by design. Splitting it would change the product.
• Organization address / contactPoint: there is no public postal address or dedicated inbox in this repository. GitHub issues and private vulnerability reporting remain the real channels.
• MCP, OAuth, OpenAPI, GraphQL, and commerce: this site does not run those surfaces, so this PR does not add them.

  

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
stripedex	Ready	Preview	Aug 21, 2026 10:09pm