From 1f37622c750a3cc13f5818f6d23432e1a407ba47 Mon Sep 17 00:00:00 2001 From: Rob Jacques <2531474+robbyrob42@users.noreply.github.com> Date: Fri, 4 Sep 2026 19:08:10 -0700 Subject: [PATCH 1/2] feat(forensics): declare content signals, and advertise llms.txt in a Link header Two gaps from the 2026-09-04 Cloudflare Radar agent-readiness scan, which put the property at Level 1 with Bot Access Control at 1/2 and Discoverability at 2/4. CONTENT SIGNALS. robots.txt now declares search=yes, ai-input=yes, ai-train=yes. All three are a decision rather than a default. This property exists to be found by counsel who increasingly ask an assistant before they ask a colleague, and each permission is a way these words reach that reader. Withholding training consent while asking to be cited would be taking one side of the same bargain. The declaration is repeated in all fourteen user-agent groups, and that repetition is load-bearing for the same reason the named crawler blocks are: a named group replaces the wildcard rather than adding to it. A Content-Signal written only under `User-agent: *` would be invisible to GPTBot, ClaudeBot, PerplexityBot and every other crawler named below it -- precisely the audience it addresses. Getting this half-right would have been worse than not doing it, because the file would have read as a declaration while making none. LINK HEADER. A new public/_headers advertises /llms.txt with RFC 8288 `rel="describedby"`, the registered relation for "the target describes this resource". An agent reading response headers finds the map without parsing HTML or guessing at a well-known path. Applied to /* rather than enumerated per route: an enumerated list silently misses the next route added, and the cost of the broader match is one short header on asset responses nobody reads. The Cloudflare adapter prepends its own /_astro/* cache rule to this file at build time, so both rules survive -- verified in dist/client/_headers. Not addressed, deliberately. Radar's Protocol Discovery is 0/9 -- API catalog, OAuth/OIDC, MCP server card, A2A agent card, agent skills, WebMCP, ARD. Every one of those describes an API or an agent that a site exposes. This is a brochure site for an expert-witness practice and exposes neither, so publishing any of them would be a fabrication. 0/9 is the correct score. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_01CpEXBMByKmpV1f9G7UUZhF --- sites/forensics/public/_headers | 17 +++++++++++++++++ sites/forensics/public/robots.txt | 30 ++++++++++++++++++++++++++++++ 2 files changed, 47 insertions(+) create mode 100644 sites/forensics/public/_headers diff --git a/sites/forensics/public/_headers b/sites/forensics/public/_headers new file mode 100644 index 0000000..a0bdcd2 --- /dev/null +++ b/sites/forensics/public/_headers @@ -0,0 +1,17 @@ +# Response headers for forensics.rootsystem.com. +# +# The Cloudflare adapter prepends its own immutable-cache rule for /_astro/* to +# this file at build time, so rules written here are additive rather than a +# replacement, and both rules match an /_astro asset. Check dist/client/_headers +# after a build if that ever changes. + +/* + # RFC 8288 discovery. `describedby` is the registered relation for "the target + # describes this resource", which is exactly what /llms.txt is: the property's + # own map of itself. An agent that reads response headers finds it without + # parsing HTML or guessing at a well-known path. + # + # Applied to every path rather than enumerated per route. Enumerating would + # mean a list that silently misses the next route added, and the cost of the + # broader match is one short header on asset responses that no one reads. + Link: ; rel="describedby"; type="text/plain" diff --git a/sites/forensics/public/robots.txt b/sites/forensics/public/robots.txt index ce01d5e..6037342 100644 --- a/sites/forensics/public/robots.txt +++ b/sites/forensics/public/robots.txt @@ -4,7 +4,24 @@ # IA spec, and the launch gate is ship-now-learn-from-live, so there is no # reason to hold the property out of search. If that ever changes, the change # is `Disallow: /` here rather than per-page noindex tags. +# +# CONTENT SIGNALS. Every group below carries the same declaration: +# +# search=yes index this and link to it in ordinary search results +# ai-input=yes retrieve this to ground an AI-generated answer, and cite it +# ai-train=yes use this to train a model +# +# All three are yes, which is a decision and not a default. This property exists +# to be found by counsel who are increasingly asking an assistant before they +# ask a colleague, and every one of those three permissions is a way the words +# on these pages reach that reader. Withholding training consent while asking +# to be cited would be trying to take one side of the same bargain. +# +# The declaration is repeated in every group for the reason given below: a +# named group replaces the wildcard rather than adding to it, so a signal +# declared only here would be invisible to precisely the crawlers it addresses. User-agent: * +Content-Signal: search=yes, ai-input=yes, ai-train=yes Allow: / # The answer-engine crawlers are named individually below, and every block says @@ -32,50 +49,63 @@ Allow: / # OpenAI. GPTBot trains, OAI-SearchBot indexes for ChatGPT search, and # ChatGPT-User fetches a page because a person asked for it in a conversation. User-agent: GPTBot +Content-Signal: search=yes, ai-input=yes, ai-train=yes Allow: / User-agent: OAI-SearchBot +Content-Signal: search=yes, ai-input=yes, ai-train=yes Allow: / User-agent: ChatGPT-User +Content-Signal: search=yes, ai-input=yes, ai-train=yes Allow: / # Anthropic. Same three-way split: training, search index, user-initiated fetch. User-agent: ClaudeBot +Content-Signal: search=yes, ai-input=yes, ai-train=yes Allow: / User-agent: Claude-SearchBot +Content-Signal: search=yes, ai-input=yes, ai-train=yes Allow: / User-agent: Claude-User +Content-Signal: search=yes, ai-input=yes, ai-train=yes Allow: / # Perplexity. User-agent: PerplexityBot +Content-Signal: search=yes, ai-input=yes, ai-train=yes Allow: / User-agent: Perplexity-User +Content-Signal: search=yes, ai-input=yes, ai-train=yes Allow: / # Google. Googlebot is ordinary search; Google-Extended is the separate opt-in # that governs Gemini and AI Overviews, and declining it would remove this # property from the AI surface it is being optimized for. User-agent: Googlebot +Content-Signal: search=yes, ai-input=yes, ai-train=yes Allow: / User-agent: Google-Extended +Content-Signal: search=yes, ai-input=yes, ai-train=yes Allow: / # Microsoft/Bing, which is also the retrieval layer behind Copilot. User-agent: Bingbot +Content-Signal: search=yes, ai-input=yes, ai-train=yes Allow: / # Apple. Applebot serves Siri and Spotlight; Applebot-Extended is the separate # grant covering Apple Intelligence. User-agent: Applebot +Content-Signal: search=yes, ai-input=yes, ai-train=yes Allow: / User-agent: Applebot-Extended +Content-Signal: search=yes, ai-input=yes, ai-train=yes Allow: / Sitemap: https://forensics.rootsystem.com/sitemap-index.xml From 46f2f561aefcbcda80cf5217f150a5a47a698eec Mon Sep 17 00:00:00 2001 From: Rob Jacques <2531474+robbyrob42@users.noreply.github.com> Date: Fri, 4 Sep 2026 19:11:18 -0700 Subject: [PATCH 2/2] =?UTF-8?q?docs:=20retire=20roadmap.md=20=E2=80=94=20t?= =?UTF-8?q?he=20roadmap=20lives=20in=20Linear?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The file was last touched 2026-08-08 and had drifted past usefulness: its status table still recorded that forensics.rootsystem.com does not resolve, which stopped being true when the route and DNS record landed on 2026-09-03. Everything else in it is either done or superseded. A stale roadmap is worse than no roadmap for the same reason a hand-kept dateModified is worse than none: it is a claim nobody verifies, and it is believed until someone acts on it. Planning now lives in Linear. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_01CpEXBMByKmpV1f9G7UUZhF --- docs/roadmap.md | 115 ------------------------------------------------ 1 file changed, 115 deletions(-) delete mode 100644 docs/roadmap.md diff --git a/docs/roadmap.md b/docs/roadmap.md deleted file mode 100644 index 9e8f961..0000000 --- a/docs/roadmap.md +++ /dev/null @@ -1,115 +0,0 @@ -# Roadmap - -Ordering set by Rob on 2026-08-06. Five tracks. Track 1 is specced; tracks 2–5 -carry their open questions here and get their own spec when they come up. - -Standing operational items are not roadmap items and are not listed below. - -Work goes on a short-lived branch off `primary` with a pull request. A pull -request runs `wrangler versions upload`, which publishes a Worker version that -takes no traffic; only a push to `primary` runs `wrangler deploy`. Merging is -therefore a production event. See `.github/workflows/deploy.yml`. - -## Where things stand - -*Source: `curl -I` against each hostname on 2026-08-06, plus the repository at -`61b1343`.* - -| Hostname | Status | Served by | -|---|---|---| -| `rootsystem.com` | 200 | Cloudflare Worker `rootsystem-www` | -| `www.rootsystem.com` | 301 to apex | Cloudflare ruleset | -| `rootsystem.com/taproot` | 307 to `/taproot/`, then 200 | same Worker | -| `insights.rootsystem.com` | 200 | Vercel | -| `forensics.rootsystem.com` | does not resolve | — | - -Netlify serves no traffic and is the rollback target; the apex A record still -points at it. `docs/apex-cutover-runbook.md` holds the executed record and -`docs/teardown-order.md` the decommission order. - -Taproot holds one post (`sites/www/src/content/taproot/why-taproot.mdx`). - -## Track 1 — spam gate on both forms - -**Deployed 2026-08-06.** Specced in -`docs/superpowers/specs/2026-08-06-form-spam-gate-design.md`. - -Both endpoints verify a Turnstile token server-side, record a spam verdict on -every stored row, and email the clean ones to `contact@rootsystem.com` via -Resend. Migration `0002` is applied to the remote D1; secrets are set on both -Workers; both Workers are deployed. - -Every path is verified end to end, including a real browser submission that -passed Turnstile and was accepted by Resend. See §9 of the spec. - -This track also touches a gate in `docs/teardown-order.md` §1, which holds the -Netlify deletion until at least one genuine inbound contact submission from a -stranger has landed. The form path is now proven working, so that gate is -waiting only on a real inbound rather than on the mechanism. - -## Track 2 — forensics case flow - -`forensics.rootsystem.com` does not resolve. The information architecture -question tracked as open in `docs/forensics-ia-open-problem.md` is settled, -per `docs/superpowers/specs/2026-08-08-forensics-ia-design.md`: a root page -plus three spoke routes, `/method`, `/matters`, and `/engagements`, each a -full-length treatment of a section the root only summarizes. The copy and the -visual direction were already settled (decision D11, commit `0ed72da`). - -That structure has shipped, including the forensics-specific OG card. -`robots.txt` deliberately stays open — nothing about the IA decision changed -the case for blocking crawlers. - -What is left in this track is the hostname: the DNS record and Worker route -for `forensics.rootsystem.com`. Both are deliberately unrouted until there is -something worth routing to; now that the site is built, that is the remaining -work. - -Expert-witness directory listings and the side brand are downstream of a live -hostname and are mostly not code. - -## Track 3 — legacy decommission, parallel and low effort - -Ordered by irreversibility in `docs/teardown-order.md`. Nothing in this track is -urgent and each step gates the next. - -Step 0 is the Sanity dataset export. The `insights` posts exist only in Sanity -and in the live Vercel deployment; Taproot has zero of them. The export needs an -interactive `npx sanity login`, which is why it has not run. - -Two items from the design spec §10 also sit here, both cheap: - -- `rob.` and `kyle.` are flat records inside the `rootsystem.com` zone at - DNSimple rather than child zones. The arrangement answers correctly today - because the DNSimple zone still exists; `scripts/dns-verify.sh` reports it as a - WARN for that reason. -- `rootsystem.com` is registered under a personal DNSimple account rather than - the Root System account. DNSimple supports pushing a domain between accounts. - -Open questions: - -- When to run the Sanity export (blocked on an interactive login). - -## Track 4 — SEO and content - -Taproot has one post. The per-post `insights` → `taproot` mapping is a content -decision, and it gates `docs/teardown-order.md` §2: a 301 from a set of indexed -article URLs onto an empty index is a soft-404 pattern. - -Also here: the trailing-slash 307 on internal navigation, and per-property OG -cards — the forensics property currently shares the main brand's card. - -Open questions: - -- Map `insights` posts individually as they are ported, or leave `insights` - serving until Taproot has landing targets. - -## Track 5 — PostHog - -Not started. Instrumentation across both properties, plus a consent posture. The -forensics property collects case intake from prospective litigants, which makes -the consent question a deliberate one rather than a default. - -Open questions: - -- Consent posture, and whether it differs between the two properties.