From 4aedc360cf434380e718d1d7eb89bcc9489372e0 Mon Sep 17 00:00:00 2001 From: Ali S Date: Tue, 25 Aug 2026 12:32:26 +0000 Subject: [PATCH 1/5] chore(repo): add ignore rules and editor conventions MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Precedes the tooling that needs them: node_modules/ and *.tsbuildinfo for the test suite, .devcontainer/.env and .task/ for the container and task runner. The .env entry matters most — the template beside it is committed, the filled-in copy never is. Co-Authored-By: Claude Opus 5 (1M context) --- .editorconfig | 28 ++++++++++++++++++++++++++++ .gitignore | 18 ++++++++++++++++++ 2 files changed, 46 insertions(+) create mode 100644 .editorconfig create mode 100644 .gitignore diff --git a/.editorconfig b/.editorconfig new file mode 100644 index 0000000..99244b5 --- /dev/null +++ b/.editorconfig @@ -0,0 +1,28 @@ +# Editor behaviour, for every editor. +# +# This is the single source for whitespace, line endings and encoding. The +# devcontainer deliberately does NOT restate these in +# customizations.vscode.settings: two statements of the same rule drift, and +# these two already had — VS Code trimmed trailing whitespace from Markdown +# while the rule below preserves it. +# +# It stays at the repo root because EditorConfig walks up from the file being +# edited and accepts no config-path flag. +root = true + +[*] +charset = utf-8 +end_of_line = lf +insert_final_newline = true +trim_trailing_whitespace = true +indent_style = space +indent_size = 2 + +# Two trailing spaces are a hard line break in Markdown, so trimming them +# silently changes how the prose renders. +[*.md] +trim_trailing_whitespace = false + +# Tabs are syntax in a Makefile, not indentation. +[Makefile] +indent_style = tab diff --git a/.gitignore b/.gitignore new file mode 100644 index 0000000..b89dbde --- /dev/null +++ b/.gitignore @@ -0,0 +1,18 @@ +# Tooling +node_modules/ +*.tsbuildinfo + +# Dev container local state +.devcontainer/.env + +# Personal lefthook overrides, auto-merged beside .config/lefthook.yml. +/.config/lefthook-local.yml + +# Task runner cache +.task/ + +# Editor / OS +.DS_Store +Thumbs.db +.idea/ +*.swp From b4bf1ceb8e403d1751733385417ded46787f906e Mon Sep 17 00:00:00 2001 From: Ali S Date: Tue, 25 Aug 2026 12:32:47 +0000 Subject: [PATCH 2/5] feat(tests): hold the corpus to the published spec contracts MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The seed commit left this repository with no validation of its own, on the grounds that the platform is the sole validator. It still is — but "the platform will reject it" is a slow way to learn that an item is malformed, and nothing here caught it at commit time. These tests apply the same contracts earlier. They are not a second authority: the schemas are fetched at run time from the tip of the public musher-dev/spec repository rather than vendored, so the corpus is judged against the contract as it currently stands. A copy held here would drift, and a stale copy is worse than none — it passes a corpus the live spec would reject, silently. The repository is public, so no credential is read. Covers the three phases component spec §7 lets a client decide offline: parser the Musher YAML profile — one document per file, string keys, no anchors, aliases, merge keys or tags, and the bounds structural each document against its family's fetched JSON Schema semantic the cross-document rules — identity agreement, reference resolution, path containment, media, the description Markdown profile, image pinning, endpoint resolution, connection compatibility, parameter coverage `capability` is deliberately absent. Whether a Compute Profile is offered, whether a published component exists and whether a version is monotonic are decided against the platform catalog over the network, and an implementation MUST NOT report a rule it cannot check. An item green here can still be rejected at sync. rules.test.ts guards the guards: a suite that passes on a clean corpus passes just as readily when a rule is unreachable, so each case breaks one thing in a synthetic item and asserts the normative diagnostic fires. Nothing is restated from the spec where it can be read from it — the media path grammar and the component value-schema defaults are pulled out of the fetched bundles at run time, so they cannot drift from their definition. Run by Node's type stripping and built-in test runner: no build step, no framework. 377 tests across the 13 items. Co-Authored-By: Claude Opus 5 (1M context) --- package-lock.json | 184 +++++++++ package.json | 21 + tests/README.md | 100 +++++ tests/layout.test.ts | 129 ++++++ tests/lib/catalog.ts | 173 ++++++++ tests/lib/markdown.ts | 95 +++++ tests/lib/media.ts | 53 +++ tests/lib/paths.ts | 14 + tests/lib/semantic.ts | 801 ++++++++++++++++++++++++++++++++++++++ tests/lib/spec-schemas.ts | 188 +++++++++ tests/lib/yaml-profile.ts | 176 +++++++++ tests/parser.test.ts | 99 +++++ tests/rules.test.ts | 679 ++++++++++++++++++++++++++++++++ tests/semantic.test.ts | 148 +++++++ tests/spec.test.ts | 51 +++ tests/structural.test.ts | 63 +++ tsconfig.json | 25 ++ 17 files changed, 2999 insertions(+) create mode 100644 package-lock.json create mode 100644 package.json create mode 100644 tests/README.md create mode 100644 tests/layout.test.ts create mode 100644 tests/lib/catalog.ts create mode 100644 tests/lib/markdown.ts create mode 100644 tests/lib/media.ts create mode 100644 tests/lib/paths.ts create mode 100644 tests/lib/semantic.ts create mode 100644 tests/lib/spec-schemas.ts create mode 100644 tests/lib/yaml-profile.ts create mode 100644 tests/parser.test.ts create mode 100644 tests/rules.test.ts create mode 100644 tests/semantic.test.ts create mode 100644 tests/spec.test.ts create mode 100644 tests/structural.test.ts create mode 100644 tsconfig.json diff --git a/package-lock.json b/package-lock.json new file mode 100644 index 0000000..217dcf5 --- /dev/null +++ b/package-lock.json @@ -0,0 +1,184 @@ +{ + "name": "@musher/catalog", + "version": "0.0.0", + "lockfileVersion": 3, + "requires": true, + "packages": { + "": { + "name": "@musher/catalog", + "version": "0.0.0", + "license": "Apache-2.0", + "devDependencies": { + "@types/commonmark": "^0.27.9", + "@types/node": "^24.0.0", + "ajv": "^8.17.1", + "commonmark": "^0.31.2", + "typescript": "^5.8.0", + "yaml": "^2.8.1" + }, + "engines": { + "node": ">=23.6" + } + }, + "node_modules/@types/commonmark": { + "version": "0.27.10", + "resolved": "https://registry.npmjs.org/@types/commonmark/-/commonmark-0.27.10.tgz", + "integrity": "sha512-iEZobUnvlM+UX5fXWCmC4eQXwCs01Z8Xa1W0VjiWUF/XsNy4BHtskqJ9MyLZVMHbA0ezhyonCDqz3hMvsCm6Hg==", + "dev": true, + "license": "MIT" + }, + "node_modules/@types/node": { + "version": "24.13.3", + "resolved": "https://registry.npmjs.org/@types/node/-/node-24.13.3.tgz", + "integrity": "sha512-Dh8vAsV36ig5wa9OX4pXvMc9D3Veibfw2wix0CUwYODLD8nkj9UsLjASr49nPg+2eKzxhBV+v7L8pXvT4e639Q==", + "dev": true, + "license": "MIT", + "dependencies": { + "undici-types": "~7.18.0" + } + }, + "node_modules/ajv": { + "version": "8.20.0", + "resolved": "https://registry.npmjs.org/ajv/-/ajv-8.20.0.tgz", + "integrity": "sha512-Thbli+OlOj+iMPYFBVBfJ3OmCAnaSyNn4M1vz9T6Gka5Jt9ba/HIR56joy65tY6kx/FCF5VXNB819Y7/GUrBGA==", + "dev": true, + "license": "MIT", + "dependencies": { + "fast-deep-equal": "^3.1.3", + "fast-uri": "^3.0.1", + "json-schema-traverse": "^1.0.0", + "require-from-string": "^2.0.2" + }, + "funding": { + "type": "github", + "url": "https://github.com/sponsors/epoberezkin" + } + }, + "node_modules/commonmark": { + "version": "0.31.2", + "resolved": "https://registry.npmjs.org/commonmark/-/commonmark-0.31.2.tgz", + "integrity": "sha512-2fRLTyb9r/2835k5cwcAwOj0DEc44FARnMp5veGsJ+mEAZdi52sNopLu07ZyElQUz058H43whzlERDIaaSw4rg==", + "dev": true, + "license": "BSD-2-Clause", + "dependencies": { + "entities": "~3.0.1", + "mdurl": "~1.0.1", + "minimist": "~1.2.8" + }, + "bin": { + "commonmark": "bin/commonmark" + }, + "engines": { + "node": "*" + } + }, + "node_modules/entities": { + "version": "3.0.1", + "resolved": "https://registry.npmjs.org/entities/-/entities-3.0.1.tgz", + "integrity": "sha512-WiyBqoomrwMdFG1e0kqvASYfnlb0lp8M5o5Fw2OFq1hNZxxcNk8Ik0Xm7LxzBhuidnZB/UtBqVCgUz3kBOP51Q==", + "dev": true, + "license": "BSD-2-Clause", + "engines": { + "node": ">=0.12" + }, + "funding": { + "url": "https://github.com/fb55/entities?sponsor=1" + } + }, + "node_modules/fast-deep-equal": { + "version": "3.1.3", + "resolved": "https://registry.npmjs.org/fast-deep-equal/-/fast-deep-equal-3.1.3.tgz", + "integrity": "sha512-f3qQ9oQy9j2AhBe/H9VC91wLmKBCCU/gDOnKNAYG5hswO7BLKj09Hc5HYNz9cGI++xlpDCIgDaitVs03ATR84Q==", + "dev": true, + "license": "MIT" + }, + "node_modules/fast-uri": { + "version": "3.1.6", + "resolved": "https://registry.npmjs.org/fast-uri/-/fast-uri-3.1.6.tgz", + "integrity": "sha512-7Ical1vFEMr0onbVzEDIreM22I4khW+fzyQPwvAFWBp1iwdshSZRsL4jjRvPG9JP1uiqMHRto+YU6R2/CzDz5Q==", + "dev": true, + "funding": [ + { + "type": "github", + "url": "https://github.com/sponsors/fastify" + }, + { + "type": "opencollective", + "url": "https://opencollective.com/fastify" + } + ], + "license": "BSD-3-Clause" + }, + "node_modules/json-schema-traverse": { + "version": "1.0.0", + "resolved": "https://registry.npmjs.org/json-schema-traverse/-/json-schema-traverse-1.0.0.tgz", + "integrity": "sha512-NM8/P9n3XjXhIZn1lLhkFaACTOURQXjWhV4BA/RnOv8xvgqtqpAX9IO4mRQxSx1Rlo4tqzeqb0sOlruaOy3dug==", + "dev": true, + "license": "MIT" + }, + "node_modules/mdurl": { + "version": "1.0.1", + "resolved": "https://registry.npmjs.org/mdurl/-/mdurl-1.0.1.tgz", + "integrity": "sha512-/sKlQJCBYVY9Ers9hqzKou4H6V5UWc/M59TH2dvkt+84itfnq7uFOMLpOiOS4ujvHP4etln18fmIxA5R5fll0g==", + "dev": true, + "license": "MIT" + }, + "node_modules/minimist": { + "version": "1.2.8", + "resolved": "https://registry.npmjs.org/minimist/-/minimist-1.2.8.tgz", + "integrity": "sha512-2yyAR8qBkN3YuheJanUpWC5U3bb5osDywNB8RzDVlDwDHbocAJveqqj1u8+SVD7jkWT4yvsHCpWqqWqAxb0zCA==", + "dev": true, + "license": "MIT", + "funding": { + "url": "https://github.com/sponsors/ljharb" + } + }, + "node_modules/require-from-string": { + "version": "2.0.2", + "resolved": "https://registry.npmjs.org/require-from-string/-/require-from-string-2.0.2.tgz", + "integrity": "sha512-Xf0nWe6RseziFMu+Ap9biiUbmplq6S9/p+7w7YXP/JBHhrUDDUhwa+vANyubuqfZWTveU//DYVGsDG7RKL/vEw==", + "dev": true, + "license": "MIT", + "engines": { + "node": ">=0.10.0" + } + }, + "node_modules/typescript": { + "version": "5.9.3", + "resolved": "https://registry.npmjs.org/typescript/-/typescript-5.9.3.tgz", + "integrity": "sha512-jl1vZzPDinLr9eUt3J/t7V6FgNEw9QjvBPdysz9KfQDD41fQrC2Y4vKQdiaUpFT4bXlb1RHhLpp8wtm6M5TgSw==", + "dev": true, + "license": "Apache-2.0", + "bin": { + "tsc": "bin/tsc", + "tsserver": "bin/tsserver" + }, + "engines": { + "node": ">=14.17" + } + }, + "node_modules/undici-types": { + "version": "7.18.2", + "resolved": "https://registry.npmjs.org/undici-types/-/undici-types-7.18.2.tgz", + "integrity": "sha512-AsuCzffGHJybSaRrmr5eHr81mwJU3kjw6M+uprWvCXiNeN9SOGwQ3Jn8jb8m3Z6izVgknn1R0FTCEAP2QrLY/w==", + "dev": true, + "license": "MIT" + }, + "node_modules/yaml": { + "version": "2.9.0", + "resolved": "https://registry.npmjs.org/yaml/-/yaml-2.9.0.tgz", + "integrity": "sha512-2AvhNX3mb8zd6Zy7INTtSpl1F15HW6Wnqj0srWlkKLcpYl/gMIMJiyuGq2KeI2YFxUPjdlB+3Lc10seMLtL4cA==", + "dev": true, + "license": "ISC", + "bin": { + "yaml": "bin.mjs" + }, + "engines": { + "node": ">= 14.6" + }, + "funding": { + "url": "https://github.com/sponsors/eemeli" + } + } + } +} diff --git a/package.json b/package.json new file mode 100644 index 0000000..a901475 --- /dev/null +++ b/package.json @@ -0,0 +1,21 @@ +{ + "name": "@musher/catalog", + "version": "0.0.0", + "private": true, + "description": "Conformance tests for the platform-curated Musher catalog corpus.", + "license": "Apache-2.0", + "type": "module", + "engines": { "node": ">=23.6" }, + "scripts": { + "test": "node --disable-warning=ExperimentalWarning --test \"tests/**/*.test.ts\"", + "typecheck": "tsc --noEmit" + }, + "devDependencies": { + "@types/commonmark": "^0.27.9", + "@types/node": "^24.0.0", + "ajv": "^8.17.1", + "commonmark": "^0.31.2", + "typescript": "^5.8.0", + "yaml": "^2.8.1" + } +} diff --git a/tests/README.md b/tests/README.md new file mode 100644 index 0000000..ed9958f --- /dev/null +++ b/tests/README.md @@ -0,0 +1,100 @@ +# Catalog validation + +These tests hold every item under `items/` to the contracts published in +[`musher-dev/spec`](https://github.com/musher-dev/spec). + +The schemas are **fetched at run time** from the tip of the public +`musher-dev/spec` repository, never vendored. The catalog is not the authority on +what a valid item looks like — the spec is — and a copy held here would be a +second authority that drifts. A stale copy is worse than no copy at all, because +it passes a corpus the live spec would reject, silently. + +There is exactly one source, and it is constant: + +``` +https://raw.githubusercontent.com/musher-dev/spec/main/specifications//v1/schemas/dist/.schema.json +``` + +`musher-dev/spec` is **public**, so this is fetched with **no credential**. +Nothing in the suite reads a token and none should be configured: a token +attached to a public read is a secret handed to a host that never asked for one, +and it makes the suite pass on a machine that has it and fail on one that does +not. + +**Nothing about the source is configurable** — not the ref, not a mirror, not a +local checkout. Each of those would be a second answer to "what is the contract", +which is the thing this suite exists to not have. `main` is the tip of the +contract, and the tip is what this corpus is held to. If the fetch fails the +suite fails, loudly, naming the URL: a run that quietly validated against +something else would be reporting on a contract nobody published. + +```sh +npm install +npm test # validate the corpus +npm run typecheck # tsc --noEmit +``` + +Tests are TypeScript run directly by Node's type stripping and its built-in test +runner. There is no build step and no test framework to install. + +## What runs + +The files map to the four validation phases component spec §7 defines, which are +applied in order — a later-phase diagnostic is never reported before the earlier +phases pass. + +| File | Phase | What it checks | +|---|---|---| +| `spec.test.ts` | — | The bundles resolve, name their own family, and are self-contained. Fails first, so a corpus is never judged against a 404 page. | +| `parser.test.ts` | `parser` | Every document satisfies the Musher YAML profile (component §7.1): one document per file, string keys, no anchors, aliases, merge keys or explicit tags, and the size, depth and scalar bounds. | +| `structural.test.ts` | `structural` | Every document validates against its family's fetched JSON Schema. | +| `semantic.test.ts` | `semantic` | The cross-document rules: identity agreement, reference resolution, path containment, media, the description Markdown profile, image pinning, endpoint resolution, connection compatibility, and parameter coverage. | +| `layout.test.ts` | — | The item folder structure, and the catalog's own additions to it. | +| `rules.test.ts` | — | The rules themselves, against deliberately broken synthetic items. | + +`capability` is deliberately absent. Whether a Compute Profile is offered, +whether a published component exists, and whether a version is monotonic are all +decided against the platform catalog over the network, and an implementation +MUST NOT report a rule it has not been given the means to check. + +### Why `rules.test.ts` exists + +A suite that passes on a clean corpus proves nothing on its own — it passes just +as readily when a rule is silently unreachable. Each case there breaks one thing +in a synthetic item and asserts the normative diagnostic fires, so the checks +guarding `items/` are themselves guarded. + +They are not conformance fixtures. `musher-dev/spec` publishes those under +`conformance/`, and its corpus is the authority on what an implementation must +report; these cases pin the subset this repository enforces. + +## Where the rules come from + +Nothing is restated from the spec where it can be read from it instead. The +media-path grammar and the component value-schema defaults are pulled out of the +fetched bundles at run time rather than copied, so the two places the semantic +phase needs them cannot drift. + +What is written down here is what JSON Schema cannot express, and each rule +carries the clause it implements: `BP-ID-001`, `LIST-MEDIA-003`, `COMP-SRC-001` +and the rest. A rule whose spelling has to live in this repository — the +floating-tag blocklist, for instance, which is `semantic` precisely so it can +grow in a minor release — says so at the definition. + +## Configuration + +| Variable | Default | Purpose | +|---|---|---| +| `MUSHER_SPEC_TIMEOUT_MS` | `15000` | Per-request timeout when fetching a schema. | + +That is the whole of it. The timeout is the only knob because it is the only one +that changes how a schema is fetched rather than *which* schema is fetched. + +## Adding a rule + +Put it in the phase it belongs to. If JSON Schema can express it, it belongs in +`musher-dev/spec` and not here — opening a PR there is the fix, and this suite +picks it up on the next run with no change. If it needs a second document or the +filesystem, it is `semantic`: add it to `tests/lib/semantic.ts` with its +diagnostic code, wire it into `semantic.test.ts`, and add a case to +`rules.test.ts` proving it fires. diff --git a/tests/layout.test.ts b/tests/layout.test.ts new file mode 100644 index 0000000..d8fbf68 --- /dev/null +++ b/tests/layout.test.ts @@ -0,0 +1,129 @@ +/** + * Item layout — the folder structure every catalog item must follow. + * + * Two shapes are defined. Blueprint spec §3.1 anchors the item root on + * `blueprint.yaml`; listing spec §3.1 anchors it on `listing.yaml` for an item + * that holds no blueprint. Only three names in the tree are fixed — + * `blueprint.yaml`, `listing.yaml`, and `media/` — so this file tests those and + * the catalog's own additions on top of them, which the repository README states + * as hard requirements for a platform sync. + */ +import fs from 'node:fs'; +import path from 'node:path'; +import assert from 'node:assert/strict'; +import { describe, it } from 'node:test'; + +import { + BLUEPRINT_FILE, + COMPONENTS_DIR, + LISTING_FILE, + MEDIA_DIR, + SLUG_PATTERN, + discoverItems, +} from './lib/catalog.ts'; +import { ITEMS_DIR, rel } from './lib/paths.ts'; + +/** Extensions listing spec §5 permits for a media path. */ +const MEDIA_EXTENSIONS = new Set(['.png', '.jpg', '.jpeg', '.webp']); + +/** What a catalog item directory is allowed to hold at its top level. */ +const PERMITTED_ENTRIES = new Set([LISTING_FILE, BLUEPRINT_FILE, COMPONENTS_DIR, MEDIA_DIR]); + +const items = discoverItems(); + +describe('catalog layout', () => { + it('holds at least one item', () => { + assert.ok(items.length > 0, `no item directories found under ${rel(ITEMS_DIR)}`); + }); + + it('puts every item in its own directory directly under items/', () => { + const strays = fs + .readdirSync(ITEMS_DIR, { withFileTypes: true }) + .filter((entry) => !entry.isDirectory()) + .map((entry) => entry.name); + assert.deepEqual(strays, [], `items/ holds entries that are not item directories: ${strays.join(', ')}`); + }); + + it('derives slug uniqueness from the directory name', () => { + // The directory name *is* the slug, so uniqueness is structural. This asserts + // the property the corpus relies on rather than re-deriving it. + const slugs = items.map((item) => item.slug); + assert.equal(new Set(slugs).size, slugs.length); + }); +}); + +for (const item of items) { + describe(`items/${item.slug}`, () => { + it('is named by a valid slug', () => { + assert.match(item.slug, SLUG_PATTERN); + }); + + it(`holds ${LISTING_FILE}`, () => { + // The storefront wrapper is what makes a directory a catalog item, and it + // is the item root for an item holding no blueprint. + assert.ok(item.listingPath, `${rel(item.root)} holds no ${LISTING_FILE}`); + }); + + it(`holds ${BLUEPRINT_FILE}`, () => { + // A catalog rule rather than a spec one: the spec lets a COMPONENT-kind + // listing ship without a blueprint, and the platform deploys exactly one + // blueprint per listing — so this corpus authors a trivial single-node + // blueprint even there, per the repository README. + assert.ok(item.blueprintPath, `${rel(item.root)} holds no ${BLUEPRINT_FILE}`); + }); + + it('holds at least one component document', () => { + assert.ok(item.componentPaths.length > 0, `${rel(item.root)} holds no component document`); + }); + + it(`keeps its component documents under ${COMPONENTS_DIR}/`, () => { + // `components/` is a convention the spec does not impose — a flat sibling + // is equally valid — but this corpus keeps it so a reader can find the + // building blocks of any item in the same place. + const misplaced = item.componentPaths + .filter((absolute) => path.dirname(absolute) !== path.join(item.root, COMPONENTS_DIR)) + .map(rel); + assert.deepEqual(misplaced, []); + }); + + it('holds no unexpected top-level entries', () => { + const unexpected = item.entries + .map((entry) => entry.name) + .filter((name) => !PERMITTED_ENTRIES.has(name)) + .sort(); + assert.deepEqual( + unexpected, + [], + `${rel(item.root)} holds ${unexpected.join(', ')}; permitted: ${[...PERMITTED_ENTRIES].join(', ')}`, + ); + }); + + it('uses the .yaml spelling for the two fixed names', () => { + const wrongSpelling = ['listing.yml', 'blueprint.yml'].filter((name) => + fs.existsSync(path.join(item.root, name)), + ); + assert.deepEqual(wrongSpelling, [], 'the fixed names are blueprint.yaml and listing.yaml'); + }); + + it(`ships assets only from ${MEDIA_DIR}/`, () => { + // listing spec §5: one fixed directory means a reader can find every asset + // an item ships without first reading its listing, and a publisher can copy + // that directory without walking the document to work out what to take. + const wrongExtension = item.mediaPaths + .filter((absolute) => !MEDIA_EXTENSIONS.has(path.extname(absolute).toLowerCase())) + .map(rel); + assert.deepEqual( + wrongExtension, + [], + `media/ may hold only ${[...MEDIA_EXTENSIONS].join(', ')} files`, + ); + }); + + it('contains no symbolic links', () => { + // Defence in depth for the containment rules the semantic phase enforces. + // A link committed inside an item can point anywhere the process can read, + // and this corpus has no use for one, so the whole class is excluded here. + assert.deepEqual(item.symlinkPaths.map(rel), []); + }); + }); +} diff --git a/tests/lib/catalog.ts b/tests/lib/catalog.ts new file mode 100644 index 0000000..b9410a8 --- /dev/null +++ b/tests/lib/catalog.ts @@ -0,0 +1,173 @@ +/** + * Discovery of the catalog corpus and the item shape the spec defines. + * + * A **catalog item** is one directory holding one deployable thing + * (blueprint spec §3.1, listing spec §3.1). The directory holding `blueprint.yaml` + * — or, for an item that holds none, `listing.yaml` — is the **item root**, and + * it is what every containment and identity rule is measured against. + */ +import fs from 'node:fs'; +import path from 'node:path'; + +import { ITEMS_DIR, rel } from './paths.ts'; +import { parseDocument, type ParserDiagnostic } from './yaml-profile.ts'; + +/** Only two names in an item tree are fixed by the blueprint family. */ +export const LISTING_FILE = 'listing.yaml'; +export const BLUEPRINT_FILE = 'blueprint.yaml'; +/** `media/` is fixed too, but by the listing family (listing spec §5). */ +export const MEDIA_DIR = 'media'; +/** `components/` is a convention rather than a rule; a flat sibling is equally valid. */ +export const COMPONENTS_DIR = 'components'; + +/** metadata.slug grammar, shared with blueprint node names. */ +export const SLUG_PATTERN = /^[a-z][a-z0-9-]{0,61}[a-z0-9]$/; + +const isYamlFile = (name: string): boolean => name.endsWith('.yaml') || name.endsWith('.yml'); + +export type Item = { + slug: string; + /** The item root — absolute. */ + root: string; + listingPath: string | null; + blueprintPath: string | null; + /** + * Every YAML document under the root that is neither of the two fixed names. + * Component documents MAY sit anywhere under the root, so this is a walk rather + * than a listing of `components/` — which is what makes `ERR_UNREFERENCED_COMPONENT` + * able to see a stray document parked outside the conventional directory. + */ + componentPaths: string[]; + /** Every regular file under `media/`, whether or not the listing declares it. */ + mediaPaths: string[]; + /** Top-level entries of the item directory, for the layout rules. */ + entries: fs.Dirent[]; + /** Symlinks found anywhere in the item, which the containment rules turn on. */ + symlinkPaths: string[]; +}; + +/** Every file below `dir`, with symlinks reported rather than followed. */ +function walk(dir: string, onSymlink: (absolute: string) => void): string[] { + if (!fs.existsSync(dir)) return []; + const found: string[] = []; + for (const entry of fs.readdirSync(dir, { withFileTypes: true })) { + const absolute = path.join(dir, entry.name); + if (entry.isSymbolicLink()) { + onSymlink(absolute); + found.push(absolute); + } else if (entry.isDirectory()) { + found.push(...walk(absolute, onSymlink)); + } else if (entry.isFile()) { + found.push(absolute); + } + } + return found; +} + +export function discoverItems(itemsDir: string = ITEMS_DIR): Item[] { + if (!fs.existsSync(itemsDir)) return []; + + return fs + .readdirSync(itemsDir, { withFileTypes: true }) + .filter((entry) => entry.isDirectory()) + .map((entry) => entry.name) + .sort() + .map((slug) => readItem(path.join(itemsDir, slug))); +} + +/** The item model for one directory, whether or not it lives under `items/`. */ +export function readItem(root: string): Item { + const symlinkPaths: string[] = []; + const noteSymlink = (absolute: string) => symlinkPaths.push(absolute); + + const listingPath = path.join(root, LISTING_FILE); + const blueprintPath = path.join(root, BLUEPRINT_FILE); + const mediaRoot = path.join(root, MEDIA_DIR); + + const componentPaths = walk(root, noteSymlink) + .filter((absolute) => isYamlFile(absolute)) + .filter((absolute) => absolute !== listingPath && absolute !== blueprintPath) + .filter((absolute) => !absolute.startsWith(mediaRoot + path.sep)) + .sort(); + + return { + slug: path.basename(root), + root, + listingPath: fs.existsSync(listingPath) ? listingPath : null, + blueprintPath: fs.existsSync(blueprintPath) ? blueprintPath : null, + componentPaths, + mediaPaths: walk(mediaRoot, noteSymlink).sort(), + entries: fs.existsSync(root) ? fs.readdirSync(root, { withFileTypes: true }) : [], + symlinkPaths: [...new Set(symlinkPaths)].sort(), + }; +} + +export type LoadedDocument = { + path: string; + /** Repo-relative, for messages. */ + label: string; + value: Record | undefined; + parserDiagnostics: ParserDiagnostic[]; +}; + +export async function loadDocument(absolutePath: string): Promise { + const { value, diagnostics } = await parseDocument(absolutePath); + return { + path: absolutePath, + label: rel(absolutePath), + value: isRecord(value) ? value : undefined, + parserDiagnostics: diagnostics, + }; +} + +export type ItemDocuments = { + listing: LoadedDocument | null; + blueprint: LoadedDocument | null; + /** Keyed by absolute path. */ + components: Map; +}; + +export async function loadItemDocuments(item: Item): Promise { + const [listing, blueprint, components] = await Promise.all([ + item.listingPath ? loadDocument(item.listingPath) : Promise.resolve(null), + item.blueprintPath ? loadDocument(item.blueprintPath) : Promise.resolve(null), + Promise.all(item.componentPaths.map(loadDocument)), + ]); + return { listing, blueprint, components: new Map(components.map((doc) => [doc.path, doc])) }; +} + +export const isRecord = (value: unknown): value is Record => + value !== null && typeof value === 'object' && !Array.isArray(value); + +/** + * Whether `target` resolves inside `root`. + * + * Containment is a property of the **resolved** location rather than of the + * string (blueprint spec §10, listing spec §5): a link committed inside an item + * can point anywhere the process can read. A dangling link whose target lies + * outside the root is an escape too — existence is not what the rule turns on — + * so an unresolvable leaf is resolved through its parent instead. + */ +export function resolvesInside(root: string, target: string): boolean { + const realRoot = realpathOrNearest(root); + const realTarget = realpathOrNearest(target); + if (realRoot === null || realTarget === null) return false; + const relative = path.relative(realRoot, realTarget); + return relative !== '..' && !relative.startsWith(`..${path.sep}`) && !path.isAbsolute(relative); +} + +/** realpath of `target`, or of the deepest ancestor that exists, with the rest re-appended. */ +function realpathOrNearest(target: string): string | null { + let current = path.resolve(target); + const trailing: string[] = []; + for (;;) { + try { + return path.join(fs.realpathSync(current), ...trailing); + } catch { + const parent = path.dirname(current); + if (parent === current) return null; + trailing.unshift(path.basename(current)); + current = parent; + } + } +} diff --git a/tests/lib/markdown.ts b/tests/lib/markdown.ts new file mode 100644 index 0000000..9464e42 --- /dev/null +++ b/tests/lib/markdown.ts @@ -0,0 +1,95 @@ +/** + * The `description` Markdown profile — listing spec §4.1. + * + * `description` is CommonMark 0.31.2 narrowed by three rules. It is authored by a + * third party and rendered by the storefront, so it is untrusted content + * displayed in a first-party origin. + * + * The rules are written in CommonMark's terms rather than as a search for angle + * brackets, and this module parses accordingly: a code fence or a code span is + * its own construct, so a listing MAY document `\n```\n' } }) }); + }); + + it('ERR_DISALLOWED_SCHEME for a javascript: link', async () => { + await assertReports({ listing: listing({ spec: { description: '[go](javascript:alert(1))' } }) }, 'ERR_DISALLOWED_SCHEME'); + }); + + it('permits https, mailto and a fragment', async () => { + await assertClean({ + listing: listing({ spec: { description: '[a](https://x.test) [b](mailto:h@x.test) [c](#usage)\n\n## Usage\n' } }), + }); + }); + + it('ERR_IMAGE_NOT_LOCAL for a remote image', async () => { + // A remote image discloses every storefront viewer's IP address to a host + // the listing author chose, on every page view, with no interaction. + await assertReports( + { listing: listing({ spec: { description: '![shot](https://cdn.test/shot.png)' } }) }, + 'ERR_IMAGE_NOT_LOCAL', + ); + }); + + it('ERR_MEDIA_NOT_FOUND for a local description image the item does not ship', async () => { + await assertReports({ listing: listing({ spec: { description: '![shot](media/shot.png)' } }) }, 'ERR_MEDIA_NOT_FOUND'); + }); +}); + +describe('image pinning — COMP-SRC-001', () => { + for (const tag of ['latest', 'main', 'LATEST', 'edge', 'nightly', 'rolling']) { + it(`ERR_UNPINNED_IMAGE for :${tag}`, async () => { + await assertReports( + { components: { 'components/web.yaml': component({ spec: { workload: { kind: 'SERVICE', source: { type: 'IMAGE', ref: `ghcr.io/acme/web:${tag}` }, endpoints: { primary: { containerPort: 8080, protocol: 'HTTP', visibility: 'PUBLIC' } }, health: { readiness: { path: '/healthz' } } } } }) } }, + 'ERR_UNPINNED_IMAGE', + ); + }); + } + + it('reads the tag as the colon after the final slash', () => { + // A registry port must not read as a tag. + assert.equal(tagOf('localhost:5000/nginx'), null); + assert.equal(tagOf('localhost:5000/nginx:1.27'), '1.27'); + assert.equal(tagOf('redis:8.8.1-alpine'), '8.8.1-alpine'); + }); + + it('accepts a floating tag accompanied by a digest', () => { + // A digest pin satisfies the rule whatever tag accompanies it, because the + // digest is what resolves. + assert.equal(tagOf(`ghcr.io/acme/web:latest@sha256:${'a'.repeat(64)}`), null); + }); +}); + +describe('endpoint resolution — component §5.2, §5.4, §6.1', () => { + const workloadWith = (endpoints: Doc, health: Doc = {}, contract?: Doc): Doc => + component({ + spec: { + workload: { kind: 'SERVICE', source: { type: 'IMAGE', ref: 'ghcr.io/acme/web:1.2.3' }, endpoints, health }, + contract: contract ?? { inputs: {}, outputs: {} }, + }, + }); + + it('ERR_UNKNOWN_ENDPOINT when a probe names an endpoint the workload does not declare', async () => { + await assertReports( + { + components: { + 'components/web.yaml': workloadWith( + { primary: { containerPort: 8080, protocol: 'HTTP', visibility: 'PUBLIC' } }, + { readiness: { path: '/healthz', endpoint: 'console' } }, + ), + }, + }, + 'ERR_UNKNOWN_ENDPOINT', + ); + }); + + it('ERR_AMBIGUOUS_ENDPOINT when a probe omits the endpoint and none is elected', async () => { + // Electing the first name in sort order would let a new endpoint silently + // re-point a probe that has worked for a year. + await assertReports( + { + components: { + 'components/web.yaml': workloadWith( + { + api: { containerPort: 8080, protocol: 'HTTP', visibility: 'PUBLIC' }, + console: { containerPort: 9090, protocol: 'HTTP', visibility: 'PUBLIC' }, + }, + { readiness: { path: '/healthz' } }, + ), + }, + }, + 'ERR_AMBIGUOUS_ENDPOINT', + ); + }); + + it('ERR_ENDPOINT_NOT_HTTP when a probe resolves to a TCP endpoint', async () => { + await assertReports( + { + components: { + 'components/web.yaml': workloadWith( + { primary: { containerPort: 6379, protocol: 'TCP', visibility: 'PRIVATE' } }, + { liveness: { path: '/healthz' } }, + ), + }, + }, + 'ERR_ENDPOINT_NOT_HTTP', + ); + }); + + it('elects the sole PUBLIC endpoint as primary', async () => { + await assertClean({ + components: { + 'components/web.yaml': workloadWith( + { + api: { containerPort: 8080, protocol: 'HTTP', visibility: 'PUBLIC' }, + metrics: { containerPort: 9090, protocol: 'HTTP', visibility: 'PRIVATE' }, + }, + { readiness: { path: '/healthz' } }, + ), + }, + }); + }); + + it('ERR_ENDPOINT_NOT_PUBLIC when a platform default reads a private endpoint', async () => { + await assertReports( + { + components: { + 'components/web.yaml': workloadWith( + { primary: { containerPort: 8080, protocol: 'HTTP', visibility: 'PRIVATE' } }, + {}, + { + inputs: { host: { schema: { type: 'STRING' }, suppliedBy: 'USER', ui: { label: 'Host' }, platformDefault: { source: 'PUBLIC_HOSTNAME' } } }, + outputs: {}, + }, + ), + }, + }, + 'ERR_ENDPOINT_NOT_PUBLIC', + ); + }); + + it('ERR_ENDPOINT_NOT_L4 when PUBLIC_ADDRESS reads an HTTP endpoint', async () => { + // Such an endpoint is published through the shared ingress, so what the + // derivation would yield is the ingress address — true, and not the thing + // an author asking for an edge address is asking for. + await assertReports( + { + components: { + 'components/web.yaml': workloadWith( + { primary: { containerPort: 8080, protocol: 'HTTP', visibility: 'PUBLIC' }, }, + { readiness: { path: '/healthz' } }, + { + inputs: { addr: { schema: { type: 'STRING' }, suppliedBy: 'USER', ui: { label: 'Address' }, platformDefault: { source: 'PUBLIC_ADDRESS' } } }, + outputs: {}, + }, + ), + }, + }, + 'ERR_ENDPOINT_NOT_L4', + ); + }); + + it('ERR_ENDPOINT_NOT_HTTP when PUBLIC_URL reads a TCP endpoint', async () => { + await assertReports( + { + components: { + 'components/web.yaml': workloadWith( + { broker: { containerPort: 1883, protocol: 'TCP', visibility: 'PUBLIC' } }, + {}, + { + inputs: { url: { schema: { type: 'STRING' }, suppliedBy: 'USER', ui: { label: 'URL' }, platformDefault: { source: 'PUBLIC_URL' } } }, + outputs: {}, + }, + ), + }, + }, + 'ERR_ENDPOINT_NOT_HTTP', + ); + }); +}); + +describe('connections — blueprint §4.2', () => { + const db = component({ + spec: { + workload: { + kind: 'SERVICE', + source: { type: 'IMAGE', ref: 'postgres:18.2-alpine' }, + endpoints: { primary: { containerPort: 5432, protocol: 'TCP', visibility: 'PRIVATE' } }, + }, + contract: { + inputs: {}, + outputs: { connectionString: { schema: { type: 'STRING', semanticType: 'POSTGRES' }, valueFrom: 'DERIVED', value: null } }, + }, + }, + }); + + const webConsuming = (inputSchema: Doc): Doc => + component({ + spec: { + workload: { + kind: 'SERVICE', + source: { type: 'IMAGE', ref: 'ghcr.io/acme/web:1.2.3' }, + endpoints: { primary: { containerPort: 8080, protocol: 'HTTP', visibility: 'PUBLIC' } }, + health: { readiness: { path: '/healthz' } }, + }, + contract: { + inputs: { DATABASE_URL: { schema: inputSchema, suppliedBy: 'CONNECTION', ui: null, target: { envVarKey: 'DATABASE_URL' } } }, + outputs: {}, + }, + }, + }); + + const twoNode = (connections: Doc): Doc => + blueprint({ + spec: { + components: { + db: { component: './components/db.yaml', size: 'general.standard.small', connections: {} }, + web: { component: './components/web.yaml', size: 'general.standard.small', connections }, + }, + parameters: {}, + }, + }); + + const files = (inputSchema: Doc) => ({ 'components/db.yaml': db, 'components/web.yaml': webConsuming(inputSchema) }); + + it('accepts a wire whose two ends fit', async () => { + await assertClean({ + blueprint: twoNode({ DATABASE_URL: { fromRole: 'db', fromOutput: 'connectionString' } }), + components: files({ type: 'STRING', semanticType: 'POSTGRES' }), + }); + }); + + it('ERR_UNKNOWN_ROLE when fromRole names no node', async () => { + await assertReports( + { blueprint: twoNode({ DATABASE_URL: { fromRole: 'cache', fromOutput: 'connectionString' } }), components: files({ type: 'STRING', semanticType: 'POSTGRES' }) }, + 'ERR_UNKNOWN_ROLE', + ); + }); + + it('ERR_UNKNOWN_OUTPUT when fromOutput names no output of the producer', async () => { + await assertReports( + { blueprint: twoNode({ DATABASE_URL: { fromRole: 'db', fromOutput: 'dsn' } }), components: files({ type: 'STRING', semanticType: 'POSTGRES' }) }, + 'ERR_UNKNOWN_OUTPUT', + ); + }); + + it('ERR_UNKNOWN_INPUT when the map key names no input of the consumer', async () => { + // A wire whose two ends are each checked and whose consumer end is not is a + // wire that can be misspelled at one end only. + await assertReports( + { blueprint: twoNode({ DATABSE_URL: { fromRole: 'db', fromOutput: 'connectionString' } }), components: files({ type: 'STRING', semanticType: 'POSTGRES' }) }, + 'ERR_UNKNOWN_INPUT', + ); + }); + + it('ERR_UNWIRED_REQUIRED_INPUT when a required CONNECTION input has no wire', async () => { + await assertReports({ blueprint: twoNode({}), components: files({ type: 'STRING', semanticType: 'POSTGRES' }) }, 'ERR_UNWIRED_REQUIRED_INPUT'); + }); + + it('ERR_INCOMPATIBLE_TYPE when the two ends declare different types', async () => { + // No widening in either direction: 5432, 5432.0 and 5.432e3 are one value + // with three spellings. + await assertReports( + { blueprint: twoNode({ DATABASE_URL: { fromRole: 'db', fromOutput: 'connectionString' } }), components: files({ type: 'NUMBER', semanticType: 'POSTGRES' }) }, + 'ERR_INCOMPATIBLE_TYPE', + ); + }); + + it('ERR_INCOMPATIBLE_SEMANTIC_TYPE when a constrained consumer meets a differently tagged producer', async () => { + await assertReports( + { blueprint: twoNode({ DATABASE_URL: { fromRole: 'db', fromOutput: 'connectionString' } }), components: files({ type: 'STRING', semanticType: 'MYSQL' }) }, + 'ERR_INCOMPATIBLE_SEMANTIC_TYPE', + ); + }); + + it('accepts an untagged consumer taking a tagged producer', async () => { + // A consumer declaring null has said the value is not specific to a backing + // service, and nothing it receives can contradict that. + await assertClean({ + blueprint: twoNode({ DATABASE_URL: { fromRole: 'db', fromOutput: 'connectionString' } }), + components: files({ type: 'STRING' }), + }); + }); + + it('ERR_INCOMPATIBLE_SEMANTIC_TYPE when a tagged consumer meets an untagged producer', async () => { + const untaggedDb = component({ + spec: { + workload: { kind: 'SERVICE', source: { type: 'IMAGE', ref: 'postgres:18.2-alpine' }, endpoints: { primary: { containerPort: 5432, protocol: 'TCP', visibility: 'PRIVATE' } } }, + contract: { inputs: {}, outputs: { connectionString: { schema: { type: 'STRING' }, valueFrom: 'DERIVED', value: null } } }, + }, + }); + await assertReports( + { + blueprint: twoNode({ DATABASE_URL: { fromRole: 'db', fromOutput: 'connectionString' } }), + components: { 'components/db.yaml': untaggedDb, 'components/web.yaml': webConsuming({ type: 'STRING', semanticType: 'POSTGRES' }) }, + }, + 'ERR_INCOMPATIBLE_SEMANTIC_TYPE', + ); + }); +}); + +describe('parameters — blueprint §5.2, §5.3', () => { + const withInput = (input: Doc): Doc => + component({ + spec: { + workload: { + kind: 'SERVICE', + source: { type: 'IMAGE', ref: 'ghcr.io/acme/web:1.2.3' }, + endpoints: { primary: { containerPort: 8080, protocol: 'HTTP', visibility: 'PUBLIC' } }, + health: { readiness: { path: '/healthz' } }, + }, + contract: { inputs: { adminPassword: input }, outputs: {} }, + }, + }); + + const required: Doc = { schema: { type: 'STRING' }, isRequired: true, suppliedBy: 'USER', ui: { label: 'Admin password' } }; + + it('ERR_UNBOUND_PARAMETER when a parameter key names no USER input', async () => { + await assertReports( + { + blueprint: blueprint({ spec: { components: { web: { component: './components/web.yaml', size: 'general.standard.small', connections: {} } }, parameters: { legacyMode: { schema: { type: 'STRING' }, isRequired: true } } } }), + components: { 'components/web.yaml': withInput(required) }, + }, + 'ERR_UNBOUND_PARAMETER', + ); + }); + + it('ERR_UNCOVERED_REQUIRED_INPUT when an override forgets a required input', async () => { + await assertReports( + { + blueprint: blueprint({ spec: { components: { web: { component: './components/web.yaml', size: 'general.standard.small', connections: {} } }, parameters: { other: { schema: { type: 'STRING' }, isRequired: true } } } }), + components: { 'components/web.yaml': withInput({ ...required, schema: { type: 'STRING' } }) }, + }, + 'ERR_UNCOVERED_REQUIRED_INPUT', + ); + }); + + it('ERR_UNCOVERED_REQUIRED_INPUT when a parameter names the key but guarantees no value', async () => { + // isRequired defaults to true on a component input and false on a blueprint + // parameter, so an override that copies the key and says nothing else has + // quietly made it optional. + await assertReports( + { + blueprint: blueprint({ spec: { components: { web: { component: './components/web.yaml', size: 'general.standard.small', connections: {} } }, parameters: { adminPassword: { schema: { type: 'STRING' } } } } }), + components: { 'components/web.yaml': withInput(required) }, + }, + 'ERR_UNCOVERED_REQUIRED_INPUT', + ); + }); + + it('ERR_INCOMPATIBLE_PARAMETER_TYPE when a parameter and the input it covers disagree', async () => { + await assertReports( + { + blueprint: blueprint({ spec: { components: { web: { component: './components/web.yaml', size: 'general.standard.small', connections: {} } }, parameters: { adminPassword: { schema: { type: 'NUMBER' }, isRequired: true } } } }), + components: { 'components/web.yaml': withInput(required) }, + }, + 'ERR_INCOMPATIBLE_PARAMETER_TYPE', + ); + }); + + it('accepts an override that guarantees the value', async () => { + await assertClean({ + blueprint: blueprint({ spec: { components: { web: { component: './components/web.yaml', size: 'general.standard.small', connections: {} } }, parameters: { adminPassword: { schema: { type: 'STRING', isSensitive: true }, isRequired: true, ui: { label: 'Admin password' } } } } }), + components: { 'components/web.yaml': withInput(required) }, + }); + }); + + it('ERR_CONFLICTING_INPUT_SCHEMA when two nodes declare one key differently', async () => { + const api = component({ + spec: { + workload: { kind: 'WORKER', source: { type: 'IMAGE', ref: 'ghcr.io/acme/api:1.2.3' } }, + contract: { inputs: { adminPassword: { schema: { type: 'NUMBER' }, suppliedBy: 'USER', ui: { label: 'Admin password' } } }, outputs: {} }, + }, + }); + await assertReports( + { + blueprint: blueprint({ + spec: { + components: { + api: { component: './components/api.yaml', size: 'general.standard.small', connections: {} }, + web: { component: './components/web.yaml', size: 'general.standard.small', connections: {} }, + }, + parameters: {}, + }, + }), + components: { 'components/api.yaml': api, 'components/web.yaml': withInput(required) }, + }, + 'ERR_CONFLICTING_INPUT_SCHEMA', + ); + }); + + it('absorbs an identical redeclaration in silence', async () => { + // Two components that agree on what adminPassword is are not in conflict, + // and `ui` and `isRequired` take no part in the comparison. + const api = component({ + spec: { + workload: { kind: 'WORKER', source: { type: 'IMAGE', ref: 'ghcr.io/acme/api:1.2.3' } }, + contract: { inputs: { adminPassword: { schema: { type: 'STRING' }, isRequired: false, suppliedBy: 'USER', ui: { label: 'Password (api)' } } }, outputs: {} }, + }, + }); + await assertClean({ + blueprint: blueprint({ + spec: { + components: { + api: { component: './components/api.yaml', size: 'general.standard.small', connections: {} }, + web: { component: './components/web.yaml', size: 'general.standard.small', connections: {} }, + }, + parameters: {}, + }, + }), + components: { 'components/api.yaml': api, 'components/web.yaml': withInput(required) }, + }); + }); +}); diff --git a/tests/semantic.test.ts b/tests/semantic.test.ts new file mode 100644 index 0000000..242e49f --- /dev/null +++ b/tests/semantic.test.ts @@ -0,0 +1,148 @@ +/** + * The `semantic` phase — the rules JSON Schema cannot express. + * + * These are the item's cross-document obligations: identity agreement, reference + * resolution, path containment, and the compatibility of the two ends of every + * wire. All of them need a second document or the filesystem, and none of them + * needs the network — which is what makes a repo-local item validate completely + * offline, all the way through this phase. + * + * The `capability` phase is deliberately absent. Whether a Compute Profile is on + * offer and whether a component version is monotonic are decided against the + * platform catalog, and an implementation MUST NOT report a rule it has not been + * given the means to check. + */ +import assert from 'node:assert/strict'; +import { describe, it } from 'node:test'; + +import { discoverItems, loadItemDocuments, type Item, type ItemDocuments } from './lib/catalog.ts'; +import { mediaPathPatternFrom } from './lib/media.ts'; +import { loadSchema } from './lib/spec-schemas.ts'; +import { + buildContext, + checkComponentReferences, + checkConnections, + checkDescription, + checkHealthProbes, + checkIdentity, + checkImagePinning, + checkMedia, + checkParameters, + checkPlatformDefaults, + valueSchemaDefaultsFrom, + type Diagnostic, + type SemanticContext, +} from './lib/semantic.ts'; + +const items = discoverItems(); + +/** + * The media-path grammar and the value-schema defaults are read back out of the + * fetched bundles rather than restated here, so the two places this phase needs + * them cannot drift from what the spec publishes. + */ +async function contextFor(item: Item): Promise<{ context: SemanticContext; documents: ItemDocuments }> { + const [listingBundle, componentBundle, documents] = await Promise.all([ + loadSchema('listing'), + loadSchema('component'), + loadItemDocuments(item), + ]); + + const mediaPathPattern = mediaPathPatternFrom(listingBundle.schema); + const context = buildContext( + item, + documents, + (value) => mediaPathPattern.test(value), + valueSchemaDefaultsFrom(componentBundle.schema), + ); + + return { context, documents }; +} + +const report = (diagnostics: Diagnostic[]): string[] => + diagnostics.map((diagnostic) => `${diagnostic.code} at ${diagnostic.where}: ${diagnostic.message}`); + +for (const item of items) { + describe(`items/${item.slug}`, () => { + it('slug and version agree across the item — BP-ID-001/002, LIST-ID-001/002', async () => { + const { context } = await contextFor(item); + assert.deepEqual(report(checkIdentity(context)), []); + }); + + it('every component reference resolves to a document inside the item root — §4.1, BP-ID-003', async () => { + const { context } = await contextFor(item); + assert.deepEqual(report(checkComponentReferences(context)), []); + }); + + it('every declared media path resolves to a file inside the item root — LIST-MEDIA-001/002/003', async () => { + const { context } = await contextFor(item); + assert.deepEqual(report(checkMedia(context)), []); + }); + + it('the description satisfies the Markdown profile — LIST-MD-001/002/003', async () => { + const { context } = await contextFor(item); + assert.deepEqual(report(checkDescription(context)), []); + }); + + it('every image reference is pinned — COMP-SRC-001', async () => { + const { context } = await contextFor(item); + assert.deepEqual(report(checkImagePinning(context)), []); + }); + + it('every health probe resolves to an HTTP-family endpoint — §5.4', async () => { + const { context } = await contextFor(item); + assert.deepEqual(report(checkHealthProbes(context)), []); + }); + + it('every platform default resolves to a public endpoint of the right address form — §6.1', async () => { + const { context } = await contextFor(item); + assert.deepEqual(report(checkPlatformDefaults(context)), []); + }); + + it('every connection resolves at both ends and the two fit — §4.2', async () => { + const { context } = await contextFor(item); + assert.deepEqual(report(checkConnections(context)), []); + }); + + it('the install form covers what a deploying user must supply — §5.2, §5.3', async () => { + const { context } = await contextFor(item); + assert.deepEqual(report(checkParameters(context)), []); + }); + }); +} + +describe('the corpus as a whole', () => { + it('declares every media file it ships', async () => { + // Not a spec rule — nothing rejects an item for shipping an asset it never + // declares. It is a catalog rule: an undeclared file is bytes the storefront + // never serves, and the reason it is here is that a rename leaves the old + // file behind and nothing else in the pipeline notices. + const orphans: string[] = []; + + for (const item of items) { + const { context, documents } = await contextFor(item); + if (!documents.listing?.value) continue; + + const declared = new Set(); + const spec = documents.listing.value['spec'] as Record | undefined; + const icon = spec?.['icon']; + if (typeof icon === 'string') declared.add(icon); + for (const shot of (spec?.['screenshots'] as { file?: unknown }[] | undefined) ?? []) { + if (typeof shot?.file === 'string') declared.add(shot.file); + } + const description = spec?.['description']; + if (typeof description === 'string') { + for (const match of description.matchAll(/!\[[^\]]*\]\(\s*(\S+?)\s*[)\s]/g)) { + if (match[1] && context.isMediaPath(match[1])) declared.add(match[1]); + } + } + + for (const absolute of item.mediaPaths) { + const itemRelative = absolute.slice(item.root.length + 1).split(/[\\/]/).join('/'); + if (!declared.has(itemRelative)) orphans.push(`items/${item.slug}/${itemRelative}`); + } + } + + assert.deepEqual(orphans, []); + }); +}); diff --git a/tests/spec.test.ts b/tests/spec.test.ts new file mode 100644 index 0000000..2232c11 --- /dev/null +++ b/tests/spec.test.ts @@ -0,0 +1,51 @@ +/** + * The spec bundles themselves. + * + * These tests fail first and loudest when the contract cannot be reached or + * comes back as something other than what it claims — before any item is judged + * against it. A corpus validated against a 404 page passes everything. + */ +import assert from 'node:assert/strict'; +import { describe, it } from 'node:test'; + +import { FAMILIES, KIND_OF, externalRefs, loadSchema } from './lib/spec-schemas.ts'; + +describe('musher-dev/spec', () => { + it('resolves all three bundles from the public repository at run time', async () => { + const bundles = await Promise.all(FAMILIES.map(loadSchema)); + for (const bundle of bundles) { + console.log(` ${bundle.family.padEnd(10)} sha256:${bundle.sha256.slice(0, 12)} ${bundle.origin}`); + } + assert.equal(bundles.length, FAMILIES.length); + }); + + for (const family of FAMILIES) { + describe(family, () => { + it('is a JSON Schema 2020-12 document', async () => { + const { schema } = await loadSchema(family); + assert.equal(schema['$schema'], 'https://json-schema.org/draft/2020-12/schema'); + }); + + it(`discriminates documents on kind: ${KIND_OF[family]}`, async () => { + const { schema } = await loadSchema(family); + const properties = schema['properties'] as Record; + assert.equal(properties['kind']?.const, KIND_OF[family]); + }); + + it('is self-contained — every $ref resolves inside the bundle', async () => { + // spec README: no validator ever needs to make a network request to + // evaluate a document, which is what makes offline validation possible. + const { schema } = await loadSchema(family); + assert.deepEqual(externalRefs(schema), []); + }); + + it('closes the document envelope', async () => { + // COMP-ENV-005: unknown properties are rejected at every level, so a + // misspelled field is an error rather than a silently ignored one. + const { schema } = await loadSchema(family); + assert.equal(schema['additionalProperties'], false); + assert.deepEqual(schema['required'], ['specVersion', 'kind', 'metadata', 'spec']); + }); + }); + } +}); diff --git a/tests/structural.test.ts b/tests/structural.test.ts new file mode 100644 index 0000000..417d858 --- /dev/null +++ b/tests/structural.test.ts @@ -0,0 +1,63 @@ +/** + * The `structural` phase — each document against its family's JSON Schema. + * + * The bundles are fetched from musher-dev/spec at run time rather than vendored, + * so what the corpus is judged against is the contract as it currently stands, + * not a copy of it that has quietly fallen behind. + */ +import assert from 'node:assert/strict'; +import { describe, it } from 'node:test'; + +import { discoverItems, loadItemDocuments, type LoadedDocument } from './lib/catalog.ts'; +import { KIND_OF, formatAjvErrors, validatorFor, type Family } from './lib/spec-schemas.ts'; +import { rel } from './lib/paths.ts'; + +const items = discoverItems(); + +async function assertValidates(family: Family, document: LoadedDocument): Promise { + assert.ok(document.value, `${document.label} did not survive the parser phase`); + + // The envelope's own discriminator is checked first: validating a listing + // against the component schema produces a wall of errors that says nothing + // about the actual mistake. + assert.equal( + document.value['kind'], + KIND_OF[family], + `${document.label} declares kind ${JSON.stringify(document.value['kind'])}, expected ${KIND_OF[family]}`, + ); + + const validate = await validatorFor(family); + const valid = validate(document.value); + assert.ok(valid, `${document.label} does not validate against the ${family} schema:\n${formatAjvErrors(validate.errors)}`); +} + +for (const item of items) { + describe(`items/${item.slug}`, () => { + it('listing.yaml validates against the listing schema', async () => { + const { listing } = await loadItemDocuments(item); + assert.ok(listing, `items/${item.slug} holds no listing.yaml`); + await assertValidates('listing', listing); + }); + + it('blueprint.yaml validates against the blueprint schema', async () => { + const { blueprint } = await loadItemDocuments(item); + assert.ok(blueprint, `items/${item.slug} holds no blueprint.yaml`); + await assertValidates('blueprint', blueprint); + }); + + it('every component document validates against the component schema', async () => { + const { components } = await loadItemDocuments(item); + assert.ok(components.size > 0, `items/${item.slug} holds no component document`); + + const failures: string[] = []; + for (const [componentPath, document] of components) { + try { + await assertValidates('component', document); + } catch (error) { + failures.push(`${rel(componentPath)}\n${(error as Error).message}`); + } + } + assert.deepEqual(failures, []); + }); + }); +} diff --git a/tsconfig.json b/tsconfig.json new file mode 100644 index 0000000..3651c1f --- /dev/null +++ b/tsconfig.json @@ -0,0 +1,25 @@ +{ + "compilerOptions": { + "target": "es2023", + "lib": ["es2023"], + "module": "nodenext", + "moduleResolution": "nodenext", + "types": ["node"], + "esModuleInterop": true, + + "strict": true, + "noUncheckedIndexedAccess": true, + "noImplicitOverride": true, + "noFallthroughCasesInSwitch": true, + "exactOptionalPropertyTypes": false, + + "allowImportingTsExtensions": true, + "rewriteRelativeImportExtensions": false, + "verbatimModuleSyntax": true, + "erasableSyntaxOnly": true, + "isolatedModules": true, + "noEmit": true, + "skipLibCheck": true + }, + "include": ["tests/**/*.ts"] +} From 198cac6130b649d996848a7d140dcb3e555ebacd Mon Sep 17 00:00:00 2001 From: Ali S Date: Tue, 25 Aug 2026 12:33:13 +0000 Subject: [PATCH 3/5] ci: run the corpus validation and dev container checks MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Two workflows, for two things that fail differently. validate typechecks, lints the workflows and runs the suite on every push and pull request. It also runs on a daily schedule, which is the part that only makes sense because the schemas are not vendored: a spec change can stop accepting an item that used to validate, and without a scheduled run nothing notices until the next catalog PR happens to come along. devcontainer covers the thing nobody can check without building it — the scripts, the lockfile against devcontainer.json, and the .env template against what the tasks actually read. Path-filtered, so it stays quiet on an items-only change. Both are contents: read and take no credential. musher-dev/spec is public. Co-Authored-By: Claude Opus 5 (1M context) --- .github/workflows/devcontainer.yml | 66 ++++++++++++++++++++++++++++++ .github/workflows/validate.yml | 48 ++++++++++++++++++++++ 2 files changed, 114 insertions(+) create mode 100644 .github/workflows/devcontainer.yml create mode 100644 .github/workflows/validate.yml diff --git a/.github/workflows/devcontainer.yml b/.github/workflows/devcontainer.yml new file mode 100644 index 0000000..8dd9b12 --- /dev/null +++ b/.github/workflows/devcontainer.yml @@ -0,0 +1,66 @@ +# Dev container integrity. +# +# The container is the thing nobody can validate on their own machine without +# building it, so this workflow is where its three failure modes are caught: a +# broken script, a lockfile that has drifted from devcontainer.json, and an +# .env template that no longer matches what the tasks read. +name: Dev Container + +on: + push: + branches: [main] + paths: ['.devcontainer/**', 'Taskfile.yml', '.github/workflows/devcontainer.yml'] + pull_request: + paths: ['.devcontainer/**', 'Taskfile.yml', '.github/workflows/devcontainer.yml'] + workflow_dispatch: + +permissions: + contents: read + +concurrency: + group: devcontainer-${{ github.ref }} + cancel-in-progress: true + +jobs: + shellcheck: + name: ShellCheck + runs-on: ubuntu-latest + timeout-minutes: 5 + steps: + - uses: actions/checkout@v4 + - uses: ludeeus/action-shellcheck@master + env: + # SCRIPTDIR resolves `# shellcheck source=` from each script's own + # directory. Without it every sourced library is SC1091 noise. + SHELLCHECK_OPTS: -x -S style --source-path=SCRIPTDIR + with: + scandir: .devcontainer/scripts + + lockfile: + name: Feature Lockfile + runs-on: ubuntu-latest + timeout-minutes: 20 + steps: + - uses: actions/checkout@v4 + # --frozen-lockfile fails if devcontainer-lock.json is missing or if its + # resolved Feature digests drift from devcontainer.json. Building is also + # the only real proof the Feature set still resolves together. + - name: Build the container and enforce devcontainer-lock.json is current + run: npx -y @devcontainers/cli build --workspace-folder . --frozen-lockfile + + env-check: + name: Env Template Sync + runs-on: ubuntu-latest + timeout-minutes: 5 + steps: + - uses: actions/checkout@v4 + # No repo-token. Nothing in this repository authenticates to GitHub: the + # spec is public, and this workflow is path-filtered so it runs rarely + # enough that the unauthenticated release-download limit is not a concern. + - uses: go-task/setup-task@v2 + with: + version: 3.52.0 + - name: Verify .env and .env.example stay in sync + run: | + cp .devcontainer/.env.example .devcontainer/.env + task env:check diff --git a/.github/workflows/validate.yml b/.github/workflows/validate.yml new file mode 100644 index 0000000..2537d01 --- /dev/null +++ b/.github/workflows/validate.yml @@ -0,0 +1,48 @@ +# Validates the catalog corpus against musher-dev/spec. +# +# The schemas are fetched from the tip of the public musher-dev/spec repository +# at run time rather than vendored, so this job also tells us when a spec change +# stops accepting an item that used to validate — which is the whole point of not +# holding a copy. The scheduled run is what catches that in the window between a +# spec change and the next catalog PR. +# +# No credential is involved: musher-dev/spec is public and the suite reads no +# token. +name: validate + +on: + push: + branches: [main] + pull_request: + schedule: + - cron: '17 6 * * *' + workflow_dispatch: + +permissions: + contents: read + +concurrency: + group: validate-${{ github.ref }} + cancel-in-progress: true + +jobs: + validate: + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v4 + + - uses: actions/setup-node@v4 + with: + node-version: '24' + cache: npm + + - run: npm ci + + - name: Typecheck + run: npm run typecheck + + - name: Lint the workflows + uses: rhysd/actionlint@main + + - name: Validate the corpus + run: npm test From 5dd8f3f7ab5f28ae9ac30debd7f05e88de7eef34 Mon Sep 17 00:00:00 2001 From: Ali S Date: Tue, 25 Aug 2026 12:33:14 +0000 Subject: [PATCH 4/5] chore(devcontainer): add the dev container, task runner and git hooks MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Gives the repository a reproducible environment and the same checks CI runs, locally. Taskfile wraps the npm scripts rather than replacing them: `npm test` stays what CI runs and what works in a bare checkout with no Task installed, and `task` is the ergonomics layer on top. One taskfile, not the split musher-dev/spec uses, because there is one build surface here. The lefthook split is about the network. `npm test` fetches the spec bundles on every run, so it cannot be a pre-commit job — a commit made offline would fail for a reason that has nothing to do with the commit. Pre-commit holds what is local and fast (types, shell, workflows); pre-push holds the suite. commit-msg enforces Conventional Commits. .env.example is committed and .env is not, so env:check reports the keys a stale local copy is missing rather than letting a task read an unset one. Co-Authored-By: Claude Opus 5 (1M context) --- .config/lefthook.yml | 56 +++++++ .devcontainer/.env.example | 29 ++++ .devcontainer/devcontainer-lock.json | 34 +++++ .devcontainer/devcontainer.json | 135 +++++++++++++++++ .devcontainer/mise.toml | 22 +++ .devcontainer/scripts/env-check.sh | 57 +++++++ .devcontainer/scripts/initialize.sh | 55 +++++++ .devcontainer/scripts/lib/base-setup.sh | 185 +++++++++++++++++++++++ .devcontainer/scripts/lib/common.sh | 176 ++++++++++++++++++++++ .devcontainer/scripts/lib/env-check.sh | 79 ++++++++++ .devcontainer/scripts/lib/motd.sh | 188 ++++++++++++++++++++++++ .devcontainer/scripts/post-create.sh | 81 ++++++++++ .devcontainer/scripts/startup.sh | 45 ++++++ Taskfile.yml | 141 ++++++++++++++++++ 14 files changed, 1283 insertions(+) create mode 100644 .config/lefthook.yml create mode 100644 .devcontainer/.env.example create mode 100644 .devcontainer/devcontainer-lock.json create mode 100644 .devcontainer/devcontainer.json create mode 100644 .devcontainer/mise.toml create mode 100755 .devcontainer/scripts/env-check.sh create mode 100755 .devcontainer/scripts/initialize.sh create mode 100644 .devcontainer/scripts/lib/base-setup.sh create mode 100644 .devcontainer/scripts/lib/common.sh create mode 100644 .devcontainer/scripts/lib/env-check.sh create mode 100644 .devcontainer/scripts/lib/motd.sh create mode 100755 .devcontainer/scripts/post-create.sh create mode 100755 .devcontainer/scripts/startup.sh create mode 100644 Taskfile.yml diff --git a/.config/lefthook.yml b/.config/lefthook.yml new file mode 100644 index 0000000..8e0a612 --- /dev/null +++ b/.config/lefthook.yml @@ -0,0 +1,56 @@ +# Git hooks. Lefthook discovers this file itself, by searching +# `lefthook.*` -> `.lefthook.*` -> `.config/lefthook.*` and stopping at the +# first match — which is why it is the one config here not passed by path, and +# why a stray lefthook.yml at the repo root would silently shadow it. +# +# Keep min_version in step with .devcontainer/mise.toml. +min_version: 2.1.10 + +# The split between the two hooks is about the network. +# +# `npm test` fetches the spec bundles from musher-dev/spec on every run — that +# is the point of it — so it cannot be a pre-commit job: a commit on a train +# would fail for a reason that has nothing to do with the commit. Pre-commit +# holds the checks that are local and fast; pre-push holds the one that is +# neither. +pre-commit: + parallel: true + jobs: + - name: types + glob: 'tests/**/*.ts' + run: task typecheck + + - name: shell + glob: '.devcontainer/**/*.sh' + run: task check:shell + + - name: workflow + glob: '.github/workflows/*.{yml,yaml}' + run: task check:workflow + +pre-push: + jobs: + - name: validate + run: task test + +commit-msg: + jobs: + - name: conventional-commits + run: | + sh -ceu ' + first_line=$(head -n 1 "$1") + case "$first_line" in + Merge*) exit 0 ;; + esac + pattern="^(feat|fix|chore|docs|style|refactor|perf|test|ci|build|revert)(\\(.+\\))?(!)?: .+" + if ! printf "%s\n" "$first_line" | grep -Eq "$pattern"; then + printf "%s\n" "ERROR: Commit message does not follow Conventional Commits format." >&2 + printf "%s\n" "" >&2 + printf "%s\n" " Expected: (): " >&2 + printf " Got: %s\n" "$first_line" >&2 + printf "%s\n" "" >&2 + printf "%s\n" " Types: feat, fix, chore, docs, style, refactor, perf, test, ci, build, revert" >&2 + printf "%s\n" " Scopes: items, tests, devcontainer, ci, docs, repo, deps" >&2 + exit 1 + fi + ' -- {1} diff --git a/.devcontainer/.env.example b/.devcontainer/.env.example new file mode 100644 index 0000000..4ffef61 --- /dev/null +++ b/.devcontainer/.env.example @@ -0,0 +1,29 @@ +# ============================================================ +# Dev Container Environment Template +# ============================================================ +# On first container build, `initializeCommand` copies this file to +# `.devcontainer/.env` (gitignored) on the host. `runArgs --env-file` +# then loads it into the container, so shells and `task` runs see the +# same values. +# +# This repository holds YAML and validates it against the published +# spec. It runs no services and needs no credentials, so there is +# nothing required here — the file exists so the bootstrap has a valid +# target and so per-developer overrides have somewhere to live. +# +# Drift check: `task env:check` compares this file against your local +# `.env` and flags missing keys. +# +# Convention (three states): +# 1. Filled defaults — `VAR=value` safe defaults; override only if needed. +# 2. Required (empty) — `VAR=` must be filled in; startup warns. +# 3. Optional overrides — `# VAR=value` uncomment to enable. +# ============================================================ + + +# === Optional overrides ===================================== +# Skip the Claude Code install on a slow connection. +# MUSHER_INSTALL_CLAUDE=0 + +# Per-request timeout when fetching a schema, in milliseconds. +# MUSHER_SPEC_TIMEOUT_MS=15000 diff --git a/.devcontainer/devcontainer-lock.json b/.devcontainer/devcontainer-lock.json new file mode 100644 index 0000000..42ddbe2 --- /dev/null +++ b/.devcontainer/devcontainer-lock.json @@ -0,0 +1,34 @@ +{ + "features": { + "ghcr.io/devcontainers-extra/features/go-task:1": { + "version": "1.0.6", + "resolved": "ghcr.io/devcontainers-extra/features/go-task@sha256:4d1db153919976cadd3209ca05d655a761a01707767716994dad677b4538dc1b", + "integrity": "sha256:4d1db153919976cadd3209ca05d655a761a01707767716994dad677b4538dc1b" + }, + "ghcr.io/devcontainers/features/common-utils:2": { + "version": "2.5.9", + "resolved": "ghcr.io/devcontainers/features/common-utils@sha256:cb0c4d3c276f157eed17935747e364178d75fee17f55c4e129966f64633deb3a", + "integrity": "sha256:cb0c4d3c276f157eed17935747e364178d75fee17f55c4e129966f64633deb3a" + }, + "ghcr.io/devcontainers/features/git:1": { + "version": "1.3.8", + "resolved": "ghcr.io/devcontainers/features/git@sha256:fd75977de13a9979000e0e78baf949adb0ca71d2398995fa22e0a36d7e7e7fe2", + "integrity": "sha256:fd75977de13a9979000e0e78baf949adb0ca71d2398995fa22e0a36d7e7e7fe2" + }, + "ghcr.io/devcontainers/features/github-cli:1": { + "version": "1.1.1", + "resolved": "ghcr.io/devcontainers/features/github-cli@sha256:94879eebb6a0e4e2f197de9f12db7427cb4a25b82d93c55239ce8c8fc394a1b4", + "integrity": "sha256:94879eebb6a0e4e2f197de9f12db7427cb4a25b82d93c55239ce8c8fc394a1b4" + }, + "ghcr.io/devcontainers/features/node:2": { + "version": "2.1.0", + "resolved": "ghcr.io/devcontainers/features/node@sha256:586c9a6f7dd40bd3ba2cd41e7f2f88dcc31fbe5d1442afcbf07ffbc66b686857", + "integrity": "sha256:586c9a6f7dd40bd3ba2cd41e7f2f88dcc31fbe5d1442afcbf07ffbc66b686857" + }, + "ghcr.io/lukewiwa/features/shellcheck:0": { + "version": "0.2.3", + "resolved": "ghcr.io/lukewiwa/features/shellcheck@sha256:9d61a060404fb127ae0c3aea3adbf85869ba575ab8e612f168c82c48ef0c502e", + "integrity": "sha256:9d61a060404fb127ae0c3aea3adbf85869ba575ab8e612f168c82c48ef0c502e" + } + } +} diff --git a/.devcontainer/devcontainer.json b/.devcontainer/devcontainer.json new file mode 100644 index 0000000..4e37d5e --- /dev/null +++ b/.devcontainer/devcontainer.json @@ -0,0 +1,135 @@ +// musher-dev/catalog dev container. +// +// Trimmed from the same musher-dev/development-container template +// musher-dev/spec uses, and trimmed further. This repository holds YAML and +// validates it against the published spec: it builds no images, runs no +// services, needs no database, and has no compose stack. Docker-in-Docker, +// Python, Go, Java, Deno, uv, Bun and the postgres client are all deliberately +// absent. +// +// Kept: Node for the test suite, Task as the runner, shellcheck and actionlint +// for the scripts and workflows, gh for release work, and Claude Code. +{ + "name": "Musher Catalog", + + // Pinned to the LTS, not the floating :ubuntu tag — that one rolls to interim + // releases some upstream Features do not support. + "image": "mcr.microsoft.com/devcontainers/base:ubuntu-24.04", + + "workspaceMount": "source=${localWorkspaceFolder},target=/workspaces/${localWorkspaceFolderBasename},type=bind,consistency=cached", + "workspaceFolder": "/workspaces/${localWorkspaceFolderBasename}", + + // Every Feature is version-pinned and captured in devcontainer-lock.json. + // Tools with no Feature (lefthook, actionlint) live in + // .devcontainer/mise.toml; Claude Code self-updates via its own installer. + // + // Node is pinned to 24 because the test suite is TypeScript run directly by + // Node's type stripping — there is no build step, and no version below 23.6 + // can execute it. + "features": { + "ghcr.io/devcontainers/features/common-utils:2": { + "installZsh": true, + "configureZshAsDefaultShell": true, + "installOhMyZsh": true, + "installOhMyZshConfig": true, + "upgradePackages": true + }, + "ghcr.io/devcontainers/features/git:1": {}, + "ghcr.io/devcontainers/features/github-cli:1": { "version": "2.96.0" }, + "ghcr.io/devcontainers/features/node:2": { "version": "24.18.0" }, + "ghcr.io/devcontainers-extra/features/go-task:1": { "version": "3.52.0" }, + "ghcr.io/lukewiwa/features/shellcheck:0": { "version": "v0.11.0" } + }, + + "containerUser": "vscode", + "remoteUser": "vscode", + + "init": true, + "shutdownAction": "stopContainer", + + // Loaded at `docker run` time so shells and `task` runs see the same values. + "runArgs": ["--env-file", "${localWorkspaceFolder}/.devcontainer/.env"], + + // Named volumes, so a `gh auth login` and a Claude sign-in survive a rebuild. + "mounts": [ + "source=musher-catalog-${devcontainerId}-gh-config,target=/home/vscode/.config/gh,type=volume", + "source=musher-catalog-${devcontainerId}-claude-config,target=/home/vscode/.claude,type=volume" + ], + + "containerEnv": { + // mise's manifest lives under .devcontainer/, not the repo root. + "MISE_GLOBAL_CONFIG_FILE": "${containerWorkspaceFolder}/.devcontainer/mise.toml", + "MISE_TRUSTED_CONFIG_PATHS": "${containerWorkspaceFolder}", + + "XDG_CACHE_HOME": "/home/vscode/.cache", + "NPM_CONFIG_CACHE": "/home/vscode/.cache/npm", + + "MUSHER_INSTALL_CLAUDE": "1" + }, + + "remoteEnv": { + "PATH": "/home/vscode/.local/share/mise/shims:/home/vscode/.local/bin:${containerEnv:PATH}" + }, + + // Host-side: creates .devcontainer/.env from the template and strips CRLF so + // --env-file above has a valid target on a fresh clone. Run through `bash -c` + // so CRLF in the script files cannot break the strip-and-run bootstrap. + "initializeCommand": [ + "bash", + "-c", + "find .devcontainer/scripts -name '*.sh' -exec sed -i 's/\\r$//' {} + 2>/dev/null; bash .devcontainer/scripts/initialize.sh" + ], + + "waitFor": "postCreateCommand", + "postCreateCommand": { + "fix-crlf": "find .devcontainer/scripts -type f -name '*.sh' -exec sed -i 's/\\r$//' {} +", + "setup": ["bash", ".devcontainer/scripts/post-create.sh"] + }, + "postStartCommand": ["bash", ".devcontainer/scripts/startup.sh"], + + "customizations": { + "vscode": { + "extensions": [ + // VS Code does not read .editorconfig on its own, and .editorconfig is + // this repository's single source for whitespace and line endings. + "editorconfig.editorconfig", + "redhat.vscode-yaml", + "timonwong.shellcheck", + "eamodio.gitlens", + "bierner.markdown-mermaid", + "github.copilot", + "github.copilot-chat" + ], + "settings": { + "editor.formatOnSave": true, + "editor.rulers": [80, 100], + + // Whitespace, line endings and encoding are NOT set here. .editorconfig + // states them once, for every editor, and the EditorConfig extension + // above applies them inside the container. Restating them made VS Code + // trim trailing whitespace from Markdown, where two trailing spaces are + // a hard line break and .editorconfig deliberately preserves them. + + "terminal.integrated.defaultProfile.linux": "zsh", + "terminal.integrated.profiles.linux": { + "bash": { "path": "/bin/bash" }, + "zsh": { "path": "/bin/zsh" } + }, + + // Authoring aid: red-squiggle an item in the editor against the same + // schemas `npm test` validates it with — the tip of the public + // musher-dev/spec repository. One source here and in the test suite, or + // the editor and the build disagree about what an item has to satisfy. + "yaml.schemas": { + "https://raw.githubusercontent.com/musher-dev/spec/main/specifications/listing/v1/schemas/dist/listing.schema.json": "items/*/listing.yaml", + "https://raw.githubusercontent.com/musher-dev/spec/main/specifications/blueprint/v1/schemas/dist/blueprint.schema.json": "items/*/blueprint.yaml", + "https://raw.githubusercontent.com/musher-dev/spec/main/specifications/component/v1/schemas/dist/component.schema.json": "items/*/components/*.yaml" + }, + + // The corpus is authored YAML, not generated. Keep the formatter off it + // so a drive-by save does not reflow a document nobody edited. + "[yaml]": { "editor.formatOnSave": false } + } + } + } +} diff --git a/.devcontainer/mise.toml b/.devcontainer/mise.toml new file mode 100644 index 0000000..6047310 --- /dev/null +++ b/.devcontainer/mise.toml @@ -0,0 +1,22 @@ +# Developer tools with no devcontainer Feature. +# +# Everything that has a Feature is pinned in devcontainer.json and baked into +# the image. Everything else is pinned here, whatever it is distributed as — +# mise resolves each tool for the container's own CPU architecture, which is +# what a hand-placed release binary does not do. Claude Code is the exception: +# it self-updates through its own installer (scripts/lib/base-setup.sh). +# +# actionlint must stay in step with .github/workflows/*.yml if a workflow ever +# installs it itself. Two environments, one version — CI is not a mise host and +# does not read this file. +# +# Pinned exactly rather than to a range. A range resolves the newest match +# through the GitHub API on every run, so a degraded API takes the whole setup +# down before it reaches a single check. +# +# Change a version here, then rebuild the container or run `mise install`. +# Docs: https://mise.jdx.dev + +[tools] +"actionlint" = "1.7.11" +"npm:lefthook" = "2.1.10" diff --git a/.devcontainer/scripts/env-check.sh b/.devcontainer/scripts/env-check.sh new file mode 100755 index 0000000..f27cb5e --- /dev/null +++ b/.devcontainer/scripts/env-check.sh @@ -0,0 +1,57 @@ +#!/usr/bin/env bash +# env-check.sh — CLI wrapper around the env-check library. +# +# Reports keys present in .devcontainer/.env.example but missing from the +# developer's .devcontainer/.env, and keys left at the empty "please fill this +# in" state. +# +# The logic lives in lib/env-check.sh so the MOTD can reuse it. This file +# exists so `task env:check` has something to invoke that is a real script — +# Task runs commands through its own shell, and `source` is not something to +# rely on there. +# +# Usage: bash .devcontainer/scripts/env-check.sh +# Returns: 0 when the local .env matches the template, 1 on drift. +set -euo pipefail + +SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" +readonly SCRIPT_DIR +readonly DEVCONTAINER_DIR="${SCRIPT_DIR}/.." +readonly ENV_FILE="${DEVCONTAINER_DIR}/.env" +readonly ENV_EXAMPLE="${DEVCONTAINER_DIR}/.env.example" + +# shellcheck source=lib/env-check.sh +source "${SCRIPT_DIR}/lib/env-check.sh" + +main() { + if [[ ! -f "${ENV_FILE}" ]]; then + echo "env: no .devcontainer/.env — run 'task env:reset' to create one" >&2 + return 1 + fi + + local status=0 + + local missing + missing="$(env_check_drift "${ENV_FILE}" "${ENV_EXAMPLE}" 2>&1 || true)" + if [[ -n "${missing}" ]]; then + echo "env: keys in .env.example missing from .env:" >&2 + awk '{print " - " $0}' <<< "${missing}" >&2 + echo "env: run 'task env:reset' to sync" >&2 + status=1 + fi + + local required + required="$(env_check_required "${ENV_FILE}" || true)" + if [[ -n "${required}" ]]; then + # Informational, not a failure: the three-state grammar in .env.example + # uses an empty value to mean "you must fill this in", and this repository + # ships none — but a developer may add one. + echo "env: keys awaiting a value:" >&2 + awk '{print " - " $0}' <<< "${required}" >&2 + fi + + ((status == 0)) && echo "env: .env matches .env.example" + return "${status}" +} + +main "$@" diff --git a/.devcontainer/scripts/initialize.sh b/.devcontainer/scripts/initialize.sh new file mode 100755 index 0000000..2b4b333 --- /dev/null +++ b/.devcontainer/scripts/initialize.sh @@ -0,0 +1,55 @@ +#!/usr/bin/env bash +# initialize.sh — Host-side bootstrap for the dev container. +# +# Runs on the host (via devcontainer.json `initializeCommand`) BEFORE +# `docker run` is invoked. Because `runArgs --env-file` is evaluated at +# `docker run` time, the .env file must exist on the host before the +# container starts — that's why this work lives here, not in +# post-create.sh. +# +# Responsibilities: +# * Create .devcontainer/.env from .env.example on first clone. +# * Touch an empty .env if no example exists, so --env-file never hard-fails. +# * Strip CRLF from .env (Windows/WSL safety — docker --env-file +# rejects files with CRLF line endings). +# +# Idempotent: safe to run on every container start. +set -euo pipefail + +SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" +readonly SCRIPT_DIR +readonly DEVCONTAINER_DIR="${SCRIPT_DIR}/.." +readonly ENV_FILE="${DEVCONTAINER_DIR}/.env" +readonly ENV_EXAMPLE="${DEVCONTAINER_DIR}/.env.example" + +log() { + echo "[initialize] $*" >&2 +} + +ensure_env_file() { + if [[ -f "${ENV_FILE}" ]]; then + return 0 + fi + if [[ -f "${ENV_EXAMPLE}" ]]; then + log "Creating .devcontainer/.env from .env.example" + cp "${ENV_EXAMPLE}" "${ENV_FILE}" + else + log "No .env.example found; creating empty .devcontainer/.env" + : > "${ENV_FILE}" + fi +} + +strip_crlf() { + [[ -f "${ENV_FILE}" ]] || return 0 + if grep -q $'\r' "${ENV_FILE}" 2>/dev/null; then + log "Stripping CRLF from .devcontainer/.env" + sed -i 's/\r$//' "${ENV_FILE}" + fi +} + +main() { + ensure_env_file + strip_crlf +} + +main "$@" diff --git a/.devcontainer/scripts/lib/base-setup.sh b/.devcontainer/scripts/lib/base-setup.sh new file mode 100644 index 0000000..102cee5 --- /dev/null +++ b/.devcontainer/scripts/lib/base-setup.sh @@ -0,0 +1,185 @@ +#!/usr/bin/env bash +# base-setup.sh — Reusable setup orchestrator for musher dev containers. +# +# This file is intended to be sourced, not executed directly. +# Source it and call base_setup, or call individual functions to customize. +# +# Usage: +# source "path/to/base-setup.sh" +# base_setup +set -euo pipefail + +# Guard against direct execution — this file must be sourced. +if [[ "${BASH_SOURCE[0]}" == "${0}" ]]; then + echo "Error: source this file, don't execute it" >&2 + exit 1 +fi + +_LIB_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" +readonly _LIB_DIR +readonly _HOME="/home/${REMOTE_USER:-vscode}" + +# shellcheck source=common.sh +source "${_LIB_DIR}/common.sh" + +# --- Config directories --- + +# Creates standard config directories for dev tools. +# +# Globals: +# _HOME — read, user home directory +# Outputs: +# Writes progress to stderr via log() +base_setup_config_dirs() { + setup_config_dirs \ + "gh config:${_HOME}/.config/gh" \ + "claude:${_HOME}/.claude" +} + +# --- Cache directories --- + +# Creates cache directories so caches land under a single tree instead of +# scattering across the filesystem. +# +# Only the caches this repository can actually produce are created. The +# template's uv, ruff, pip, mypy, deno, go and bun entries are absent because +# none of those runtimes is installed — a directory for a tool that does not +# exist is a claim the container does not honour. +# +# Globals: +# _HOME — read, user home directory +# Outputs: +# Writes progress to stderr via log() +base_setup_cache_dirs() { + setup_config_dirs \ + "xdg cache:${_HOME}/.cache" \ + "npm cache:${_HOME}/.cache/npm" +} + +# --- NVM --- + +# The Node feature installs Node via nvm; fix nvm's ownership so global npm +# installs work. Delegates to fix_nvm_permissions from common.sh. +base_fix_nvm_permissions() { + fix_nvm_permissions +} + +# --- mise (pins the CLIs that have no devcontainer Feature) --- + +readonly _MISE_BIN="${_HOME}/.local/bin/mise" +readonly _MISE_SHIMS="${_HOME}/.local/share/mise/shims" + +# Puts the mise shims and ~/.local/bin on PATH for the rest of this script, so +# mise-managed CLIs and Claude are visible to base_verify_tools (lifecycle +# hooks don't always inherit devcontainer.json remoteEnv). +# +# Globals: +# PATH — modified (export) +base_setup_path() { + export PATH="${_MISE_SHIMS}:${_HOME}/.local/bin:${PATH}" +} + +# Installs mise via the official installer if not already present. +# +# Outputs: +# Writes progress to stderr via log() +# Returns: +# 0 on success, non-zero on failure +base_install_mise() { + if has_cmd mise; then + log "mise already installed, skipping" + return 0 + fi + log "Installing mise (https://mise.run)..." + retry 3 5 bash -c 'curl -fsSL https://mise.run | sh' +} + +# Installs the CLIs pinned in .devcontainer/mise.toml (tools with no Feature), +# then regenerates shims. MISE_GLOBAL_CONFIG_FILE (devcontainer.json → +# containerEnv) points mise at that manifest. +# +# Globals: +# MISE_GLOBAL_CONFIG_FILE — read, path to the tool manifest +# Outputs: +# Writes progress to stderr via log() +# Returns: +# 0 on success, non-zero on failure +base_install_tools() { + local mise + mise="$(command -v mise || echo "${_MISE_BIN}")" + local config="${MISE_GLOBAL_CONFIG_FILE:-${_LIB_DIR}/../../mise.toml}" + log "Installing pinned CLIs from ${config}..." + "${mise}" trust "${config}" >/dev/null 2>&1 || true + retry 3 5 "${mise}" install + "${mise}" reshim >/dev/null 2>&1 || true +} + +# --- Claude Code --- + +# Installs Claude Code via the native installer if not already present. +# +# MUSHER_INSTALL_CLAUDE=0 skips it. The flag is read here rather than only +# documented: .env.example offers it as the escape hatch for a slow connection, +# and an advertised switch that nothing honours is worse than no switch. +# +# Globals: +# MUSHER_INSTALL_CLAUDE — read, "0" to skip (default: install) +# Outputs: +# Writes progress to stderr via log() +# Returns: +# 0 on success, non-zero on failure +base_install_claude() { + if [[ "${MUSHER_INSTALL_CLAUDE:-1}" == "0" ]]; then + log "MUSHER_INSTALL_CLAUDE=0, skipping Claude Code install" + return 0 + fi + if has_cmd claude; then + log "Claude Code already installed, skipping" + return 0 + fi + log "Installing Claude Code (native installer)..." + retry 3 5 bash -c 'curl -fsSL https://claude.ai/install.sh | bash' +} + +# --- Verify --- + +# Verifies the CLIs this script installs (plus a couple of key Feature tools) +# are on PATH. Runtimes are validated by the container build itself. +# +# actionlint and shellcheck are here because `task check` runs both, and a tool +# that is absent or built for the wrong CPU architecture should fail loudly at +# container build rather than at the first push. +# +# Claude is checked only when it was meant to be installed, so a container +# built with MUSHER_INSTALL_CLAUDE=0 does not fail its own verification. +# +# Globals: +# MUSHER_INSTALL_CLAUDE — read, "0" drops claude from the checked set +# Outputs: +# Writes tool status to stderr via log() +# Returns: +# 0 if all tools found, 1 if any are missing +base_verify_tools() { + local tools=(gh task node npm lefthook actionlint shellcheck) + [[ "${MUSHER_INSTALL_CLAUDE:-1}" == "0" ]] || tools+=(claude) + verify_tools "${tools[@]}" +} + +# --- Orchestrator --- + +# Runs the complete base setup sequence. +# +# Outputs: +# Writes progress to stderr via log() +base_setup() { + log "Running base setup..." + base_setup_config_dirs + base_setup_cache_dirs + base_fix_nvm_permissions + base_setup_path + base_install_mise + base_install_tools + base_install_claude + base_verify_tools + log "Base setup complete" +} diff --git a/.devcontainer/scripts/lib/common.sh b/.devcontainer/scripts/lib/common.sh new file mode 100644 index 0000000..e6bb13c --- /dev/null +++ b/.devcontainer/scripts/lib/common.sh @@ -0,0 +1,176 @@ +#!/usr/bin/env bash +# common.sh — Shared utility functions for dev container setup scripts. +# +# This is a library file meant to be sourced, not executed directly. +# Usage: source "path/to/common.sh" +# +# Provides logging, command helpers, directory setup, NVM/NPM utilities, +# and tool verification functions used by all setup scripts. +set -euo pipefail + +# --- Logging --- + +# Logs a timestamped message to stderr. +# +# Arguments: +# $@ — message text +# Outputs: +# Writes timestamped message to stderr +log() { + echo "[$(date '+%H:%M:%S')] $*" >&2 +} + +# --- Command helpers --- + +# Checks whether a command exists on the PATH. +# +# Arguments: +# $1 — command name +# Returns: +# 0 if found, 1 otherwise +has_cmd() { + command -v "$1" &>/dev/null +} + +# Runs a command with sudo if available, otherwise without. +# +# Arguments: +# $@ — command and arguments +maybe_sudo() { + if sudo -n true 2>/dev/null; then + sudo "$@" + else + "$@" + fi +} + +# Retries a command with exponential back-off. +# +# Arguments: +# $1 — max attempts +# $2 — delay in seconds between attempts +# $@ — command and arguments to execute +# Returns: +# 0 on success, 1 after exhausting all attempts +retry() { + local attempts="${1:?usage: retry }" + local delay="${2:?}" + shift 2 + local attempt=1 + while true; do + if "$@"; then + return 0 + fi + if ((attempt >= attempts)); then + log "FAIL: '$*' failed after ${attempts} attempts" + return 1 + fi + log "Attempt ${attempt}/${attempts} failed, retrying in ${delay}s..." + sleep "$delay" + ((attempt++)) + done +} + +# --- Directory helpers --- + +# Ensures a directory exists and is owned by the given user. +# +# Arguments: +# $1 — directory path +# $2 — owner (default: "vscode") +ensure_writable_dir() { + local dir="${1:?usage: ensure_writable_dir }" + local owner="${2:-vscode}" + if [[ ! -d "$dir" ]]; then + maybe_sudo mkdir -p "$dir" + fi + maybe_sudo chown -R "${owner}:${owner}" "$dir" +} + +# Creates config directories from "label:path" pairs. +# +# Globals: +# REMOTE_USER — read, falls back to "vscode" +# Arguments: +# $@ — entries in "label:path" format +# Outputs: +# Writes progress to stderr via log() +setup_config_dirs() { + local owner="${REMOTE_USER:-vscode}" + for entry in "$@"; do + local label="${entry%%:*}" + local dir="${entry#*:}" + log "Ensuring config dir: ${label} (${dir})" + ensure_writable_dir "$dir" "$owner" + done +} + +# --- NVM helpers --- + +# Fixes NVM directory ownership to the current user. +# +# Globals: +# NVM_DIR — read, defaults to /usr/local/share/nvm +# Outputs: +# Writes progress to stderr via log() +fix_nvm_permissions() { + local nvm_dir="${NVM_DIR:-/usr/local/share/nvm}" + if [[ -d "$nvm_dir" ]]; then + log "Fixing NVM permissions in ${nvm_dir}..." + maybe_sudo chown -R "$(id -un):$(id -gn)" "$nvm_dir" + fi +} + +# --- NPM install helper --- + +# Installs an npm package globally with retry logic. +# +# Arguments: +# $1 — package name +# $2 — max attempts (default: 3) +# Outputs: +# Writes progress to stderr via log() +install_npm_cli() { + local package="${1:?usage: install_npm_cli [attempts]}" + local attempts="${2:-3}" + log "Installing npm CLI: ${package}..." + retry "$attempts" 5 npm install -g "$package" +} + +# --- Verification --- + +# Verifies that a list of commands are available on the PATH and can run. +# +# Being on the PATH is not enough. A release binary fetched for the wrong CPU +# architecture resolves fine and then dies with `exec format error` (exit 126) +# the first time anything invokes it, which is a failure worth catching at +# container build rather than mid-task. Every tool checked here exits 0 on +# `--version`, so a non-zero status means the binary is not runnable. +# +# Arguments: +# $@ — command names to check +# Outputs: +# Writes status of each tool to stderr via log() +# Returns: +# 0 if all tools found and runnable, 1 otherwise +verify_tools() { + log "Verifying installed tools..." + local all_ok=true + local version + for cmd in "$@"; do + if ! has_cmd "$cmd"; then + log " ✗ ${cmd}: MISSING" + all_ok=false + elif ! version="$("$cmd" --version 2>/dev/null)"; then + log " ✗ ${cmd}: on PATH at $(command -v "$cmd") but does not run" + all_ok=false + else + # First line only — shellcheck and actionlint both print a banner, and the + # status line is meant to be one line per tool. Trimmed after the status + # check rather than by piping through `head`, which would report the + # pipeline's exit code instead of the tool's. + log " ✓ ${cmd}: ${version%%$'\n'*}" + fi + done + [[ "${all_ok}" == true ]] +} diff --git a/.devcontainer/scripts/lib/env-check.sh b/.devcontainer/scripts/lib/env-check.sh new file mode 100644 index 0000000..d08d321 --- /dev/null +++ b/.devcontainer/scripts/lib/env-check.sh @@ -0,0 +1,79 @@ +#!/usr/bin/env bash +# env-check.sh — Library: compare a local .env against its template. +# +# This is a library file meant to be sourced, not executed directly. +# Usage: source "path/to/env-check.sh" +# +# Exposes: +# env_check_keys — print KEY names (one per line) to stdout +# env_check_drift — print missing keys; non-zero if any drift +# env_check_required — print keys with empty values to stdout +set -euo pipefail + +# Extracts variable names from a dotenv-style file. +# +# Recognizes lines that start with an uppercase identifier followed by '='. +# Skips blanks, comments (`#`), and commented-out overrides. +# +# A file declaring no keys is a legitimate state, not an error: this +# repository's template is entirely commented defaults by design. grep exits 1 +# when it matches nothing, and `set -o pipefail` above would turn that into a +# failure of the whole check — so the match is captured before the pipeline +# rather than inside it. +# +# Arguments: +# $1 — path to env file +# Outputs: +# One KEY per line on stdout (deduplicated, in file order) +env_check_keys() { + local file="${1:?usage: env_check_keys }" + [[ -f "${file}" ]] || return 0 + + local declarations + declarations="$(grep -E '^[A-Z_][A-Z0-9_]*=' "${file}" || true)" + [[ -n "${declarations}" ]] || return 0 + printf '%s\n' "${declarations}" | sed 's/=.*//' | awk '!seen[$0]++' +} + +# Reports keys present in but missing from . +# +# Arguments: +# $1 — path to .env (the user's local file) +# $2 — path to .env.example (the template) +# Outputs: +# Missing keys, one per line, to stderr +# Returns: +# 0 if no drift, 1 if any keys are missing +env_check_drift() { + local env_file="${1:?usage: env_check_drift }" + local example_file="${2:?}" + [[ -f "${example_file}" ]] || return 0 + + local expected actual missing + expected="$(env_check_keys "${example_file}" | sort -u)" + actual="$(env_check_keys "${env_file}" | sort -u)" + # An empty key set echoes as one blank line, which comm would read as a key + # named "". Dropping blanks keeps an empty set genuinely empty. + missing="$(comm -23 <(echo "${expected}") <(echo "${actual}") | grep -v '^$' || true)" + + if [[ -n "${missing}" ]]; then + echo "${missing}" >&2 + return 1 + fi + return 0 +} + +# Reports keys present in whose value is empty (the "required, please +# fill in" state from the three-state grammar). +# +# Arguments: +# $1 — path to .env +# Outputs: +# Empty-valued keys, one per line, to stdout +# Returns: +# 0 always (informational) +env_check_required() { + local env_file="${1:?usage: env_check_required }" + [[ -f "${env_file}" ]] || return 0 + grep -E '^[A-Z_][A-Z0-9_]*=$' "${env_file}" | sed 's/=$//' || true +} diff --git a/.devcontainer/scripts/lib/motd.sh b/.devcontainer/scripts/lib/motd.sh new file mode 100644 index 0000000..6be4571 --- /dev/null +++ b/.devcontainer/scripts/lib/motd.sh @@ -0,0 +1,188 @@ +#!/usr/bin/env bash +# motd.sh — Renders a startup MOTD summarizing the dev container state. +# +# This is a library file meant to be sourced, not executed directly. +# Requires common.sh (has_cmd, log) to be sourced first. +# +# Usage: source "path/to/motd.sh"; show_motd "/path/to/.devcontainer" +# +# Trimmed from musher-dev/spec's: the services block is gone with the compose +# stack, and the runtime rows list what this container actually installs rather +# than every runtime the template could have installed. + +if [[ -z "${_MOTD_SH_LOADED:-}" ]]; then +readonly _MOTD_SH_LOADED=1 + +# --- Color setup --- + +_motd_setup_colors() { + if [[ -t 1 ]] && has_cmd tput; then + _BOLD="$(tput bold)" + _DIM="$(tput dim)" + _GREEN="$(tput setaf 2)" + _YELLOW="$(tput setaf 3)" + _CYAN="$(tput setaf 6)" + _RESET="$(tput sgr0)" + else + _BOLD="" _DIM="" _GREEN="" _YELLOW="" _CYAN="" _RESET="" + fi +} + +# --- Sub-functions --- + +_motd_header() { + local line + line="$(printf '═%.0s' {1..58})" + echo "${_BOLD}${line}${_RESET}" + echo "${_BOLD} Musher Catalog Dev Container${_RESET}" + echo "${_BOLD}${line}${_RESET}" +} + +# Prints a detected tool version, or a dash if it is not on PATH. +# +# Arguments: +# $1 — command name +# $2 — display label +# $3 — version extraction command (eval'd) +_motd_tool_entry() { + local cmd="$1" label="$2" version_cmd="$3" + # ASCII, not an em dash: printf pads %-16s by bytes, and a 3-byte dash + # displaying as one character knocks the second column out by two. + local ver="-" color="${_YELLOW}" + if has_cmd "$cmd"; then + ver="$(eval "$version_cmd" 2>/dev/null || echo '?')" + color="${_GREEN}" + fi + printf " ${_CYAN}%-11s${_RESET} ${color}%-16s${_RESET}" "$label" "$ver" +} + +_motd_tools() { + local sep + sep="$(printf '─%.0s' {1..54})" + echo "" + echo " ${_BOLD}Tooling${_RESET}" + echo " ${_DIM}${sep}${_RESET}" + + _motd_tool_entry node "node" "node -v" + _motd_tool_entry npm "npm" "npm -v" + echo "" + + _motd_tool_entry task "task" "task --version | grep -oE '[0-9]+\\.[0-9]+\\.[0-9]+'" + _motd_tool_entry gh "gh" "gh --version | head -1 | grep -oE '[0-9]+\\.[0-9]+\\.[0-9]+'" + echo "" + + _motd_tool_entry shellcheck "shellcheck" "shellcheck --version | grep -oE '[0-9]+\\.[0-9]+\\.[0-9]+' | head -1" + _motd_tool_entry actionlint "actionlint" "actionlint --version | head -1" + echo "" + + _motd_tool_entry lefthook "lefthook" "lefthook version" + _motd_tool_entry claude "claude" "claude --version | grep -oE '[0-9]+\\.[0-9]+\\.[0-9]+' | head -1" + echo "" +} + +# Names where the test suite reads the contract from. Constant by design — the +# line is here so it is never a question what a run was judged against. +_motd_spec_source() { + local sep + sep="$(printf '─%.0s' {1..54})" + echo "" + echo " ${_BOLD}Validating against${_RESET}" + echo " ${_DIM}${sep}${_RESET}" + echo " musher-dev/spec @ main" + echo " ${_DIM}public repo, fetched at run time; no token, no cache, no fallback${_RESET}" +} + +_motd_quickref() { + local sep + sep="$(printf '─%.0s' {1..54})" + echo "" + echo " ${_BOLD}Quick Reference${_RESET}" + echo " ${_DIM}${sep}${_RESET}" + echo " task List tasks" + echo " task check Everything CI runs" + echo " task test Validate the catalog corpus" + echo " task test:item -- Validate one item" + echo " claude Claude Code" +} + +# Warns when .env is missing keys from .env.example, or has empty +# required values. Silent when env is healthy. +# +# Globals: +# _BOLD, _DIM, _YELLOW, _RESET — color codes set by _motd_setup_colors +# Arguments: +# $1 — .devcontainer directory (where .env / .env.example live) +_motd_env_warnings() { + local devcontainer_dir="${1:-}" + [[ -d "${devcontainer_dir}" ]] || return 0 + + local env_file="${devcontainer_dir}/.env" + local example_file="${devcontainer_dir}/.env.example" + local lib_file="${devcontainer_dir}/scripts/lib/env-check.sh" + [[ -f "${lib_file}" ]] || return 0 + + # shellcheck source=./env-check.sh + source "${lib_file}" + + local missing="" required="" + if [[ -f "${example_file}" ]]; then + missing="$(env_check_drift "${env_file}" "${example_file}" 2>&1 || true)" + fi + required="$(env_check_required "${env_file}" 2>/dev/null || true)" + + if [[ -z "${missing}" && -z "${required}" ]]; then + return 0 + fi + + local sep + sep="$(printf '─%.0s' {1..54})" + echo "" + echo " ${_BOLD}${_YELLOW}Environment${_RESET}" + echo " ${_DIM}${sep}${_RESET}" + if [[ -n "${missing}" ]]; then + echo " ${_YELLOW}Missing keys in .env (run 'task env:reset' to sync):${_RESET}" + awk '{print " - " $0}' <<< "${missing}" + fi + if [[ -n "${required}" ]]; then + echo " ${_YELLOW}Required keys with empty values:${_RESET}" + awk '{print " - " $0}' <<< "${required}" + fi +} + +_motd_tips() { + local sep + sep="$(printf '─%.0s' {1..54})" + echo "" + echo " ${_BOLD}Tips${_RESET}" + echo " ${_DIM}${sep}${_RESET}" + echo " * Add an item: README.md (Adding an item)" + echo " * How validation works: tests/README.md" + echo " * Item contracts: README.md (Item contracts)" + echo " * Tool versions: .devcontainer/{devcontainer.json,mise.toml}" +} + +# Renders the full MOTD to stdout. +# +# Arguments: +# $1 — path to .devcontainer/ directory (may be empty to skip env warnings) +# Outputs: +# MOTD text to stdout +show_motd() { + local devcontainer_dir="${1:-}" + _motd_setup_colors + + local border + border="$(printf '═%.0s' {1..58})" + + echo "" + _motd_header + _motd_tools + _motd_spec_source + _motd_env_warnings "$devcontainer_dir" + _motd_quickref + _motd_tips + echo "${_BOLD}${border}${_RESET}" + echo "" +} + +fi diff --git a/.devcontainer/scripts/post-create.sh b/.devcontainer/scripts/post-create.sh new file mode 100755 index 0000000..67367ac --- /dev/null +++ b/.devcontainer/scripts/post-create.sh @@ -0,0 +1,81 @@ +#!/usr/bin/env bash +# post-create.sh — DevContainer post-create command hook. +# +# Runs once after the container is created. Sets up config and cache +# directories, installs the pinned CLIs and Claude Code, then installs this +# repository's own dependencies. +# +# Usage: Called automatically by devcontainer.json postCreateCommand. +set -euo pipefail + +SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" +readonly SCRIPT_DIR +REPO_ROOT="$(cd "${SCRIPT_DIR}/../.." && pwd)" +readonly REPO_ROOT + +# shellcheck source=lib/common.sh +source "${SCRIPT_DIR}/lib/common.sh" +# shellcheck source=lib/base-setup.sh +source "${SCRIPT_DIR}/lib/base-setup.sh" + +# Logs the failing command and line number on ERR. +# +# Arguments: +# $1 — line number +# $2 — failed command string +# Outputs: +# Writes error details to stderr via log() +on_error() { + local line="${1}" + local cmd="${2}" + log "ERROR: command '${cmd}' failed at line ${line}" +} +trap 'on_error ${LINENO} "${BASH_COMMAND}"' ERR + +# Installs lefthook git hooks for this repo. Best-effort: silently +# skips if lefthook isn't on PATH yet or no .config/lefthook.yml exists. +# +# Outputs: +# Writes progress to stderr via log() +install_lefthook_hooks() { + command -v lefthook >/dev/null 2>&1 || return 0 + [[ -f "${REPO_ROOT}/.config/lefthook.yml" ]] || return 0 + log "Installing lefthook git hooks..." + (cd "${REPO_ROOT}" && lefthook install >/dev/null 2>&1) || true +} + +# Installs the validation suite's dependencies. +# +# Best-effort: a failure here leaves the container usable, and `task setup` +# recovers. `npm ci` rather than `npm install` so the lockfile decides — a +# container that silently resolved different versions than CI is a container +# that reproduces nothing. +# +# Outputs: +# Writes progress to stderr via log() +install_test_dependencies() { + command -v npm >/dev/null 2>&1 || return 0 + [[ -f "${REPO_ROOT}/package.json" ]] || return 0 + log "Installing validation dependencies (npm ci)..." + (cd "${REPO_ROOT}" && npm ci) || { + log "WARNING: npm ci failed; run 'task setup' once the container is up" + return 0 + } +} + +# Entry point: runs the full post-create setup sequence. +# +# Arguments: +# $@ — passed through (unused, reserved for future use) +# Outputs: +# Writes progress to stderr via log() +main() { + log "Starting post-create setup..." + base_setup + install_lefthook_hooks + # --- Repo-specific setup --- + install_test_dependencies + log "Post-create setup completed" +} + +main "$@" diff --git a/.devcontainer/scripts/startup.sh b/.devcontainer/scripts/startup.sh new file mode 100755 index 0000000..c7a7b7f --- /dev/null +++ b/.devcontainer/scripts/startup.sh @@ -0,0 +1,45 @@ +#!/usr/bin/env bash +# startup.sh — Runs on every container start. +# +# The template's version brings up a compose stack and waits for health checks. +# This repository has no services to start — it holds YAML and validates it — +# so all that remains is the MOTD. The file is kept rather than dropped so the +# postStartCommand hook has somewhere to grow, and so the lifecycle matches +# musher-dev/spec's. +# +# Usage: Called automatically by devcontainer.json postStartCommand. +set -euo pipefail + +SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" +readonly SCRIPT_DIR +DEVCONTAINER_DIR="$(cd "${SCRIPT_DIR}/.." && pwd)" +readonly DEVCONTAINER_DIR + +# shellcheck source=lib/common.sh +source "${SCRIPT_DIR}/lib/common.sh" +# shellcheck source=lib/motd.sh +source "${SCRIPT_DIR}/lib/motd.sh" + +# Logs the failing command and line number on ERR. +# +# Arguments: +# $1 — line number +# $2 — failed command string +# Outputs: +# Writes error details to stderr via log() +on_error() { + local line="${1}" + local cmd="${2}" + log "ERROR: command '${cmd}' failed at line ${line}" +} +trap 'on_error ${LINENO} "${BASH_COMMAND}"' ERR + +# Entry point. +# +# Outputs: +# MOTD to stdout +main() { + show_motd "${DEVCONTAINER_DIR}" +} + +main "$@" diff --git a/Taskfile.yml b/Taskfile.yml new file mode 100644 index 0000000..bfade3f --- /dev/null +++ b/Taskfile.yml @@ -0,0 +1,141 @@ +# Musher Catalog Taskfile +# Usage: task [task-name] +# List tasks: task --list +# +# This file stays at the repo root because Task only discovers `Taskfile.*` +# there: `--taskfile` would work but would break bare `task `. +# +# musher-dev/spec splits its tasks across taskfiles/. This repository has one +# build surface — validate the corpus — so splitting would cost a reader two +# files to answer one question. Split it when there is a second surface. +# +# The tasks wrap npm scripts rather than replacing them. `npm test` stays the +# thing CI runs and the thing that works in a bare checkout with no Task +# installed; `task` is the ergonomics layer the dev container advertises. + +version: '3' + +vars: + SHELL_SOURCES: .devcontainer/scripts + ENV_FILE: .devcontainer/.env + ENV_EXAMPLE: .devcontainer/.env.example + +tasks: + default: + desc: List available tasks + cmds: + - task --list + silent: true + + # =========================================================================== + # Entry points + # =========================================================================== + + setup: + desc: Install dependencies and git hooks + cmds: + - task: setup:deps + - task: setup:hooks + + check: + desc: Run every check CI runs + cmds: + - task: check:types + - task: check:catalog + - task: check:shell + - task: check:workflow + + # =========================================================================== + # Validation + # =========================================================================== + + test: + desc: Validate the catalog corpus against musher-dev/spec + aliases: [check:catalog] + cmds: + - npm test + + test:item: + desc: Validate one item — task test:item -- redis + summary: | + Runs only the suites for a single item. + + Every per-item suite is named `items/`, so the runner's name filter + is anchored: an unanchored `redis` would also pull in a future + `redis-stack`. + + Usage: task test:item -- + cmds: + - >- + node --disable-warning=ExperimentalWarning --test + --test-name-pattern='^items/{{.CLI_ARGS}}$' + "tests/**/*.test.ts" + + typecheck: + desc: Typecheck the test suite + aliases: [check:types] + cmds: + - npm run typecheck + + # =========================================================================== + # Linting + # + # Both linters are pinned in .devcontainer/mise.toml (actionlint) or as a + # devcontainer Feature (shellcheck), so a container has them and a bare + # checkout may not. Each skips with a note rather than failing the run. + # =========================================================================== + + check:shell: + desc: Lint the dev container scripts + status: + - '! command -v shellcheck' + cmds: + # -P makes shellcheck resolve `# shellcheck source=` directives from the + # script directories rather than from wherever it was invoked, which is + # the whole difference between following a sourced library and reporting + # SC1091 against every one of them. + - >- + shellcheck -x -S style + -P {{.SHELL_SOURCES}}:{{.SHELL_SOURCES}}/lib + {{.SHELL_SOURCES}}/*.sh {{.SHELL_SOURCES}}/lib/*.sh + + check:workflow: + desc: Lint the GitHub Actions workflows + status: + - '! command -v actionlint' + cmds: + - actionlint + + # =========================================================================== + # Setup internals + # =========================================================================== + + setup:deps: + desc: Install npm dependencies from the lockfile + cmds: + # `ci` rather than `install`: the lockfile decides. An environment that + # silently resolved different versions than CI reproduces nothing. + - npm ci + + setup:hooks: + desc: Install lefthook git hooks + status: + - '! command -v lefthook' + cmds: + - lefthook install + + # =========================================================================== + # Dev container environment + # =========================================================================== + + env:check: + desc: Report keys in .env.example missing from your .env + cmds: + - bash {{.SHELL_SOURCES}}/env-check.sh + + env:reset: + desc: Recreate .devcontainer/.env from the template + prompt: This overwrites {{.ENV_FILE}}. Continue? + cmds: + - cp {{.ENV_EXAMPLE}} {{.ENV_FILE}} + - 'echo "env: {{.ENV_FILE}} recreated from {{.ENV_EXAMPLE}}"' From c7bd394432e0dc41144684b819293e87d66d9229 Mon Sep 17 00:00:00 2001 From: Ali S Date: Tue, 25 Aug 2026 12:33:16 +0000 Subject: [PATCH 5/5] docs: describe the local suite, and correct two contract details MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The validation section said this repository runs no validation of its own and that a mistake is not caught at commit time. Both stopped being true; it now says what runs, what is fetched at run time and why, and which phase is deliberately missing. The platform is still named as the sole authority — the tests are the same contracts applied early, not a second set — and the one-item-per-PR guidance stays, since that is what makes a rejection only the platform can raise easy to attribute. Two details the suite forced into the open while it was being written: blueprint component references are repo-local paths that must begin ./ and end .yaml, not bare stems. The prefix is load-bearing — a bare name is not distinguishable from the UUID a published reference uses, so without it no validator can tell which resolver the reference wanted. The example said `component: my-app`, which the structural phase rejects. A connection's two ends must agree on schema.type, and on semanticType wherever the consuming input names one. The multi-service paragraph said only that connections wire outputs to inputs, which is the shape of the thing and not the constraint on it. Co-Authored-By: Claude Opus 5 (1M context) --- README.md | 43 ++++++++++++++++++++++++++++++++----------- 1 file changed, 32 insertions(+), 11 deletions(-) diff --git a/README.md b/README.md index ac7095c..2ff650d 100644 --- a/README.md +++ b/README.md @@ -89,8 +89,8 @@ spec: outputs: {} ``` -**`blueprint.yaml`** — references the component file by stem and binds compute -per node: +**`blueprint.yaml`** — references the component file by repo-local path and +binds compute per node: ```yaml specVersion: v1 @@ -99,12 +99,16 @@ metadata: { slug: my-app, version: 1 } spec: components: web: # graph-local node name (map order = graph order) - component: my-app # components/my-app.yaml - size: general.standard.small # binding Compute Profile + component: ./components/my-app.yaml # must begin ./ and end .yaml + size: general.standard.small # binding Compute Profile connections: {} # inbound wires, keyed by consumer input parameters: {} # empty ⇒ derived from merged USER inputs ``` +The `./` prefix is load-bearing, not decorative: a bare name is not +distinguishable from the UUID a published reference uses, so without it no +validator could tell which resolver the reference wanted. + **`listing.yaml`** ```yaml @@ -131,18 +135,35 @@ spec: A multi-service item adds more entries under `spec.components` — unique node names, one `components/.yaml` per reference — and wires `connections` -between declared component outputs and inputs. A `COMPONENT`-kind listing still +between declared component outputs and inputs. A wire's two ends must agree on +`schema.type`, and on `schema.semanticType` wherever the consuming input names +one. A `COMPONENT`-kind listing still authors a trivial single-node `blueprint.yaml` wrapping its one component. ## Validation -This repository runs **no validation of its own**. The Musher platform is the -sole validator: item contracts are enforced when the platform syncs this repo, -and a malformed item is rejected there. +```sh +npm install +npm test +``` -The practical consequence is that a mistake here is not caught at commit time. -Review carefully, and keep changes to one item per pull request so a rejection -is easy to attribute. +Every item is validated against the schemas at the tip of the public +[`musher-dev/spec`](https://github.com/musher-dev/spec) repository, **fetched at +run time rather than vendored** — so what the corpus is judged against is the +contract as it currently stands, not a copy of it that has quietly fallen +behind. The repository is public, so no credential is involved. The suite +covers the three phases a client can decide offline: the YAML profile, the JSON +Schema bundles, and the semantic rules that bind an item's documents to each +other and to its directory. See [`tests/README.md`](tests/README.md). + +The Musher platform remains the **sole authority**. These tests are the same +contracts applied early, not a second one: they run the phases that need no +network, and they cannot see the `capability` phase at all — whether a Compute +Profile is actually offered, whether a published component exists, whether a +version is monotonic. An item that passes here can still be rejected at sync. + +Keep changes to one item per pull request, so a rejection that only the platform +can raise is easy to attribute. ## Contributing