diff --git a/.ai/BRIEF.md b/.ai/BRIEF.md deleted file mode 100644 index 1841c79b..00000000 --- a/.ai/BRIEF.md +++ /dev/null @@ -1,24 +0,0 @@ -# Brief - atlas - -## Story - -Atlas is a unified AI SDK for Laravel — one fluent, provider-agnostic API for text, image, audio, video, music, speech, embeddings, moderation, rerank, agents, voice, and batch generation. It exists so PHP developers can build AI-enabled products against a single consistent surface instead of stitching together per-vendor SDKs and payload formats. In its v3 line it owns its entire provider layer, talking directly to vendor HTTP APIs, so the outcome is a self-contained package that swaps providers, models, and modalities without touching application code. - -## Users / ICP - -- PHP and Laravel developers building AI features who want one API across many providers and modalities. -- They accomplish generation, embeddings/similarity search, tool-using agents, real-time voice, and cost-saving batch jobs without vendor lock-in. -- Qualities that matter: provider-agnostic and swappable, self-contained (no hard app dependency), framework-aware, deterministic and testable, with persistence/audit trails when needed. - -## Scope - -- **Active areas:** the `atlas` package — provider layer (OpenAI, Anthropic, Google, xAI, ElevenLabs, Cohere, Jina, OpenAI-compatible), all generation modalities, agents/executor/tool loop, embeddings and chunking, persistence, voice, batch, and the VitePress docs site. -- **Out of scope:** any other Atlas repo (reference only, unless a task explicitly names it); the upstream Prism repo, kept only as a temporary working copy for upstream PRs and never Atlas's dependency. - -## External Systems - -- `OpenAI`, `Anthropic`, `Google Gemini`, `xAI` — text, image, audio, video, embedding, moderation, and batch generation. -- `ElevenLabs` — voice, music, and sound-effect audio generation. -- `Cohere`, `Jina` — document reranking. -- `OpenAI-compatible endpoints (Ollama, LM Studio)` — self-hosted / local model access via the shared ChatCompletions and Responses drivers. -- `atlasphp.org` — the public documentation site built from `docs/`. diff --git a/.ai/CODEMAP.md b/.ai/CODEMAP.md deleted file mode 100644 index 88dd2a51..00000000 --- a/.ai/CODEMAP.md +++ /dev/null @@ -1,175 +0,0 @@ -# Codemap - atlas - -> As of 2026-06-30, branch `3.x`, HEAD `3a22f27`, CHANGELOG v3.6.0. Re-verify counts against the current branch HEAD if this date is stale. - -`Atlasphp\Atlas` — unified AI SDK for Laravel. PHP 8.2+, Laravel 11/12/13 package. Owns its own provider layer (no external AI SDK dependency). Integrated providers: OpenAI, Anthropic, Google Gemini, xAI, ElevenLabs (voice/audio), Cohere (rerank), Jina (rerank); the shared `ChatCompletions/` driver covers OpenAI-compatible endpoints (Ollama, LM Studio). - -## Root (src/) - -- **Atlas** — the facade (`class Atlas extends Facade`, namespace `Atlasphp\Atlas`; carries the `@method static` map). No `src/Facades/` dir exists. -- **AtlasManager** — manager/orchestrator (facade accessor) -- **AtlasServiceProvider** — bootstrap, voice route registration, agent + chunkable auto-discovery, config + migrations publish -- **AtlasConfig**, **RequestConfig** — config DTOs -- **AtlasCache** — model/voice/embedding cache -- **Agent**, **AgentRegistry** — agent definition + registry - -## Facade methods - -`Atlas::` — `text()`, `image()`, `audio()`, `music()`, `sfx()`, `speech()`, `video()`, `embed()`, `moderate()`, `voice()`, `rerank()`, `batch()`, `batchGroup()`, `provider()`, `agent()`, `providers()`, `registerChunkable()`, `chunkables()`, `similaritySearch()`, `fake()` - -## Enums (src/Enums/ — 11) - -BatchResultStatus, BatchStatus, ChunkType, FinishReason, Modality, Provider, ReasoningEffort (Minimal/Low/Medium/High), Role, ToolChoiceMode, TurnDetectionMode, VoiceTransport - -## Messages (src/Messages/ — 6) - -Message (base), UserMessage, AssistantMessage, SystemMessage, ToolCall, ToolResultMessage - -## Pending Builders (src/Pending/ — 15) - -Fluent builders returned by the facade. - -- **Request types:** TextRequest, ImageRequest, AudioRequest, VideoRequest, SpeechRequest, MusicRequest, SfxRequest, EmbedRequest, ModerateRequest, RerankRequest, VoiceRequest, AgentRequest, BatchRequest (`add()`/`addMany()`/`group()`/`completionWindow()`/`submit()`), GenerativeAudioRequest, ProviderRequest (base) -- **Concerns/ (9):** ConvertsResultToChunks, HasMeta, HasMiddleware, HasProviderOptions, HasQueueDispatch, HasRequestConfig, HasVariables, NormalizesMessages, ResolvesProvider -- **Contracts/:** Batchable (the contract BatchRequest accepts) - -## Requests — DTOs (src/Requests/ — 11) - -Immutable request DTOs (class names mirror the `Pending/` builders): AudioRequest, Batch (+ BatchLine), EmbedRequest, ImageRequest, ModerateRequest, Reasoning (`budgetTokens()`; threaded into TextRequest as `?Reasoning $reasoning`), RerankRequest, TextRequest, VideoRequest, VoiceRequest - -## Responses (src/Responses/ — 18) - -- TextResponse, StreamResponse, StructuredResponse, ImageResponse, AudioResponse, VideoResponse, RerankResponse (+ RerankResult), EmbeddingsResponse, ModerationResponse, BatchResponse (+ BatchResult, RequestCounts), Usage, TokenCount (pre-flight input-token count from `->countTokens()`), VoiceSession, VoiceEvent, StreamChunk -- **Contracts/** — Storable (interface) - -## Executor (src/Executor/ — 7) - -Tool loop + step orchestration: AgentExecutor, ToolExecutor, ToolRegistry, ExecutionContext, Step, ExecutorResult, ToolResult - -## Providers (src/Providers/) - -Provider layer (driver → handlers + resolvers). - -- **Core:** Driver, ResponsesDriver (neutral OpenAI-Responses-API driver — Responses text handler with no org header + shared image/audio/video/embed/moderate handlers; for Ollama and other Responses-API proxies), ProviderRegistry, ProviderConfig, ProviderCapabilities, ModelList, VoiceList, WebSocketConnection, SseParser (HttpClient + RetryDecider live in `src/Http/`) -- **Responses/** — shared OpenAI Responses API resolver set: Handlers/Text, MediaResolver, ResponseParser, ToolMapper. Composed by both `OpenAiDriver` and `ResponsesDriver`, so `OpenAi/` carries no own MediaResolver/ResponseParser/ToolMapper. -- **Handlers/ (12)** — modality handler interfaces/abstracts: AbstractProviderHandler, AbstractRerankHandler, ProviderHandler, TextHandler, ImageHandler, AudioHandler, VideoHandler, ModerateHandler, EmbedHandler, RerankHandler, VoiceHandler, BatchHandler -- **Contracts/ (5)** — resolver seams: MessageFactoryContract, ResponseParserContract, ToolMapperContract, MediaResolverContract, ProviderRegistryContract -- **Concerns/ (7)** — shared provider traits: AppliesToolChoice, BuildsHeaders, BuildsResponsesMessages, BuildsVoiceBody, CountsTokens (heuristic estimate for providers without a native count endpoint), ResolvesAudioFile, ResolvesMediaUri -- **Tools/ (9)** — provider-native tools: ProviderTool (base), ProviderToolRegistry, CodeExecution, CodeInterpreter, FileSearch, GoogleSearch, WebFetch, WebSearch, XSearch -- **Per-vendor:** - - **OpenAi/** — OpenAiDriver (extends Driver, Responses API); MessageFactory; Concerns/HasOrganizationHeader; Handlers: Audio, Batch, Embed, Image, Moderate, Provider, Text, Video, Voice (resolvers from shared `Responses/`) - - **Anthropic/** — AnthropicDriver; MediaResolver, MessageFactory, ResponseParser, ToolMapper; Concerns/BuildsAnthropicHeaders; Handlers: Batch, Provider, Text - - **Google/** — GoogleDriver, GoogleToolCall; MediaResolver, MessageFactory, ResponseParser, ToolMapper; Concerns/BuildsGoogleHeaders; Handlers: Batch, Embed, Image, Provider, Text - - **Xai/** — XaiDriver, MessageFactory, ResponseParser, ToolMapper; Handlers: Audio, Image, Provider, Text, Video, Voice - - **ChatCompletions/** — ChatCompletionsDriver; MediaResolver, MessageFactory, ResponseParser, ToolMapper; Handlers: Provider, Text (OpenAI-compatible: Ollama, LM Studio) - - **ElevenLabs/** — ElevenLabsDriver; Concerns/BuildsElevenLabsHeaders; Handlers: Audio, Music, Provider, Sfx, Voice - - **Cohere/** — CohereDriver, CohereRerankHandler - - **Jina/** — JinaDriver, JinaRerankHandler - -## Persistence (src/Persistence/) - -- Root: ProcessQueuedMessage, ToolAssets -- **Models/ (12)** — Asset, BatchGroup, BatchJob (status/counts/usage + `open()` scope + `applyStatus()`/`markCompleted()`/`markFailed()`), BatchResult (per-line, unique `(batch_job_id, custom_id)`), Chunk, Conversation, ConversationMessage, ConversationMessageAsset, Execution (sub-agent lineage: `parent_execution_id`/`parent_tool_call_id`/`depth` + `parent()`/`children()`/`parentToolCall()` relations + `totalUsage()` subtree roll-up), ExecutionStep, ExecutionToolCall, VoiceCall -- **Services/ (5)** — ConversationService, ExecutionService, ChunkContentService, ChunkSearchService, RecordSearchService -- **Middleware/ (5)** — PersistConversation, TrackExecution, TrackProviderCall, TrackStep, TrackToolCall -- **Concerns/ (7)** — consumer-app model traits: HasAtlasTable, HasChunkedEmbeddings, HasConversations, HasExecutionStatus, HasOwner, HasVectorEmbeddings, ResolvesChunkModel -- **Enums/ (7)** — AssetType, ExecutionStatus, ExecutionType, MessageRole, MessageStatus, ToolCallType, VoiceCallStatus -- **Schema/** — ChunkedEmbeddingColumns -- **Support/** — MimeTypeMap -- **Http/** — StoreVoiceTranscriptController - -## Embeddings (src/Embeddings/ — 7) - -- EmbeddingResolver, VectorQueryMacros, Chunkable, ChunkableRegistry, ChunkData, SearchResult, VectorEmbeddable -- **Chunkers/ (3)** — Chunker (base), BaseTokenAwareChunker, MarkdownChunker - -## Voice (src/Voice/Http/ — 2) - -VoiceToolController, CloseVoiceSessionController. VoiceSession + VoiceEvent live under `Responses/`; the `BuildsVoiceBody` trait (session body + `https:` → `wss:` URL rewrite) is in `Providers/Concerns/`, shared by OpenAI and xAI voice handlers. - -## Queue (src/Queue/) - -- PendingExecution -- **Contracts/** — QueueableRequest -- **Jobs/ (3)** — ExecuteAtlasJob, ChunkContentJob, TracksExecution - -## Middleware (src/Middleware/ — 6) - -ProviderContext, ToolContext, StepContext, AgentContext, MiddlewareStack, MiddlewareResolver - -- **Contracts/ (11)** — marker interfaces routed by type: AgentMiddleware, AudioMiddleware, EmbedMiddleware, ImageMiddleware, ProviderMiddleware, StepMiddleware, TextMiddleware, ToolMiddleware, VideoMiddleware, VoiceMiddleware, VoiceHttpMiddleware - -## Tools — infrastructure (src/Tools/ — 6) - -Tool, ToolDefinition, ToolSerializer, ToolChoice, SimilaritySearch, AgentTool (wraps an Agent as a sub-agent delegation tool; `Tool::isDelegation()` flag) - -## Schema (src/Schema/ — 3) - -- SchemaBuilder, Schema, StrictSchema (normalizes a JSON Schema to OpenAI strict structured-output form: recursive `additionalProperties:false` + all-required, optionals → nullable) -- **Fields/ (9)** — Field (base), StringField, IntegerField, NumberField, BooleanField, ArrayField, ObjectField, ObjectFieldBuilder, EnumField - -## Events (src/Events/ — 40, + 3 Concerns/) - -- **Agent:** AgentStarted, AgentStepStarted, AgentStepCompleted, AgentCompleted, AgentMaxStepsExceeded, AgentToolCallStarted, AgentToolCallCompleted, AgentToolCallFailed -- **Execution:** ExecutionEvent (base), ExecutionQueued, ExecutionProcessing, ExecutionCompleted, ExecutionFailed -- **Modality:** ModalityStarted, ModalityCompleted -- **Provider:** ProviderRequestStarted, ProviderRequestCompleted, ProviderRequestFailed, ProviderRequestRetrying -- **Streaming:** StreamStarted, StreamCompleted, StreamChunkReceived, StreamThinkingReceived, StreamToolCallReceived -- **Voice:** VoiceSessionCreated, VoiceSessionEnded, VoiceCallStarted, VoiceCallCompleted, VoiceToolCallStarted, VoiceToolCallCompleted, VoiceToolCallFailed, VoiceAudioDeltaReceived, VoiceTranscriptDeltaReceived -- **Chunking/Conversation:** ContentChunked, ContentChunkingFailed, ConversationMessageStored -- **Batch:** BatchSubmitted, BatchCompleted, BatchFailed, BatchGroupCompleted -- **Concerns/ (3):** BroadcastsOnChannel, BroadcastsOnOptionalChannel, CapsBroadcastPayload (configurable cap for broadcasted tool payloads) - -## Exceptions (src/Exceptions/ — 17) - -AtlasException, ProviderException, AuthenticationException, AuthorizationException, RateLimitException, ProviderNotFoundException, AgentNotFoundException, ToolNotFoundException, MaxStepsExceededException, UnsupportedFeatureException, MaxDelegationDepthException, DelegationCycleException, BatchException, ConnectionException, InvalidRequestException, ModelNotFoundException, ServerException - -## Misc - -- `src/Input/ (5)` — Input (base), Image, Audio, Video, Document -- `src/Console/ (9)` — MakeAgentCommand, MakeToolCommand, MiddlewareCommand, CleanStaleVoiceSessionsCommand, ChunkCommand, RechunkCommand, PruneChunksCommand, PollBatchJobsCommand, PruneBatchJobsCommand -- `src/Batch/` — BatchService (domain orchestration: `submitAndTrack()` + `syncFromProvider()` hydration in a transaction, shared by the poll command). Batch = deferred provider jobs at ~50% cost; OpenAi (text/embed) + Anthropic (text) + Google (text). `Pending\BatchRequest::submit()` tracks a BatchJob when persistence is on, else returns a stateless `BatchResponse`. -- `src/Support/ (3)` — VariableRegistry, VariableInterpolator, TokenCounter (pure utilities) -- `src/Concerns/ (2)` — StoresMedia, ResolvesDatabaseScope (multi-tenant DB scoping for unique job locks) -- `src/Http/ (3)` — HttpClient, RetryDecider, ProviderRequestContext (shared transport; context carries provider/model + a stamped correlation id onto `ProviderRequest*` events) - -## Testing Fakes (src/Testing/ — 13) - -- AtlasFake, FakeDriver, RecordedRequest -- Response fakes: TextResponseFake, ImageResponseFake, AudioResponseFake, VideoResponseFake, EmbeddingsResponseFake, ModerationResponseFake, RerankResponseFake, StreamResponseFake, StructuredResponseFake, VoiceSessionFake - -## Config - -`config/atlas.php` — keys: `defaults`, `agents`, `prompt_cache`, `providers`, `retry`, `queue`, `batch`, `stream`, `broadcast`, `middleware`, `variables`, `storage`, `embeddings`, `cache`, `persistence` (`auto_store_assets`, `message_limit`, `connection`, `table_prefix`, custom model bindings), `voice` (`route_prefix`, `route_middleware`, `session_ttl`) - -## Routes - -Registered programmatically by `AtlasServiceProvider::registerVoiceRoutes()` in `boot()`. Session-scoped paths: - -- `POST {prefix}/voice/{sessionId}/tool` → VoiceToolController -- `POST {prefix}/voice/{sessionId}/close` → CloseVoiceSessionController -- `POST {prefix}/voice/{sessionId}/transcript` → StoreVoiceTranscriptController - -Prefix from `voice.route_prefix` (default `atlas`); applies `voice.route_middleware` + `MiddlewareResolver::forVoiceHttp()`. - -## Tests - -`tests/` — Pest. **Feature:** Console, Persistence, Testing, Variables, Voice. **Unit:** mirrors `src/` (per-domain dirs incl. Embeddings/Chunkers, Persistence/* subdirs, Providers per-vendor + Concerns/Tools/Responses). **Fixtures/**. - -## Sandbox - -`sandbox/` — real-API test harness (Laravel app shell, `bootstrap.php`); Horizon must be running for queue-backed features. - -- **Provider smoke tests:** `test-{openai,anthropic,google,xai,elevenlabs}-provider.php`, `test-custom-driver.php` (Ollama), `test-lmstudio-provider.php` -- **Feature tests:** `test-{agent,subagents,subagents-concurrent,conversation,middleware,streaming,tools,provider-tools-live,voice,prompt-caching,vision-replay,media-config,multitenant-job-locks,queued-message-dispatch}.php`; live/coverage: `test-{error-context-live,force-tools-live,provider-tools-coverage-live,token-counting}.php`; `seed-demo.php` -- **Reasoning:** `test-reasoning{,-forced-tools,-recording,-tools}.php` -- **Batch:** `test-batch{,-demo,-modes,-tables,-google,-google-e2e}.php` -- **Embeddings:** `test-chunked-embeddings{,-dispatch-on-save,-edge-cases}.php`, `test-embeddings-full-suite.php`, `test-record-embeddings.php` - -## Documentation - -`docs/` — VitePress site (atlasphp.org). Sections: getting-started, modalities/capabilities, features, guides, advanced. - -## Composer Scripts - -`composer check` (lint:test + analyse + test) · `lint`/`lint:test` (Pint) · `analyse` (PHPStan) · `test` (Pest) diff --git a/.ai/MEMORY.md b/.ai/MEMORY.md deleted file mode 100644 index abb32d53..00000000 --- a/.ai/MEMORY.md +++ /dev/null @@ -1,14 +0,0 @@ -# Memory - atlas - -## Lessons - -- The real facade method set lives in `src/Atlas.php` (the `@method static` map) — read it before naming any `Atlas::` call in docs or examples; do not invent fluent methods from memory, they drift from the actual surface. - -## Preferences - -- An Atlas feature isn't "done" on fakes alone: it needs full unit coverage, a runnable live real-provider script modeled on `sandbox/test-*-provider.php`, and a reported pass count. If it persists data, the lineage/audit trail must be queryable and demonstrably reconstructable. - -## Known Traps / Gotchas - -- `FakeDriver` skips the base `Driver` constructor, so `Driver::$config` is never set — any base method that reads `$config` will hit uninitialized state. Override such methods in the fake (e.g. `providerName()` is overridden to return `name()`) so base paths like `batch()` stay safe. -- Gemini can emit streamed tool calls one per chunk, so parsers must still handle bundled parallel calls defensively. Raw driver streams can expose finish-reason edge cases that the agent executor loop otherwise hides — test at the driver level, not only through the executor. diff --git a/.github/workflows/doc-lint.yml b/.github/workflows/doc-lint.yml new file mode 100644 index 00000000..e87a2954 --- /dev/null +++ b/.github/workflows/doc-lint.yml @@ -0,0 +1,11 @@ +name: Knowledge Lint +on: [push, pull_request] +jobs: + doc-lint: + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v4 + - uses: actions/setup-python@v5 + with: { python-version: '3.x' } + - run: python3 .knowledge/scripts/test_doc_lint.py + - run: python3 .knowledge/scripts/doc-lint .knowledge diff --git a/.knowledge/.payload-manifest b/.knowledge/.payload-manifest new file mode 100644 index 00000000..79efee46 --- /dev/null +++ b/.knowledge/.payload-manifest @@ -0,0 +1,12 @@ +# payload integrity — sha256 of every file knowledge-template versions. +# The adopted version is stamped in `.version`; these lines prove the files match it. +# Never edit by hand. Regenerate: python3 .knowledge/scripts/doc-lint --write-manifest .knowledge +4edfb17fa0d543f4b8fd4570eda8ec5e9ff0c8360fd3c870db5149693d6d1691 guides/docs-agents.md +38f4b3fa2a16b3b0945f39a606e5409fd57fd4aa7b4cdfcd7392ddbc02acaff7 guides/docs-brief.md +10414f50f2da4c92a06839e17f5c7e69c270bcc1f71aa20a80914acd61b29844 guides/docs-codemap.md +5a20ac051e3b169a85bcb97261c12f849ebec4729eb7384393b51735bddd090f guides/docs-memory.md +c12b853b37b237e95883572f83b95847242c54b9908f2bd9fe963d53376c1573 guides/docs-overview.md +7a4c01d616f84724f137e919368f7bd5ea6eca573bf9ff6b80434b291f576b31 guides/docs-prd.md +12b08fbea8bbabba545ec0345a7c1556a0219bb108415f876f20a731677e19c5 guides/docs-research.md +0cdd3cd2af746daf0cf67abceeba8afb9e321d98b55e31113b64e0c97be1ea51 scripts/doc-lint +d4ca902a1689686b6e9c7ebcdb6b77287e72a6fbb611914daf949a7358b3a0db scripts/test_doc_lint.py diff --git a/.knowledge/.version b/.knowledge/.version new file mode 100644 index 00000000..3eefcb9d --- /dev/null +++ b/.knowledge/.version @@ -0,0 +1 @@ +1.0.0 diff --git a/.knowledge/BRIEF.md b/.knowledge/BRIEF.md new file mode 100644 index 00000000..9c7c7da9 --- /dev/null +++ b/.knowledge/BRIEF.md @@ -0,0 +1,34 @@ +# Brief — atlas + +*The always-loaded briefing every agent reads first: the story of what we're building and why. One +screen, stable, PII-free.* + +## Story + +Atlas is a unified AI SDK for Laravel — one fluent, provider-agnostic API for text, images, audio, music, sound effects, video, realtime voice, embeddings, reranking, and moderation, plus a real agent framework on top. It exists so PHP developers build AI-enabled products against a single consistent surface instead of stitching together per-vendor SDKs and payload formats. In its v3 line Atlas owns its entire provider layer, talking directly to vendor HTTP APIs, so the result is a self-contained package that swaps providers, models, and modalities by changing a string — no application code changes. + +## Why it exists + +Most AI libraries return a response and stop there. Real applications hit everything after the happy path — retries, typed errors, streaming that survives interruption, tool-call loops, cost tracking, and audit trails — and each vendor exposes a different SDK and payload shape. Laravel developers who want breadth (many providers, many modalities) and depth (agents, persistence, observability) without vendor lock-in are left gluing SDKs together. Atlas absorbs that work behind one framework-aware, testable interface. When a decision is unclear, the guiding principles are: provider-agnostic and swappable, self-contained with no hard app dependency, and deterministic under test. + +## Users / ICP + +- PHP and Laravel developers building AI features who want one API across many providers and modalities. +- They accomplish generation, embeddings and similarity search, tool-using agents, realtime voice, and cost-saving batch jobs without vendor lock-in. +- What matters most to them: provider-agnostic and swappable, self-contained (no hard app dependency), framework-aware, deterministic and testable, with optional persistence and audit trails when needed. + +## Scope + +- **Active areas:** the `atlas` package — the provider layer (OpenAI, Anthropic, Google Gemini, xAI, ElevenLabs, Cohere, Jina, and any OpenAI-compatible endpoint), all generation modalities, the agent framework (executor, tool loop, sub-agents, middleware), embeddings and chunking, persistence, realtime voice, batch, and the VitePress docs site. +- **Out of scope:** any other Atlas repo (reference only, unless a task explicitly names it); the upstream Prism repo, kept only as a temporary working copy for upstream PRs and never a dependency of Atlas. + +## External Systems + +- `OpenAI`, `Anthropic`, `Google Gemini`, `xAI` — text, image, audio, video, embedding, moderation, and batch generation. +- `ElevenLabs` — realtime voice, music, and sound-effect audio generation. +- `Cohere`, `Jina` — document reranking. +- `OpenAI-compatible endpoints (Ollama, Groq, DeepSeek, LM Studio, and similar)` — self-hosted and third-party model access through the shared ChatCompletions and Responses drivers. +- `atlasphp.org` — the public documentation site built from `docs/`. + +--- +*Editing this file? Follow the standard first: [`guides/docs-brief.md`](./guides/docs-brief.md).* diff --git a/.knowledge/CODEMAP.md b/.knowledge/CODEMAP.md new file mode 100644 index 00000000..d6bbfa25 --- /dev/null +++ b/.knowledge/CODEMAP.md @@ -0,0 +1,165 @@ +# Codemap — atlas + +> As of 2026-07-20, branch `3.x`, CHANGELOG v3.6.1. Re-verify counts against the current branch HEAD if this date is stale. + +The always-loaded structural map: *where things are*, layer by layer. `Atlasphp\Atlas` is a unified AI SDK for Laravel — PHP 8.2+, Laravel 11/12/13 package that owns its own provider layer (no external AI SDK). Providers: OpenAI, Anthropic, Google Gemini, xAI, ElevenLabs (voice/audio), Cohere + Jina (rerank); the `ChatCompletions/` driver covers OpenAI-compatible endpoints (Ollama, LM Studio). Runtime flow: Executor → Driver → Handlers + Resolvers → HttpClient. + +## Entry Points (src/ root) + +- **Atlas** — the facade (`class Atlas extends Facade`, namespace `Atlasphp\Atlas`; carries the `@method static` map). **No `src/Facades/` dir exists** despite the composer alias pointing at `Atlasphp\Atlas\Facades\Atlas`. +- **AtlasManager** — manager/orchestrator behind the facade (facade accessor). +- **AtlasServiceProvider** — bootstrap: voice route registration, agent + chunkable auto-discovery, config + migrations publish. +- **AtlasConfig**, **RequestConfig** — config DTOs. **AtlasCache** — model/voice/embedding cache. **Agent**, **AgentRegistry** — agent definition + registry. +- **Facade methods** — `Atlas::` `text()` `image()` `audio()` `music()` `sfx()` `speech()` `video()` `embed()` `moderate()` `voice()` `rerank()` `batch()` `batchGroup()` `provider()` `agent()` `providers()` `registerChunkable()` `chunkables()` `similaritySearch()` `fake()`. + +## Enums (src/Enums/ — 11) + +BatchResultStatus, BatchStatus, ChunkType, FinishReason, Modality, Provider, ReasoningEffort (Minimal/Low/Medium/High), Role, ToolChoiceMode, TurnDetectionMode, VoiceTransport. + +## Messages (src/Messages/ — 6) + +Message (base), UserMessage, AssistantMessage, SystemMessage, ToolCall, ToolResultMessage. + +## Pending Builders (src/Pending/ — 15) + +Fluent builders returned by the facade: TextRequest, ImageRequest, AudioRequest, VideoRequest, SpeechRequest, MusicRequest, SfxRequest, EmbedRequest, ModerateRequest, RerankRequest, VoiceRequest, AgentRequest, BatchRequest (`add()`/`addMany()`/`group()`/`completionWindow()`/`submit()`), GenerativeAudioRequest, ProviderRequest (base). + +- **Concerns/ (9)** — ConvertsResultToChunks, HasMeta, HasMiddleware, HasProviderOptions, HasQueueDispatch, HasRequestConfig, HasVariables, NormalizesMessages, ResolvesProvider. +- **Contracts/ (1)** — Batchable (the contract BatchRequest accepts). + +## Requests — DTOs (src/Requests/ — 11) + +Immutable request DTOs (names mirror `Pending/` builders): AudioRequest, Batch (+ BatchLine), EmbedRequest, ImageRequest, ModerateRequest, Reasoning (`budgetTokens()`; threaded into TextRequest as `?Reasoning $reasoning`), RerankRequest, TextRequest, VideoRequest, VoiceRequest. + +## Responses (src/Responses/ — 18) + +TextResponse, StreamResponse, StructuredResponse, ImageResponse, AudioResponse, VideoResponse, RerankResponse (+ RerankResult), EmbeddingsResponse, ModerationResponse, BatchResponse (+ BatchResult, RequestCounts), Usage, TokenCount (pre-flight input-token count from `->countTokens()`), VoiceSession, VoiceEvent, StreamChunk. **Contracts/** — Storable. + +## Executor (src/Executor/ — 7) + +Tool loop + step orchestration: AgentExecutor, ToolExecutor, ToolRegistry, ExecutionContext, Step, ExecutorResult, ToolResult. + +## Providers (src/Providers/) + +- **Core (9)** — Driver, ResponsesDriver (neutral OpenAI-Responses-API driver for Ollama and other Responses-API proxies), ProviderRegistry, ProviderConfig, ProviderCapabilities, ModelList, VoiceList, WebSocketConnection, SseParser. (HttpClient + RetryDecider live in `src/Http/`.) +- **Responses/** — shared OpenAI Responses API resolver set: Handlers/Text, MediaResolver, ResponseParser, ToolMapper. Composed by both `OpenAiDriver` and `ResponsesDriver`. +- **Handlers/ (12)** — modality interfaces/abstracts: AbstractProviderHandler, AbstractRerankHandler, ProviderHandler, TextHandler, ImageHandler, AudioHandler, VideoHandler, ModerateHandler, EmbedHandler, RerankHandler, VoiceHandler, BatchHandler. +- **Contracts/ (5)** — resolver seams: MessageFactoryContract, ResponseParserContract, ToolMapperContract, MediaResolverContract, ProviderRegistryContract. +- **Concerns/ (7)** — AppliesToolChoice, BuildsHeaders, BuildsResponsesMessages, BuildsVoiceBody, CountsTokens (heuristic estimate where no native count endpoint), ResolvesAudioFile, ResolvesMediaUri. +- **Tools/ (9)** — provider-native tools: ProviderTool (base), ProviderToolRegistry, CodeExecution, CodeInterpreter, FileSearch, GoogleSearch, WebFetch, WebSearch, XSearch. + +### Per-vendor drivers (src/Providers/{Vendor}/ — 8) + +| Vendor | Driver + parts | Handlers | +|---|---|---| +| OpenAi | OpenAiDriver (Responses API), MessageFactory, Concerns/HasOrganizationHeader (resolvers from shared `Responses/`) | Audio, Batch, Embed, Image, Moderate, Provider, Text, Video, Voice | +| Anthropic | AnthropicDriver, MediaResolver, MessageFactory, ResponseParser, ToolMapper, Concerns/BuildsAnthropicHeaders | Batch, Provider, Text | +| Google | GoogleDriver, GoogleToolCall, MediaResolver, MessageFactory, ResponseParser, ToolMapper, Concerns/BuildsGoogleHeaders | Batch, Embed, Image, Provider, Text | +| Xai | XaiDriver, MessageFactory, ResponseParser, ToolMapper | Audio, Image, Provider, Text, Video, Voice | +| ChatCompletions | ChatCompletionsDriver, MediaResolver, MessageFactory, ResponseParser, ToolMapper (Ollama, LM Studio) | Provider, Text | +| ElevenLabs | ElevenLabsDriver, Concerns/BuildsElevenLabsHeaders | Audio, Music, Provider, Sfx, Voice | +| Cohere | CohereDriver | CohereRerankHandler | +| Jina | JinaDriver | JinaRerankHandler | + +## Persistence (src/Persistence/) + +Root: ProcessQueuedMessage, ToolAssets. Model services (`Services/`) are the single point of truth for Eloquent access; domain services orchestrate. + +- **Models/ (12)** + +| Model | Notes | +|---|---| +| Asset, ConversationMessageAsset | media assets + message join | +| Conversation, ConversationMessage | conversation persistence | +| Execution | sub-agent lineage (`parent_execution_id`/`parent_tool_call_id`/`depth`, `totalUsage()` subtree roll-up) | +| ExecutionStep, ExecutionToolCall | per-step + per-tool-call records | +| BatchGroup, BatchJob | job status/counts/usage (`open()` scope, `applyStatus()`/`markCompleted()`/`markFailed()`) | +| BatchResult | per-line, unique `(batch_job_id, custom_id)` | +| Chunk | embedding chunk rows | +| VoiceCall | voice session records | + +- **Services/ (5)** — ConversationService, ExecutionService, ChunkContentService, ChunkSearchService, RecordSearchService. +- **Middleware/ (5)** — PersistConversation, TrackExecution, TrackProviderCall, TrackStep, TrackToolCall. +- **Concerns/ (7)** — consumer-app model traits: HasAtlasTable, HasChunkedEmbeddings, HasConversations, HasExecutionStatus, HasOwner, HasVectorEmbeddings, ResolvesChunkModel. +- **Enums/ (7)** — AssetType, ExecutionStatus, ExecutionType, MessageRole, MessageStatus, ToolCallType, VoiceCallStatus. +- **Misc** — Schema/ChunkedEmbeddingColumns, Support/MimeTypeMap, Http/StoreVoiceTranscriptController. + +## Embeddings (src/Embeddings/ — 7) + +EmbeddingResolver, VectorQueryMacros, Chunkable, ChunkableRegistry, ChunkData, SearchResult, VectorEmbeddable. **Chunkers/ (3)** — Chunker (base), BaseTokenAwareChunker, MarkdownChunker. + +## Voice (src/Voice/Http/ — 2) + +VoiceToolController, CloseVoiceSessionController. VoiceSession + VoiceEvent live under `Responses/`; the `BuildsVoiceBody` trait (session body + `https:`→`wss:` rewrite) is in `Providers/Concerns/`, shared by OpenAI + xAI voice handlers. + +## Queue (src/Queue/) + +PendingExecution. **Contracts/** — QueueableRequest. **Jobs/ (3)** — ExecuteAtlasJob, ChunkContentJob, TracksExecution. + +## Middleware (src/Middleware/ — 6) + +ProviderContext, ToolContext, StepContext, AgentContext, MiddlewareStack, MiddlewareResolver. + +- **Contracts/ (11)** — marker interfaces routed by type: AgentMiddleware, AudioMiddleware, EmbedMiddleware, ImageMiddleware, ProviderMiddleware, StepMiddleware, TextMiddleware, ToolMiddleware, VideoMiddleware, VoiceMiddleware, VoiceHttpMiddleware. + +## Tools — infrastructure (src/Tools/ — 6) + +Tool, ToolDefinition, ToolSerializer, ToolChoice, SimilaritySearch, AgentTool (wraps an Agent as a sub-agent delegation tool; `Tool::isDelegation()` flag). + +## Schema (src/Schema/ — 3) + +SchemaBuilder, Schema, StrictSchema (normalizes JSON Schema to OpenAI strict form: recursive `additionalProperties:false` + all-required, optionals → nullable). **Fields/ (9)** — Field (base), StringField, IntegerField, NumberField, BooleanField, ArrayField, ObjectField, ObjectFieldBuilder, EnumField. + +## Events (src/Events/ — 40, + 3 Concerns/) + +- **Agent (8)** — AgentStarted, AgentStepStarted, AgentStepCompleted, AgentCompleted, AgentMaxStepsExceeded, AgentToolCallStarted, AgentToolCallCompleted, AgentToolCallFailed. +- **Execution (5)** — ExecutionEvent (base), ExecutionQueued, ExecutionProcessing, ExecutionCompleted, ExecutionFailed. **Modality (2)** — ModalityStarted, ModalityCompleted. +- **Provider (4)** — ProviderRequestStarted, ProviderRequestCompleted, ProviderRequestFailed, ProviderRequestRetrying. +- **Streaming (5)** — StreamStarted, StreamCompleted, StreamChunkReceived, StreamThinkingReceived, StreamToolCallReceived. +- **Voice (9)** — VoiceSessionCreated, VoiceSessionEnded, VoiceCallStarted, VoiceCallCompleted, VoiceToolCallStarted, VoiceToolCallCompleted, VoiceToolCallFailed, VoiceAudioDeltaReceived, VoiceTranscriptDeltaReceived. +- **Chunking/Conversation (3)** — ContentChunked, ContentChunkingFailed, ConversationMessageStored. **Batch (4)** — BatchSubmitted, BatchCompleted, BatchFailed, BatchGroupCompleted. +- **Concerns/ (3)** — BroadcastsOnChannel, BroadcastsOnOptionalChannel, CapsBroadcastPayload. + +## Exceptions (src/Exceptions/ — 17) + +AtlasException (base), ProviderException, AuthenticationException, AuthorizationException, RateLimitException, ProviderNotFoundException, AgentNotFoundException, ToolNotFoundException, MaxStepsExceededException, UnsupportedFeatureException, MaxDelegationDepthException, DelegationCycleException, BatchException, ConnectionException, InvalidRequestException, ModelNotFoundException, ServerException. + +## Support layers + +- **Input/ (5)** — Input (base), Image, Audio, Video, Document. +- **Console/ (9)** — MakeAgentCommand, MakeToolCommand, MiddlewareCommand, CleanStaleVoiceSessionsCommand, ChunkCommand, RechunkCommand, PruneChunksCommand, PollBatchJobsCommand, PruneBatchJobsCommand. +- **Batch/ (1)** — BatchService (domain orchestration: `submitAndTrack()` + `syncFromProvider()` in a transaction; shared by the poll command). Batch = deferred provider jobs at ~50% cost. +- **Support/ (3)** — VariableRegistry, VariableInterpolator, TokenCounter (pure utilities). +- **Concerns/ (2)** — StoresMedia, ResolvesDatabaseScope (multi-tenant scoping for unique job locks). +- **Http/ (3)** — HttpClient, RetryDecider, ProviderRequestContext (shared transport; stamps a correlation id onto `ProviderRequest*` events). + +## Testing Fakes (src/Testing/ — 13) + +AtlasFake, **FakeDriver** (skips the base `Driver` constructor, so `Driver::$config` is never set — resolves attribution from its own name), RecordedRequest, plus response fakes: TextResponseFake, ImageResponseFake, AudioResponseFake, VideoResponseFake, EmbeddingsResponseFake, ModerationResponseFake, RerankResponseFake, StreamResponseFake, StructuredResponseFake, VoiceSessionFake. + +## Config & Routes + +- `config/atlas.php` — keys: `defaults`, `agents`, `prompt_cache`, `providers`, `retry`, `queue`, `batch`, `stream`, `broadcast`, `middleware`, `variables`, `storage`, `embeddings`, `cache`, `persistence`, `voice`. Real values live in `.env` / config, not here. +- **Routes** — registered programmatically by `AtlasServiceProvider::registerVoiceRoutes()` in `boot()`, prefix from `voice.route_prefix` (default `atlas`): `POST {prefix}/voice/{sessionId}/tool` → VoiceToolController · `/close` → CloseVoiceSessionController · `/transcript` → StoreVoiceTranscriptController. + +## Database (database/ — 19 migrations, 7 factories) + +`migrations/` — `atlas_`-prefixed tables: conversations, conversation_messages, assets, voice_calls, executions, execution_steps, execution_tool_calls, chunks, batch jobs/results, plus FK-add migrations. `factories/` — Asset, Conversation, ConversationMessage, ConversationMessageAsset, Execution, ExecutionStep, ExecutionToolCall. + +## Tests (tests/ — Pest) + +- **Feature/** — Console, Persistence, Testing, Variables, Voice + top-level entry-point / facade / config / token-count tests. +- **Unit/** — mirrors `src/` per domain (Embeddings/Chunkers, Persistence subdirs, Providers per-vendor + Concerns/Tools/Responses, Streaming, etc.). **Fixtures/**. + +## Sandbox & CI + +- `sandbox/` — real-API test harness (Laravel app shell, `bootstrap.php`); Horizon must run for queue-backed features. Provider smoke tests, feature/reasoning/batch/embeddings scripts, `seed-demo.php`. +- `.github/workflows/` — `tests.yml` (CI), `deploy-docs.yml`, `doc-lint.yml` (knowledge-doc lint). `composer check` = lint:test (Pint) + analyse (PHPStan) + test (Pest) + lint:docs (doc-lint). + +## Docs + +- `.knowledge/` — the agent-facing documentation system and home of the always-loaded orientation trio (BRIEF/CODEMAP/MEMORY) + OVERVIEW, plus prd/, prd-drafts/, research/, references/, guides/. See `.knowledge/README.md`. +- `docs/` — consumer-facing VitePress site (atlasphp.org): getting-started, modalities/capabilities, features, guides, advanced. + +--- +*Editing this file? Follow the standard first: [`guides/docs-codemap.md`](./guides/docs-codemap.md).* diff --git a/.knowledge/MEMORY.md b/.knowledge/MEMORY.md new file mode 100644 index 00000000..3a0d69ca --- /dev/null +++ b/.knowledge/MEMORY.md @@ -0,0 +1,15 @@ +# Memory — atlas + +Always-loaded, read at the start of every task: the friction we've hit in **this codebase** and the +workaround for each — so you don't re-hit it. **A living list — delete an entry once it's genuinely solved; +a long MEMORY means something was solved and never pruned.** This codebase only. + +## Friction / gotchas + +*One bullet each: the trap, and the workaround. Delete when it's genuinely solved.* + +- **`FakeDriver` skips the base `Driver` constructor**, so `Driver::$config` is never set — any base `Driver` method that reads `$config` fails under the fake. Override those methods in `FakeDriver` (e.g. `providerName()` returns `name()`) rather than letting them fall through to the `$config`-reading base. +- **Gemini streams tool calls one-per-chunk and can bundle several parallel calls into one chunk.** Raw driver streams also expose finish-reason edge cases the executor loop hides. Parse defensively (handle bundled parallel calls, don't assume one call per chunk) and test tool-call / finish-reason behavior at the driver level, not only through the executor. + +--- +*Editing this file? Follow the standard first: [`guides/docs-memory.md`](./guides/docs-memory.md).* diff --git a/.knowledge/OVERVIEW.md b/.knowledge/OVERVIEW.md new file mode 100644 index 00000000..d035d7cf --- /dev/null +++ b/.knowledge/OVERVIEW.md @@ -0,0 +1,106 @@ +# Overview — Atlas + +*Atlas in plain language — written for product, marketing and anyone new, not for developers. What the +parts are and how a request moves through them.* + +*This describes the platform as designed. What is proven today is recorded row by row in the contracts — +[`prd/`](./prd/) for the ratified ones, [`prd-drafts/`](./prd-drafts/) for those still in proposal.* + +## What this is + +Atlas is an open-source toolkit that gives Laravel developers one consistent way to use AI — text, images, +audio, voice, embeddings and more — across every major provider without rewriting code for each one. It is +for PHP and Laravel teams building AI features who want to switch providers, models and capabilities by +changing a single setting instead of stitching vendor toolkits together. It is not sold: it is free, +MIT-licensed open source, and a team adopts it by installing the package. + +## The platform + +```mermaid +flowchart LR + subgraph Base["Foundation"] + config["Settings
provider, model, keys"] + api["One unified API"] + transport["Shared connection
one line out to every vendor"] + end + subgraph Provider["Providers"] + connectors["Provider connectors
OpenAI · Anthropic · Google · xAI · ElevenLabs · Cohere · Jina · compatible"] + end + subgraph Modality["What you can generate"] + text["Text & structured output"] + media["Images, audio, music & video"] + voice["Realtime voice"] + embed["Embeddings"] + rerank["Reranking"] + moderation["Moderation"] + batch["Batch"] + end + subgraph Flow["Agent behaviour"] + toolloop["Tool loop"] + subagents["Sub-agents"] + streaming["Streaming"] + search["Similarity search"] + end + + config --> api + api --> toolloop + toolloop --> text + toolloop --> subagents + toolloop --> streaming + api --> media + api --> voice + api --> embed + embed --> search + api --> rerank + api --> moderation + api --> batch + text --> connectors + media --> connectors + voice --> connectors + embed --> connectors + rerank --> connectors + moderation --> connectors + batch --> connectors + connectors --> transport +``` + +## How it works + +- **One unified API** — the single fluent surface a developer writes against for every task. +- **Settings** — choose the provider, model and keys a request uses; swap by changing one string. +- **Tool loop** — runs a request as several steps, calling the app's own tools until the answer is ready. +- **Sub-agents** — lets an agent hand parts of a job to other agents, in parallel, within guardrails. +- **Streaming** — delivers a reply piece by piece as it is produced, rather than waiting for the whole. +- **Text & structured output** — written answers, or clean structured data checked against a shape the app + defines. +- **Images, audio, music & video** — generates and edits visual and audio media from a prompt. +- **Realtime voice** — a live, two-way spoken conversation with an agent that can still use tools. +- **Embeddings** — turns text into numeric fingerprints so records can be compared by meaning. +- **Similarity search** — finds the records closest in meaning to a query, over whole records or chunks. +- **Reranking** — reorders a set of results by how well each one answers a query. +- **Moderation** — flags unsafe or disallowed content before it reaches a user. +- **Batch** — runs large jobs together at roughly half the cost when speed is not urgent. +- **Provider connectors** — the per-vendor adapters that speak each provider's dialect behind the one API. +- **Shared connection** — the single outbound line every provider call travels, with retries and tracking. + +## What you use + +- **The Laravel developer** — installs the package and builds against the one API; swaps providers, models + and modalities by changing a string, never rewriting their app. +- **The app's end users** — chat, speak or get results through features the developer builds on Atlas; they + never see Atlas itself and touch nothing here directly. +- **The maintainers** — govern the public API and the provider layer, and decide what each release is + allowed to change for the developers who depend on it. + +## What governs it + +- **The public API contract** — a documented method, signature or setting never changes without maintainer + approval, because every installed app depends on it. Set by the maintainers. + See [`../AGENTS.md`](../AGENTS.md). +- **The layer boundaries** — each part may depend only downward, which is what keeps providers swappable and + every feature testable. Set by the project's architecture rules. See [`../AGENTS.md`](../AGENTS.md). +- **Versioning & changelog discipline** — every consumer-visible change is recorded, and a breaking change + is called out with the exact upgrade steps. Set by the release process. See [`../AGENTS.md`](../AGENTS.md). + +--- +*Editing this file? Follow the standard first: [`guides/docs-overview.md`](./guides/docs-overview.md).* diff --git a/.knowledge/README.md b/.knowledge/README.md new file mode 100644 index 00000000..74de9c25 --- /dev/null +++ b/.knowledge/README.md @@ -0,0 +1,32 @@ +# .knowledge/ — what lives here + +The project's knowledge base: one home per kind of doc. This maps what's here and where to go — how to write +each lives in its `guides/*.md`. + +## The homes + +| Path | Holds | How to write it | +|---|---|---| +| `BRIEF.md` | Orientation — what & why (always-loaded) | `guides/docs-brief.md` | +| `CODEMAP.md` | Orientation — where things are (always-loaded) | `guides/docs-codemap.md` | +| `MEMORY.md` | Orientation — current friction (always-loaded) | `guides/docs-memory.md` | +| `OVERVIEW.md` | **The platform, for a person** — a diagram and the walk through it | `guides/docs-overview.md` | +| `prd/` | **Tested contracts — the source of truth** | `guides/docs-prd.md` | +| `prd-drafts/` | Proposals, not yet approved | `guides/docs-prd.md` | +| `research/` | Prior art — dated notes on how others solved a problem | `guides/docs-research.md` | +| `references/` | Visual targets — screenshots, UI to match | `references/README.md` | +| `guides/` | The writing standards (`docs-*`) + project how-tos | `guides/README.md` | +| `../AGENTS.md` | The project's law (at the repo root, not here) | `guides/docs-agents.md` | +| `scripts/` | Tooling — the linter + its teeth-test | `scripts/README.md` | +| `tmp/` | Git-ignored scratch | — | + +Work flows `research/` + `references/` → `prd-drafts/` → `prd/`. + +## Versioning + +Versioned by [knowledge-template](https://github.com/timothymarois/knowledge-template); the adopted version +is stamped in `.version`. To update, follow the upgrade steps in that repo. + +`.payload-manifest` holds a checksum for every file that version owns — the `guides/docs-*.md` standards and +the two shipped scripts. `doc-lint` verifies them on every run, so the stamp is proven rather than trusted. +Never edit those files or the manifest here; a change belongs upstream and arrives as a version bump. diff --git a/.knowledge/guides/README.md b/.knowledge/guides/README.md new file mode 100644 index 00000000..ff47c2bc --- /dev/null +++ b/.knowledge/guides/README.md @@ -0,0 +1,29 @@ +# guides/ — writing standards & how-tos (catalog) + +**`docs-*.md`** — the shipped writing standards (versioned by `knowledge-template`; don't edit per project). **`.md`** — project how-tos you author for recurring tasks in this repo. + +## Contents — maintained by hand + +Add a row when you add a guide; `doc-lint` fails the build if one is missing. + +### Shipped standards + +| Guide | Standard for | +|---|---| +| [docs-prd.md](./docs-prd.md) | PRDs in `../prd/` (and drafts) | +| [docs-research.md](./docs-research.md) | Research notes in `../research/` | +| [docs-brief.md](./docs-brief.md) | `../BRIEF.md` | +| [docs-codemap.md](./docs-codemap.md) | `../CODEMAP.md` | +| [docs-memory.md](./docs-memory.md) | `../MEMORY.md` | +| [docs-overview.md](./docs-overview.md) | `../OVERVIEW.md` | +| [docs-agents.md](./docs-agents.md) | `AGENTS.md` | + +### Project how-tos + +| Guide | How to | +|---|---| +| _(none yet)_ | | + +## Rules for a project how-to + +One task per file, named for the action. Steps in order with the actual commands and a check that confirms success. Self-contained. diff --git a/.knowledge/guides/docs-agents.md b/.knowledge/guides/docs-agents.md new file mode 100644 index 00000000..a0c4444c --- /dev/null +++ b/.knowledge/guides/docs-agents.md @@ -0,0 +1,251 @@ +# How to write AGENTS.md — the standard + +This guide **is the standard** for the project's root `AGENTS.md` — the law every agent reads before touching +the repo. Shipped and versioned by `knowledge-template`. `AGENTS.md` is *not* inside `.knowledge/`; it sits at +the repo root and routes agents into it. + +`AGENTS.md` does two jobs: **route** agents to the knowledge base, and **encode this stack's rules** so an +agent builds correctly here. Its sections are of two kinds: + +- **Ship as written** (identical in every repo): *Before you work*, *Hard gates*, *Documentation duties*, and + the top of *Never*. **Byte-identical — do not reword a line, and do not extend a bullet with project + detail.** Their whole value is that you can compare them across repos and see they match; an enriched + bullet reads better in place and quietly destroys that. + **You may append project-specific lines below the shipped ones in the same section**, or add a section of + your own (`## Project-specific hard gates`). What you may never do is edit, reword, reorder or interleave + a shipped line. The test: every shipped line still present, unmodified, in order. +- **Fill in for this project** (by researching the codebase): *Tech stack*, *Architecture*, *Best practices*, + *Directory structure*, *Build / test / run*, *Definition of done*. + +## How to write it + +1. **Research the codebase first — don't guess.** Identify the language/runtime, frameworks, the architecture + and layering (what each layer may depend on), naming/style with the actual format + lint command, the test + setup, and the one command that gates a change. Delegate the survey to subagents if your harness supports them. +2. **Keep the ship-as-written sections verbatim** — they're what make every repo behave the same. +3. **A rule that routes an obligation through a mechanism must say what happens without it.** This is the + defect that testing finds most often, and it fails silently: the obligation doesn't get flagged, it + simply disappears, because an agent hitting the uncovered case copies whatever the codebase already + does — and the codebase is usually where the hole came from. + + ``` + ❌ Authorize in the Policy via the Form Request. + → an action with no input has no Form Request, so it ships with no authorization at all + ✅ Every state-changing route is authorized — no exceptions. With input, authorize in the Form + Request via a Policy; without input, call the Policy directly from the controller. + ``` + + Write the critical ones — authorization, money, data loss, privacy — as **"no exceptions" first and + *how* second.** A rule phrased as a mechanism is only as complete as the mechanism. + +4. **If a rule is mechanically checkable, put it in the gate.** A checkable rule left as prose is the + first one skipped: in testing, the one styling rule stated only in prose was missed by every agent, + while the rules the gate enforced were followed without exception. Either wire it into the check + command, or show it inside a `✅`/`❌` example where an agent is already looking — a rule that lives + only in a bullet list is decoration. + +5. **Encode best practices as enforceable rules** — specific and checkable. **Every area whose rule is about + the shape of code carries a `✅`/`❌` pair**, not just prose: the wrong version beside the right one is the + single most-followed thing in this file. Vague conventions get ignored. +6. **Set the Definition of done** to the real gate command. +7. **Then list what research could not tell you, and ask.** Reading a codebase surfaces the rules it + *shows*; three kinds never appear in it, and a file missing them looks finished while omitting the rules + the owner cares most about: + - **Rules the code already obeys perfectly.** A ban nobody has ever broken leaves zero trace — search + for the violation, find nothing, and the rule is invisible. + - **Rules describing an intended pattern not built yet.** The directory is empty because the convention + is aspirational, not because it doesn't exist. + - **Workflow and taste.** Who runs the app, who signs off on UI, what the project refuses on principle. + These live only with the owner. + End by naming what you inferred versus what you guessed, and ask about the gaps. **Ask, don't invent.** +8. Keep it lean — rules an agent follows, not prose. Roughly a screen per section; past ~150 lines you are + explaining rather than ruling, and the file stops being reread. + +## The template + +Copy this. Fill every ``; keep the ship-as-written sections as they are. + +```md +# AGENTS + +Rules for every agent working in this repository. These rules are law; where they conflict with your general +habits, this file wins. + +This is a ****. The *what & why* lives in `.knowledge/BRIEF.md`; +the knowledge map is `.knowledge/README.md`. This file defines how you build here. + +## Before you work + +Load light; pull depth only when the task needs it. + +1. **Always read first:** `.knowledge/BRIEF.md` (what & why), `.knowledge/CODEMAP.md` (where things are), + `.knowledge/MEMORY.md` (current friction). `.knowledge/README.md` maps the rest. +2. **On demand, when the task enters an area:** `.knowledge/prd/` (ratified contracts — source of truth), + `.knowledge/prd-drafts/` (proposals), `.knowledge/research/` + `.knowledge/references/` (prior art, visual + targets), `.knowledge/guides/` (how to write each doc + project how-tos). +3. **How work flows:** `research/` -> `prd-drafts/` -> `prd/`; a `prd/` contract never cites a draft. New + guaranteed behavior is a `prd/` row backed by a test — cite its `R--` in the code. Follow a doc's + guide before writing or modifying it, and keep docs true in the same task. Run + `python3 .knowledge/scripts/doc-lint .knowledge` before finishing; scratch -> `.knowledge/tmp/`. +4. Read every file before editing it; search before writing new logic — reuse, extend, refactor. +5. When the user raises a concern, investigate before contradicting — evidence, not a hunch. + +## Hard gates — require explicit approval + +- **Persisted state.** Any change to schema, stored data, or migrations is confirmed first. +- **Dependencies.** Do not add, remove, or major-version-bump a package without approval. +- **Deletions.** Do not delete files outside the task's immediate scope without approval. +- **Commits.** Do not commit or push unless told to. +- **This file.** Never modify `AGENTS.md` without approval; when approved, follow + `.knowledge/guides/docs-agents.md`. + +## Never + +- Never touch secrets or commit credentials. +- Never leave debug output or commented-out code in completed work. +- + +## Tech stack + +- **:** +- **:** + +## Architecture — the one rule that matters + + + +## Best practices — do / don't + + + +``` +✅ +❌ +``` + +## Directory structure + + + +## Build, test & run + + + +## Documentation duties + +Keep docs true in the same task that changes reality. Before creating or editing a doc, read its home +`README.md` and follow its `guides/docs-*.md`. + +- Moved/restructured files -> update `.knowledge/CODEMAP.md`. +- Hit friction — **anything that cost you a failed attempt**: an env var or flag you had to discover, + a guard you had to satisfy, a command that only worked the second way you tried it, an error whose + message didn't say what to do -> **write the line into `.knowledge/MEMORY.md` the moment you find + the workaround, before you carry on** — by the end of the task it will feel too small to mention, + which is exactly how the next agent loses the same hour. Delete it once solved. +- Owner ratifies a draft (**the whole file**, not one row) -> `git mv` it into `prd/`; IDs carry over and + the conformance review then sets glyphs. **Approval moves a draft, not proof** — proof is the glyph + column. Behavior and its requirement row change in the same commit. +- Scratch -> `.knowledge/tmp/` (git-ignored). + +## Definition of done + +1. passes. +2. Every rule here held. +3. New guaranteed behavior is proven by a `prd/` requirement and its test. +4. **Friction you hit is in `.knowledge/MEMORY.md`, not only in your reply** — the next agent reads the + file, not this conversation. Hit none? Say that in your reply. **Never write "no friction" into the + file** — `MEMORY.md` records traps, never their absence. +``` + +## The repo already has an `AGENTS.md` + +The common case, and the one that loses work if you get it wrong. **Reconcile it — never replace it.** + +That file is the accumulated judgement of everyone who worked here: a rule in it usually exists because +something once went wrong. Adopting the standard changes the *shape*, not the content. + +1. **Read the existing file first and inventory every rule it makes** — before writing a line of the new one. +2. **Re-home each one** under the section of this standard where it belongs. A stack rule goes to *Best + practices*, a "never do X" to *Never*, an approval requirement to *Hard gates*, a command to + *Build, test & run*. +3. **Add the ship-as-written sections** it was missing, verbatim. +4. **Nothing is dropped silently.** If a rule looks obsolete, wrong, or contradicted by the standard, **say + so and ask** — do not quietly leave it out. A rule that disappears in a reformat is indistinguishable + from one that was never there. +5. **Keep its examples.** An existing `✅`/`❌` pair written against this codebase is worth more than + anything you would write from scratch. + +6. **A rule with no natural home keeps its own section.** The sections in this standard are the ones every + project needs, not the only ones allowed. If the old file had a *Testing* section stating what each + suite owns, and nothing here fits it, **add that section back** — dropping a real obligation because the + template had no slot for it is the worst possible trade. + +### Show your work — this is a step, not a courtesy + +**Before you finish, write out the inventory and hand it over:** every rule you found in the old file, and +for each one, the section it now lives in — or that you are proposing to drop it, and why. + +``` +48 rules found · 44 re-homed · 4 proposed for removal (listed below, with reasons) — approve? +``` + +Without that list, nothing is visibly missing: a lost rule leaves no trace, the new file reads as complete, +and the reviewer has no way to notice that an obligation quietly evaporated. **Tested without it, an agent +kept 32 of 48 rules and reported success** — the losses included an entire testing section binding new +commands to specific suites. Counting them out loud is what makes the omission impossible to miss. + +**Then have someone else check it — you are the worst auditor of your own omissions.** With the inventory +step in place the same test kept 43 of 49 rules, a real improvement, and the agent still reported that +nothing had been dropped while four rules were in fact gone and two more had been softened into vagueness. +A rule you never noticed you left out is one you cannot report. Hand the old file and the new one to a +fresh reviewer — another agent will do — and ask for a rule-by-rule verdict before calling it done. + +## Changing it later + +`AGENTS.md` is under its own hard gate: **never modify it without approval.** That applies to you as much +as to anyone — propose the change and say why. + +- **A `MEMORY.md` trap that hardened into a permanent rule graduates here.** When friction stops being + "work around it" and becomes "never do it", it moves into *Never* and is deleted from `MEMORY.md`. It + lives in exactly one of the two files. +- **A convention that changed changes here in the same task**, not in a comment or a commit message. +- **Ship-as-written sections are never edited** — not even to improve them. Raise it upstream instead; + they are what keep every repo behaving the same. + +## CLAUDE.md — a router, not a copy + +Claude Code reads `CLAUDE.md`. Keep it a thin **router** to `AGENTS.md` — never a second copy of the rules +(which would drift). If the project has none, create it: + +```md +# CLAUDE.md + +Before you respond to the user, do any task, or any action, you must read and follow +[AGENTS.md](./AGENTS.md) completely. + +If you have not read it, your next step must be to read it first, always. +``` + +## Lint + +`doc-lint` checks this file too — it is the entry point that sends an agent into `.knowledge/` at all, so +losing it silently unloads every doc below it. Three checks, deliberately shallow: + +- The repo root has an `AGENTS.md`. +- Somewhere in it, all three of `.knowledge/BRIEF.md`, `.knowledge/CODEMAP.md`, and `.knowledge/MEMORY.md` + are named. +- It names **this guide**, so an agent asked to revise it is routed to the standard first. That lives on + the existing *Hard gates* line — one clause, not a second block: `AGENTS.md` is read on every task and + edited almost never, so its own edit rule earns no more room than that. + +**Nothing else is inspected** — not sections, not wording, not order. How a project writes its rules is its +own business; the lint only guarantees the orientation trio still gets loaded. (Linting a payload not yet +adopted into a repo? Pass `--payload` to skip both.) + +## Rules + +- **Ship-as-written stays verbatim** across repos — that's what keeps every project consistent. +- **Rules, not prose.** Every filled-in line is something an agent can follow or a reviewer can check. +- **Show, don't tell** — a `✅`/`❌` example beats a paragraph. +- **One law file.** `AGENTS.md` is the only place the stack's rules live; the `.knowledge/` docs never restate them. diff --git a/.knowledge/guides/docs-brief.md b/.knowledge/guides/docs-brief.md new file mode 100644 index 00000000..5c489a9f --- /dev/null +++ b/.knowledge/guides/docs-brief.md @@ -0,0 +1,50 @@ +# How to write a BRIEF — the standard + +This guide **is the standard** for `../BRIEF.md`, the always-loaded orientation an agent reads first. +Shipped and versioned by `knowledge-template`. **The template is `../BRIEF.md` itself** — start from it and +fill each section by answering its question. + +A brief describes **what the project is, who it serves, and what it covers** — how the platform +fits together is [`../OVERVIEW.md`](../OVERVIEW.md)'s job, not this file's — about one screen. Not how the +code is built (that's `CODEMAP.md`), not how to work in it (that's `AGENTS.md`). + +**Top rule — never include PII.** This file is committed and world-readable: no real name, username, email, +phone, or address; no secrets or machine paths. Refer to people by role; the product by its public brand or +repo name. + +## Where the answers are + +**Most of a brief is not in the code.** Scope and external systems are; story, users, and what the project +refuses are not. Work down this list, and notice where it stops: + +1. **`README.md`, product docs, a landing page** — the story and the pitch, usually already written. +2. **An existing `AGENTS.md`, or any prior docs** — often carries the *why* and the refusals. +3. **Issues, milestones, a roadmap** — what's in scope now versus later. +4. **The code** — confirms *scope* and *external systems*: what it integrates with, what surfaces exist. + It is evidence of what was built, never of who it is for or why. + +**Then ask.** Users / ICP and the *Out of scope* refusals are owner knowledge; a repo cannot tell you who +someone sells to or what they have decided not to build. **Never infer a market from a schema** — a +plausible invented ICP is worse than a blank one, because it is loaded on every future task and nobody +re-checks it. **Adopting? This is a step, not a courtesy** — list sourced versus inferred and get the inferred +sections confirmed before the adoption is done (`ADOPT.md` step 4). + +## Section guidance + +- **Story** — *what is this and why does it exist?* One short paragraph a stranger understands: the project + and the change it creates. A second identity (a rewrite, a fork) goes in a clause. +- **Why it exists** — the problem, who feels it, and why existing options fall short. +- **Users / ICP** — who uses it, what they're trying to accomplish, and the qualities that matter most to + them. For an internal project, name the internal role and its job. +- **Scope** — the product areas, features, or surfaces the project spans, and what's explicitly out of scope. +- **External Systems** — the databases, services, and third-party systems it relies on, each with what it's + used for. + +## Quality bar + +- **Skimmable** — short bullets, one idea per line, about one screen. +- **No placeholders left** — replace every `<...>` and `_(none)_`; delete a section rather than writing "none". +- **Public and PII-free** — people by role; no private data, secrets, or machine paths. +- **Shape is universal** — the same sections fit any project; adjust the words, not the shape. +- **Change only when it changed** — update when the project's story, users, scope, or systems actually + shifted, not to reword; leave the rest untouched. diff --git a/.knowledge/guides/docs-codemap.md b/.knowledge/guides/docs-codemap.md new file mode 100644 index 00000000..99f4923b --- /dev/null +++ b/.knowledge/guides/docs-codemap.md @@ -0,0 +1,59 @@ +# How to write a CODEMAP — the standard + +This guide **is the standard** for `../CODEMAP.md`, the always-loaded structural map. Shipped and versioned +by `knowledge-template`. **The template is `../CODEMAP.md` itself** — start from its skeleton and adapt the +sections to the stack. + +A codemap is a **structural inventory** — the table of contents of a codebase. It answers "where does X live +and what exists?" layer by layer, so you can navigate without grepping first. It is not a tutorial, not a set +of conventions, and not a place for gotchas (those are `MEMORY.md`). + +**Public and PII-free.** Never copy credentials, tokens, connection strings, personal emails, or machine +paths; generalize identifiers that embed a person's name. Point to `.env.example` or config as the source of +truth rather than listing real values. + +## Building one + +Survey the repo systematically, not from memory: + +1. **Identify the stack and its layers** — highest-signal first, before opening any source file: the + dependency manifest (`package.json`, `composer.json`, `pyproject.toml`, `go.mod`…), the entry points, + the routing / wiring / DI config, the test directory layout, and the build or task scripts. Those five + name the layers in minutes; the folder tree alone will mislead you on a repo that doesn't follow its + framework's defaults. +2. **Survey each layer** folder by folder: list every artifact — name, one-line purpose, key relationships. + Split a large repo across parallel passes, one per layer. +3. **Count what you inventory** ("28 models", "66 controllers") — counts show completeness and make drift + obvious. **Count artifacts, not lines.** "5 modules", "77 commands", "201 tests" survive a refactor; + "client.py — 1,205 lines" is stale on the next commit and tells a reader nothing they can use. +4. **Compress** to names + terse notes. Aim for **under ~200 lines** — density over prose. + +## Sections are per-layer maps — adapt them to the stack + +The starter ships a generic skeleton (Entry Points · Domain/Data · Backend/Services · Frontend/UI · Tests · +Scripts · Integrations/Jobs · Docs). Rename, split, drop, and add sections so each maps a real layer in +*this* codebase; delete any skeleton section with nothing to list. The shape follows the code: + +- A web app maps as its framework layers — models, services, controllers, routes, jobs, pages, components. +- A browser extension maps as manifest & entry points, background worker, content scripts, UI, messaging. +- A native / game project maps as build targets, scenes/entities, systems, assets, scripts, tests. + +The only invariant: **one section per layer, each listing what exists.** + +**A monorepo still maps by layer, not by app.** Sections span the workspaces — *Entry points (3 — one per +app)*, *Frontend (4 across two homes)* — with the workspace named in each entry. One section per app +duplicates every layer N times and buries the thing a reader wants, which is where a *kind* of thing lives. + +## Entry format + +- **Group by folder**, with the count in the heading: `## Services (app/Services/ — 18)`. +- **One line per artifact:** name — terse purpose — key relationships/notes. +- **Use a table for dense, relational layers** (e.g. models: Name | Relationships | Notes); bullets for flat lists. +- **Point to the source of truth** rather than copying it. Abbreviate ruthlessly. + +## Keeping it current + +- **Refresh on drift, not on a timer.** When a change adds/removes a layer or shifts counts, update the + affected sections. A count that no longer matches the repo is the first sign of staleness. +- **Only the sections the change touched.** Don't reword or restructure unrelated layers for its own sake. +- Record removals too — drop the lines and note it. diff --git a/.knowledge/guides/docs-memory.md b/.knowledge/guides/docs-memory.md new file mode 100644 index 00000000..d1100c46 --- /dev/null +++ b/.knowledge/guides/docs-memory.md @@ -0,0 +1,40 @@ +# How to write MEMORY — the standard + +This guide **is the standard** for `../MEMORY.md`, the always-loaded list of friction to avoid. Shipped and +versioned by `knowledge-template`. **The template is `../MEMORY.md` itself** — one bullet per trap. + +`MEMORY.md` holds **the friction we've hit in this codebase and the workaround for each** — traps, gotchas, +non-obvious constraints an agent would pay to have known up front. + +**Write it at the moment of pain, not at the end of the task.** A workaround stays obvious for about five minutes; after that it stops feeling worth writing down, and you finish the task genuinely believing you hit nothing. Every entry that never got written was lost exactly that way. + +**The test for "is this friction?" is simple: did it cost you an attempt?** If you ran a command and it +failed, and the fix was knowledge rather than a code change — an env var, a flag, an ordering, a guard to +satisfy, a second run — that is friction, and the next agent will lose the same attempt unless you write it +down. Do not wait for something dramatic: the ordinary case is a one-line workaround you found in ninety +seconds and would otherwise forget by the end of the task. + +**Top rule — never include PII.** MEMORY is committed and world-readable: no real names, emails, secrets, +tokens, or machine paths in a trap — describe the failing thing generically, and refer to people by role. + +## A living list, not an archive + +- **When friction is genuinely solved — fixed in code, or made impossible by a guard or test — delete its + entry.** This is the one doc you *shrink* as the project matures. +- If a trap hardens into a permanent "never do X" rule, it graduates to `AGENTS.md` — move it, delete it here. +- Rely on an entry and find it no longer true? Fix or delete it in passing. + +## Scope: this codebase only + +Not user preferences, not how someone likes to work, not cross-project notes — those live in the agent's own +memory. `MEMORY.md` is about *this repo's* traps and nothing else. + +## Form + +- **One bullet each:** the trap, and the workaround or what to do instead. State the *why* in a clause. +- **Concrete, not vague.** Not "the X build is flaky" but "run `X` twice — the first run races the codegen + and fails intermittently". +- **Not a changelog.** Only what's *still* true and still bites. +- **Never record the absence of friction.** No "none hit this task", no dated all-clear. A task that + hit nothing leaves this file untouched; say so in your reply instead. +- **Only touch an entry to add, delete, or correct it.** Never reword an entry for its own sake. diff --git a/.knowledge/guides/docs-overview.md b/.knowledge/guides/docs-overview.md new file mode 100644 index 00000000..93714674 --- /dev/null +++ b/.knowledge/guides/docs-overview.md @@ -0,0 +1,206 @@ +# How to write an OVERVIEW — the standard + +This guide **is the standard** for `../OVERVIEW.md`. Shipped and versioned by `knowledge-template`. +**The template is `../OVERVIEW.md` itself** — start from it and fill each section. + +`OVERVIEW.md` is **the one doc written for a stakeholder**. Everything else in `.knowledge/` serves an +agent: `BRIEF.md` orients, `CODEMAP.md` locates, `prd/` contracts. None of them ever shows the platform. + +**Write it as if a product owner is reading it to understand the product they are about to sell.** It +answers what the parts are, how work flows between them, and what governs it. Someone should finish it able +to explain the product in a meeting without ever having opened the code. + +**Only what a customer could buy or touch.** No environments, no test strategy, no deployment, no internal +tooling — none of that is the product. An admin surface counts if it is part of the offering people +actually use. If a line would not survive being read aloud to a prospect, cut it. + +**Where an internal-sounding mechanism has a customer-facing consequence, name the consequence.** A new +seller being held in a sandbox is engineering's word for it; what a prospect needs to know is that *new +sellers cannot bill or deliver until they are approved*. Same rule, and only one of the two belongs here. + +## Say once that this is the design, not the inventory + +Every line below is written in the present tense — *a lead clears*, *an expired licence pauses the +agreement* — because that is how you describe a product. But a young platform's overview describes parts +that are built beside parts that are only agreed, and present tense flattens the two into one claim: **it +all works today.** In the one document that gets forwarded to a customer, that is the most expensive +sentence in the file, and nobody wrote it. + +The fix is not status on the parts — see below, it rots in a week and this is the file nobody re-reads. +**It is one line under the title, and the template carries it:** this describes the platform as designed, +and each contract's rows record what is proven today. Written once, it never goes stale, and it converts +every present tense underneath from a claim into a design. + +**Say where the line falls, or the disclaimer says nothing.** "Some of this may not be built" covers a +product that is five per cent done and one that is ninety-five per cent done equally, and a reader who +cannot tell which has been warned rather than informed. You do not fix that with a status column — you fix +it by naming **the two homes the links already point into**: ratified contracts in `prd/`, proposals in +`prd-drafts/`. That is not a build state anyone has to maintain, it is where the file sits, so it cannot go +stale; and a reader who notices every link going to one of them has learned the thing a percentage would +have told them. + +Tested: a reader given an overview whose parts were two-thirds unbuilt came away believing the whole +platform shipped. Nothing in the document was false. Nothing in it was honest either. Tested again after +the framing line: two cold readers both answered "no, and the document told me so" — and both then asked +for a status column, having worked the answer out for themselves from the link targets. + +## Name what the customer actually touches + +**A reader who finishes this should be able to picture using it.** The parts and the flow describe a +machine; they do not say whether a buyer signs in to a dashboard, an operator works a queue, or a seller +integrates once and never logs in again. That is the first thing a product owner needs and the last thing +an engineer thinks to write, because to them it is obvious. + +So: **one line per audience, naming the surface they touch and what they do there.** No screenshots, no +navigation, no feature list — the surface and its purpose. If an audience touches nothing (a party the +platform acts *on* rather than *for*), say that too; it is just as clarifying. + +**Surfaces, not modes.** The ban on environments applies here hardest, because this section invites the +leak: a sandbox, a staging tenant or a dry-run flag is a thing engineering built, not a thing a customer +buys. If it has a customer-facing consequence, that consequence belongs in `What governs it` as a rule — +never here as something you use. + +``` +✅ Buyers and sellers each get a workspace — set terms, watch what cleared, get paid. +✅ Sellers send traffic in through one integration; most never open the workspace day to day. +❌ The Vue SPA exposes participant and operator routes behind Inertia middleware. +❌ Dashboard with real-time analytics, reporting suite, notification centre, and more. +``` + +The first two tell a stranger what owning this product feels like. The third is written for the wrong +reader; the fourth is a brochure and says nothing. + +## It has to stand alone + +**This is the one doc that leaves the building.** It gets forwarded to a new hire, a candidate, an +investor, someone in sales — people with no access to `BRIEF.md` and no reason to want it. So it opens by +saying **what the product is, who it is for, and how it makes money**, in three sentences, before anything +else. + +Yes, that overlaps `BRIEF.md`. **Take the overlap.** "One fact, one home" protects facts that agents must +not restate in two places and let drift; three sentences of orientation on a document that is read by +strangers is the price of it being usable at all. Without them a reader can follow every step of the +machine and still not know what business they are looking at. + +Tested: handed this document alone, a reader worked out the mechanics correctly and still could not say +**how the product makes money, or which industry it serves.** Both were absent, and both are the first +things anyone asks. + +**If the commercial model isn't settled, write that** — one line, plainly. Never invent one, and never +leave the question unanswered by silence. + +**Every term you use in those three sentences must be findable below, or defined where you use it.** Name a +company, a surface or a mechanism the reader then cannot locate in the diagram or the list, and you have +manufactured a question instead of answering one. **This binds hardest on the revenue mechanism: if the +product makes money through something, that something is on the diagram** — whatever the box budget says. +A map that omits how the business earns is describing a machine, not a product. + +Tested: a reader given the opening alone asked what "the operating company" was and where "the open +exchange" lived, because neither appeared anywhere else in the document. Both were real parts of the +product; both had been left off the map. + +## Who it is for + +**A product manager, a founder, a marketer, a new joiner on their first day — not a developer.** Assume the +reader has never opened the repo, will never open the repo, and stopped reading at the first word they +didn't recognise. Every other doc here is for an engineer or an agent; this one is not, and writing it in +engineering vocabulary wastes the only artefact those readers have. + +**Use the words the business uses with its customers.** If a term appears in a sales conversation, a +pricing page or a support ticket, it belongs here. If it only appears in code, a protocol spec, or an +architecture discussion, replace it with what it *means* to someone using the product. + +``` +✅ Delivery — the buyer receives the lead in their own system, and we record whether it arrived +❌ FORM_POST delivery — posts the lead payload to the buyer's configured CRM endpoint + +✅ Price quote — a price we have offered, good for a short window before it expires +❌ Bid TTL envelope — a signed bid token with an expiry claim, required on post +``` + +Banned outright: file names, class and table names, HTTP verbs and status codes, protocol names, and any +acronym the business does not say out loud. Link the contract for anyone who wants the mechanism — +**the link is where technical detail lives, never the sentence.** + +The test before you ship it: **could someone in marketing read this aloud on a customer call without +stopping to ask what a word means?** If not, it isn't finished. + +**It is written, not generated.** A tool can list what exists and draw the links it finds; it cannot decide +what matters, what to leave out, or what to call things so a newcomer follows. That judgement is the entire +value here. **Distil — do not dump.** + +**Public and PII-free.** Assume a stranger reads it: no names, credentials, or machine paths. + +## The diagram is the point + +Everything else supports it. Rules, in order of how often they're broken: + +- **Group by component.** Use one `subgraph` per layer the project declared in + [`../prd/README.md`](../prd/README.md), in that order. The reader should see the system break into its + levels before reading a single label. +- **About fifteen boxes, hard.** Past that nobody reads it and nobody maintains it. If the platform has + forty parts, the diagram shows the fifteen that decide how it behaves — the rest live in + [`../prd/README.md`](../prd/README.md), which lists everything by design. +- **One box = one thing a *non-technical* person would name.** If a stakeholder has never said the word out + loud, it is implementation: rename it to what it does for the business, or leave it out. +- **Arrows are the point, not decoration.** An arrow means something real moves or depends: a request, a + record, an authorisation. A diagram of boxes with no arrows has said nothing — if you cannot draw the + arrows, you do not yet understand the platform, and that is worth saying out loud. +- **Point at the evidence for every arrow before you draw it** — the requirement, the field, the code path + that makes it true. **A plausible arrow is the most dangerous thing in this file:** a reader builds their + mental model from the picture and will never re-check it, and unlike a wrong sentence nobody proofreads a + wrong line. If two things merely *appear* related, leave the arrow out and ask. +- **No status in the diagram.** No suffixes, no dashed nodes, no styling for what does or doesn't exist + yet. The diagram answers one question — what the platform is and how work moves through it. +- **Label in the product's words**, matching each contract's `name:`, so a reader can jump from a box to + the contract that governs it. + +```mermaid +flowchart LR + subgraph Entity["Entities"] + a["Thing"] --> b["Other thing"] + end + subgraph Flow["Flows"] + b --> c["What happens when they meet"] + end +``` + +## Section guidance + +- **`The platform`** — the diagram, and nothing else. No prose above it; it should survive being screenshotted. +- **`How it works`** — every component in flow order, **one short line each — about fifteen words**: what + it is, and what it hands on. The whole list should be scannable in under a minute. Long paragraphs are the most common + failure here — a product owner skims this section, and a wall of prose gets skipped entirely. The + component is the unit: not a user story, not a sequence of API calls, the business naming its own parts. + + ``` + ✅ Source — where a seller's leads come from; feeds the lanes that price them. + ❌ Source — where a seller's leads come from. A seller registers one; it starts in a safe sandbox + until it is cleared for live traffic, and once live it feeds the lanes that price its leads. + ❌ The Source entity is provisioned in sandbox mode and validated against the ping taxonomy. + ``` + + The first is scannable. The second says more and communicates less, and drags in a sandbox, which is not + the product. The third is written for the wrong reader entirely. +- **`What you use`** — the surfaces, one line per audience, straight after `How it works`. Who signs in to + what, and who integrates once and never signs in. It is the shortest section and the one a product owner + reaches for first. +- **No build status anywhere in this file.** Not in the diagram, not in the list, not as an aside. Whether + something exists yet is the contracts' job — their glyph rows carry it, and where a contract lives says + the rest. Status written here is stale the week after, and it is not what this document is for. **The + framing line under the title is what makes this safe** — it says once that the file describes the design, + so no part has to carry its own disclaimer. Drop that line and the ban starts overclaiming on your behalf. +- **`What governs it`** — the rules that constrain the journey, and **who sets each one**: the terms both + sides agreed, the limits, the obligations that pause things when unmet. A stakeholder can watch a demo + and learn the flow; they cannot see the governance, which is exactly why it is here. + +## Keeping it true + +- **Update it when the shape changes** — a new system, a system removed, a flow re-routed. Not when a + requirement changes; that is the contract's job, and this file never restates a requirement. +- **Never copy requirement text here.** Link the contract instead. Copied rules go stale silently, and this + file is the one place nobody thinks to re-check. +- **If it disagrees with `prd/`, `prd/` wins** — and fix this file in the same task. +- **Where the content comes from:** the contracts in `prd/` and `prd-drafts/` (each one's *What this is*), + the components declared in `../prd/README.md`, and `CODEMAP.md` for what actually exists. Where those + leave the ordering ambiguous — and they usually do — **ask the owner rather than guessing a flow.** diff --git a/.knowledge/guides/docs-prd.md b/.knowledge/guides/docs-prd.md new file mode 100644 index 00000000..eb196be6 --- /dev/null +++ b/.knowledge/guides/docs-prd.md @@ -0,0 +1,345 @@ +# How to write a PRD — the standard + +This guide **is the standard** for every PRD in [`../prd/`](../prd/), with the template built in. It is +shipped and versioned by `knowledge-template` — do not rewrite it per project. The per-project **catalog** +(declared components + file list) lives in [`../prd/README.md`](../prd/README.md), not here. + +`../prd/` holds the **ratified contracts** for what the product does: one file per built system, every +requirement carrying an ID and a status — ✅ where a test proves it, ❌ where nothing does yet. A PRD +asserts *what must be true, why, and whether it is* — not how it's built. If a claim there has a ✅, a test +proves it. + +## The catalog is the product map + +A PRD answers for one system. **[`../prd/README.md`](../prd/README.md) answers for the product**: the +components in order, then every contract under its component with **a one-line gloss**. Someone new — human +or agent — reads that page top to bottom and knows what the product is made of before opening a single +file. + +The drafts have their own catalog, [`../prd-drafts/README.md`](../prd-drafts/README.md), in the same shape. +**Read together, the two catalogs are the whole system on one page** — what is ratified, and what is +proposed — which is why both are maintained by hand and both are linted for completeness. + +**A catalog holds what is durable: what exists, and what each one is.** Never how many rows are green. +That number moves with every test run, so it lives in the glyph column of the contract that owns it and +nowhere else — a count in a summary is stale the day after it is written, and unlike a stale row nobody +re-reads a summary to catch it. Give every Contents row its gloss; a bare filename list makes the source of +truth read worse than the proposals beside it. + +## A PRD asserts, it never explains + +If the file grows while the row count doesn't, you're writing prose instead of requirements. Every fact +has one home; the urge to explain has one, and it is not the PRD: + +| The urge | Where it goes | +|---|---| +| Explain how it's built | The design doc | +| Justify why it's built that way | The decision log | +| Say it isn't built, or isn't proven | The ❌ already says it | +| A requirement belongs in another file | **Move it** — a stop-and-ask, not a row | +| Undecided, so the requirement can't be written | `## Open questions`, one line | +| Something deliberately deferred | A `../prd-drafts/` stub — `id`, `name`, one sentence | +| Record what tests don't prove | A ❌ on the row it doesn't prove | +| Restate a rule another PRD owns | Cite its ID | +| Record a value or tunable | The header owns it — cite the symbol | + +If none of those fit, you're about to write something with no home. **Raise it, don't write it.** + +## Where a requirement lives + +Each project declares its own components (layers) in [`../prd/README.md`](../prd/README.md), **in order** — +domain ontology, not architecture. Two universal rules: + +1. **Layers are ordered; reading goes one way.** A lower layer may cite an upper one, never the reverse. +2. **Shared behavior goes up, never sideways.** If two entities obey the same rule, it belongs to their + base or the layer above — one ID, cited by both, never written twice. + +### Placement + +Read the ladder against **the components this project declared** — the shipped default is +`base-` / `entity-` / `flow-`, but a project may name its own, and the questions apply to whatever it +listed. Ask in order, **stop at the first yes**: + +1. True regardless of which kind of thing is involved? → it's about the substrate → the **lowest** layer. +2. True of exactly one kind of thing? → that kind's file, one layer up. +3. Needs two or more kinds interacting before it means anything? → the **emergent** layer above them. +4. About what appears on screen? → **not a PRD requirement.** Presentation. + +**A requirement never spans two layers.** If it seems to, it's two — split it. If it doesn't fit one layer +cleanly, **stop and ask.** + +## Naming + +### Files + +**One flat directory. The filename prefix is the layer** — never also a frontmatter field or a subdirectory. +Lowercase, hyphenated, one file per node. + +``` +prd/ + base-.md + entity-.md entity-.md + flow-.md +``` + +Entities are singular nouns. Flow files are noun phrases for the emergent thing — never a verb, question, +or computation (`-access` is a question the graph answers, not a thing). Every file matches a listed +component's prefix or the lint fails. No subdirectories; the only non-PRD file is `README.md`. + +### Namespaces and IDs + +- Form: `R--`. `` is declared once, in the file's frontmatter `id:`. +- **One namespace per file. One file per namespace.** A second namespace means a second file — as does a + table past ~15 rows. +- `` starts at one and runs unbroken. **A namespace's numbers never have a hole** — `doc-lint` fails on + a gap. A contract holds what is true now, so a withdrawn requirement is erased outright rather than kept + as struck history, and the rows below it renumber to close the gap. +- **Renumbering rewrites meaning, so it is never done alone.** `R-THING-7` erased means the old + `R-THING-8` becomes `R-THING-7` — and every citation of either now points somewhere new. Nothing will + tell you: the ID still resolves, the lint still passes, and the reader gets a confident wrong answer. + **Erasing a row is one change that also updates every citation of every ID at or after it** — in the + other contracts, and in the code comments that name their `R--`. Grep the whole repo for the + namespace before you erase, and if a citation is somewhere you cannot edit in the same change, **stop and + ask.** +- **Never compound.** `R-A-2 / R-B-2` is illegal — hoist it to a shared file, cited by both. + +## The template + +Copy this into every PRD. These headings, this order, every time. + +```md +--- +id: THING +name: Thing +last_verified: 2026-07-16 +--- + +## What this is + + + +## Why it exists + + + +- +- + +## Requirements + +| | ID | Requirement | Evidence | +|:--:|---|---|---| +| ✅ | R-THING-1 | | `` | +| ❌ | R-THING-2 | | src/thing.ext:88 — no test | +| ❌ | R-THING-3 | | — | + +## Open questions + + + +- +``` + +**The glyph is the first column and its header stays blank.** The schema is closed — the only `##` headings +are `What this is`, `Why it exists`, `Requirements`, `Open questions`. Changing it needs approval. + +An idea enters as a `../prd-drafts/` stub — two frontmatter lines and a sentence. **You do not write a PRD +to have an idea.** + +## Section guidance + +- **`What this is`** — what the thing is to someone who doesn't know the system, in the product's vocabulary. + Not what it owns or depends on. Can't say it in three sentences → it's two files. +- **`Why it exists`** — goals as **outcomes**: what is true for the user when this works. Every requirement + should serve one. +- **`Requirements`** — the document. Everything else is scaffolding. +- **`last_verified`** — the date **the whole file** was last read row-by-row against its tests. CI runs and + edits don't move it. Absent until the first review — that's what makes a file a draft. +- **`Open questions`** — design decisions not yet made, one line each. Not build gaps (a requirement that + exists and isn't proven is a ❌ row). + +## Writing a requirement + +Requirements come from the owner. **You are transcribing, not authoring** — the rewrite may tighten the +sentence and nothing else. + +- **Never add.** If the owner didn't say it, it isn't a requirement. +- **Never generalize.** "No recipients" is not "an invalid recipient set." +- **Never hedge.** "Cannot" does not become "should generally not." +- **Never turn behavior into mechanism** — *how* is the design doc's. +- **A gap is an open question, not a guess.** +- **Contradicts something already written? Stop and ask.** Never reconcile two of the owner's claims. + +### Splitting + +**Split where it would be built and tested separately.** *"A valid token is required; a missing token is +rejected as unauthorized; an out-of-scope token is refused as forbidden"* is three rows. (Note the status +*names*, not codes — a numeric literal in requirement text fails the lint.) Inverse: **one assertion proven +by several tests is still one requirement** — empty list, all-invalid, filtered-to-zero are situations, not +claims. + +### Every row stands alone + +**A requirement is read where it is cited, not where it sits.** Another contract cites `R-THING-4`; a +comment in the code names it; a reviewer opens the file at one line. In none of those does the row above it +come along. So a row that borrows its subject or its verb from its neighbour is unreadable exactly where it +matters most. + +This is what the word cap breaks first. An author trims to fit, the trimmed words are the ones the row +above already said, and the table still scans fine top-to-bottom — which is why nobody catches it. + +``` +✅ A seller-only participant cannot create a bid policy. +❌ Seller-only participants cannot. +❌ Illegal jumps are rejected. +❌ After the window it is allowed. +``` + +**The cap never buys ambiguity.** If the standalone sentence doesn't fit in 25 words, that is the format +telling you the row is two requirements — split it. Trimming the subject is not the way out. **But the cap +is only one of the two signals, and the weaker one:** a row can stand alone, fit in fifteen words, and +still assert three things joined by "and". That is the *one assertion* rule under [Form](#form), and it +catches what the cap never will. + +**A term the file itself defines is not a borrowed subject.** If a row enumerates the values or names the +thing, later rows may use that name bare — *"Exclusive enforces a single sale"* is complete, because +`Exclusive` is a value this file established. What must never be borrowed is a **pronoun**, a **bare +adjective standing in for its noun**, or an **elided verb**. Expanding every defined term back to its full +phrase is how a table stops being scannable, and the rule is not asking for that. + +Read every row cold, on its own, before you ship the file. Any row that raises *"which one?"*, *"cannot +what?"*, or *"what is allowed?"* is not finished. + +**Recovering a row that was already written this way.** The Evidence column is the anchor: open the test it +names, and it tells you the true subject and verb. Where there is no test — a `❌` row with `—` — you may +restore **only what an adjacent row states literally**, and nothing more. If the meaning needs a decision +rather than a lookup, that is authoring: leave the row alone and raise it. Recovering from a neighbour is +allowed here precisely because you are removing the dependency, not creating one. + +### Form + +- **No implementation symbols** — not a type, function, class, or file. +- **No infrastructure, and no vendor.** Not a database, connection, table, column, queue, cache, endpoint, + wire constant, or the name of a package you installed. These slip past the rule above because they are + none of those things, and they are how a contract quietly turns into a schema description. + + ``` + ✅ A trading organization has exactly one workspace. + ❌ Every participant is paired one-to-one with a Tenant sharing its UUID. + ❌ Marketplace tables stay on the central connection. + ``` + + **The test: would this row change if the storage, the framework, or the vendor were swapped out + tomorrow?** If yes, it is a design decision wearing a contract's clothes — the behavior it protects is the + requirement, and the mechanism belongs in the design doc. If the answer is that the row would become + *meaningless* rather than merely reworded, you have found a requirement with no product behind it: raise + it, don't write it. + + **When the interface *is* the deliverable, its vocabulary is product vocabulary.** For a system whose + product is an integration, the thing a customer buys is the endpoint, the field, the error they must + handle — that is the domain, and stripping it leaves nothing. The swap test still sorts it: the endpoint + survives a change of framework, so it stays; the auth scheme, the serialization format and the status + code do not, so they go. + +- **This binds the file's `name:` and its catalog gloss too**, not only the rows. A contract whose every + requirement is clean but which is *titled* after a wire constant is still named for its mechanism, and + the title is what a reader meets first. +- **Tunables by name, never by value** — and only where the name is already the owner's word. +- **One assertion.** "and", "also", or "except"? Suspect two. **A joining semicolon is rejected by the + lint** — it is the clearest sign a row is two requirements compressed to fit the word cap. Split it. +- **Under 25 words.** If it won't fit, it isn't atomic. +- **No numeric literal in the text** — name the tunable instead. +- Present tense, declarative, observable. +- **Cite IDs, never documents.** `R-OTHER-2` is an edge the lint sorts; "see the other doc" is invisible. + +**Evidence is the one place technical detail is allowed** — test names, `file:line`, paths. It answers +*where was this proven*. The row above it stays the owner's. + +**Cite the ID from the code, too.** The function or module that implements a requirement names its +`R--` in a comment or doc-block. Evidence points from the contract to the test; that comment points +from the code back to the contract — so a reader who lands anywhere in the loop can walk the other two. + +## Tests and status + +Every row answers **one** question: *does an automated test prove this?* ✅ or ❌. No third answer. + +- **❌ means failed, blocked, skipped, or not run.** Code that exists but nothing tests is ❌. +- **Evidence says which kind of ❌.** Test name → ✅. `file:line` + `— no test` → built, unproven. `—` → + nothing exists. **Evidence is empty iff nothing exists.** +- **Never write ✅ without naming the test that proves it.** Can't find one? The row is ❌. +- **A compile is not a test.** +- **Evidence names the test, never describes it** — a name a linter can assert exists. +- A **removed** requirement is deleted from the table, and the rows below it renumber to close the gap. A + contract states what must be true now; a withdrawn claim kept as struck history is stale information in + the one file that must be readable cold. The decision log is where a reversal is remembered — **erasing + the row is half the change, and updating every citation of it is the other half.** +- Untestable presentation requirements carry `Evidence: signoff:`. + +## Drafts and graduation + +Proposals live isolated in [`../prd-drafts/`](../prd-drafts/) until approved — see its README. + +- **A ratified PRD may never cite a draft.** Every ID a `prd/` file cites resolves inside `prd/`. The lint + enforces it. +- **Ratification is per file, not per row.** A draft graduates when the owner ratifies **the file's + claims — all of them**. Ratifying one new claim inside a draft adds a row *to the draft*; it does not + move the file. Graduating carries every other row into the source of truth on the owner's authority, so + if you can't tell whether they meant the file or the row, **stop and ask.** +- **The line between the two homes is approval, not proof.** A draft becomes a contract when the owner + ratifies its claims — not when its tests go green. **Proof is the glyph column**, and a ratified contract + is expected to carry ❌ rows: that is what `file:line — no test` exists to say. Waiting for an all-green + file leaves the source of truth empty and the real knowledge stranded in a home contracts may not cite. +- **Graduation is a move, not a rewrite.** `git mv ../prd-drafts/.md ./.md`; the IDs carry + across unchanged. The first conformance review then sets glyphs and stamps `last_verified`. + +## Refining + +- **Change only what the contract changed.** Edit a row when the owner's claim changed or the code drifted + — never to reword, and never a row the change doesn't touch. +- **The only writable surfaces are a table row, the three sentences, and the three bullets.** Never add prose. +- New behavior → **new row, new ID, appended.** Changed → **edit in place.** Removed → **erase the row, + renumber to close the gap, and update every citation in the same change.** +- Never restate a requirement that has an ID elsewhere — cite it. Never add a `##` heading. + +### Stop and ask + +Do not proceed if the requirement doesn't fit one layer; needs a new namespace, file, or component; would +move a requirement between files; would make this file cite one it never cited; would create a cycle; +would graduate a draft the owner ratified only one row of; would erase a requirement whose ID is cited +somewhere you cannot update in the same change; or contradicts an existing requirement. + +## Lint + +Mechanical, in CI (`doc-lint`). A red lint is a broken PRD, not a style note. **This list is exactly what +the linter checks** — every other rule in this guide is one you hold yourself to. + +- Every filename matches a component prefix listed in `../prd/README.md`. `prd/` and `prd-drafts/` are + **flat** — a subdirectory is rejected, because files inside one would escape every check here. +- One namespace per file; one file per namespace. Every ID matches its file's declared namespace, and the + namespace in `id:` is a single token. +- Every cited ID resolves; no ID is defined in two files. +- **A namespace's numbers run unbroken from one** — a gap means a row was erased without renumbering. +- **No `prd/` file cites a `prd-drafts/` id.** +- Every ✅ has something in Evidence. +- `last_verified` present iff the file has at least one ✅ row. +- **Citations only go up the stack**, and the citation graph is a DAG — a cycle *within* one layer fails too. +- No `##` outside the schema. No requirement over 25 words. No numeric literal in requirement text. +- **No semicolon in requirement text** — that's two requirements in one row. +- **The catalog is well-formed** — `../prd/README.md` has a Components list (`prefix — gloss`) and a + Contents list naming every PRD. Catalogs are **maintained by hand**; nothing regenerates them. Add the + row in the same task you add the file, or the build goes red. + +### What the lint cannot check — so you must + +The shipped linter is stack-neutral: it has no idea what your test runner is. So **a ✅ is only ever checked +for *having* Evidence — never for that test existing.** A plausible-looking name that matches nothing in the +suite passes forever, and the one claim this whole format rests on quietly becomes untrue. + +If ✅ is going to mean anything here, close that gap yourself: pull the backticked Evidence names out of +`../prd/*.md`, list your suite's test names, and fail on any Evidence name the suite doesn't contain. Wire it +into the same gate as `doc-lint` — it is a handful of lines, and it is the difference between a contract and +a claim. + +Two more the linter can't see, and therefore owns nothing of: that Evidence is empty **only** where nothing +exists (a `—` on code that does exist is a lie the lint will never catch), and that a requirement is the +owner's claim rather than one you generalized. diff --git a/.knowledge/guides/docs-research.md b/.knowledge/guides/docs-research.md new file mode 100644 index 00000000..06069c4b --- /dev/null +++ b/.knowledge/guides/docs-research.md @@ -0,0 +1,117 @@ +# How to write a research note — the standard + +This guide **is the standard** for every note in [`../research/`](../research/), with the template built in. +Shipped and versioned by `knowledge-template`; the per-project catalog of notes lives in +[`../research/README.md`](../research/README.md). + +A research note is **a dated report on how the world works outside this codebase**, written so we can decide +something. It answers one question, cites what it learned from, and says what we're doing about it. It is +**input, never truth**: it informs a draft, never dictates one, and is never the source of record for our +behavior (that's `../prd/`). Answers: *"What has the world already done about ``, and what are we aiming +at?"* + +## Report the world, then say what we think — never both at once + +`What they do` is **reporting**; Borrow, Avoid, and Verdict are **ours**. Keep them separate: our thinking +inside the reporting section becomes a borrow, then a verdict, then a requirement. The mechanism is +**sourcing at the point of claim** — every non-obvious claim in `What they do` carries a `[n]`, so anything +uncited is visibly ours. + +| The urge | Where it goes | +|---|---| +| Say what they should have done | `What to avoid` | +| Say what we'd do about it | `Verdict for us` | +| State our own design position | `Verdict for us` — **never** inside `What they do` | +| Say what the product must do | Nowhere here. Tell the owner. | +| Say what you couldn't find out | `Open questions`, one line | +| Say the world has changed since | Re-read the note whole and re-stamp it | + +## The template + +```md +# Research: + +**Last updated:** 2026-07-16 +**Question it answers:** + +## What they do + + + +## What we can borrow + + + +## What to avoid + + + +## Verdict for us + + + +## Open questions + + + +## Sources + +1. +2. +``` + +**Do not add a heading.** The schema is closed. + +## Section guidance + +- **`Question it answers`** — one line, a scope contract. Anything that doesn't serve it is a second note. +- **`What they do`** — the world, as sources describe it. Structure it however the topic wants. Absolute + rule: **nothing of ours goes here.** +- **`What we can borrow`** — transferable ideas as short bullets. Ideas, not decisions. +- **`What to avoid`** — the traps, theirs and ours. +- **`Verdict for us`** — **the decision, not a recap.** Chooses: this now, that deferred, that not at all, + and names the PRD or component it feeds. +- **`Open questions`** — one line each. Honest to have many. + +## Sourcing + +- **Every non-obvious claim carries `[n]`** — every number, date, "the industry does X", legal posture. +- **Keep the real link.** Every `## Sources` entry carries the actual URL you read — one a reader can open + and verify. A name with no link isn't checkable. +- **Cite the specific page you read, not the vendor.** A source blob at the top is not sourcing. +- **Never cite a source you didn't open.** + +## How old is it + +**`Last updated` is the date the whole note was last re-read against its sources** — not when the file was +touched. Updating means re-reading, not patching: fix one section and re-stamp and you make stale paragraphs +look fresh. Change a note only when its sources actually moved (or the owner asks) — never to reword, and +never a section the change doesn't touch. + +## What a note is not + +Nothing in it is binding. The path is always: + +``` +research → the owner reads it → the owner states a requirement → a prd/ row +``` + +Never `research → requirement`. Screenshots, competitor captures, and UI targets go in the sibling +[`../references/`](../references/) home — cite them from a note, don't embed the analysis there. + +## Lint + +Mechanical, in CI (`doc-lint`). **This list is exactly what the linter checks** — every other rule in this +guide is one you hold yourself to. + +- The note carries a `**Last updated:**` date. +- Every `[n]` resolves to a Sources entry, and every Sources entry is cited at least once. +- **Every Sources entry carries a link** (`http…`), or is marked `(internal)` when it isn't a public + page. A bare vendor name is not a source — nobody can check it, including you, later. +- No `##` outside the schema. +- The note has a row in [`../research/README.md`](../research/README.md). The catalog is **maintained by + hand** — nothing regenerates it; add the row in the same task you add the note. + +What it can't check, and you own: that the link goes where you say, that you actually opened it, and that +nothing of ours crept into `What they do`. A URL that resolves is not the same as a source that supports +the claim — the lint buys you a shape, not honesty. diff --git a/.knowledge/prd-drafts/README.md b/.knowledge/prd-drafts/README.md new file mode 100644 index 00000000..5f646c27 --- /dev/null +++ b/.knowledge/prd-drafts/README.md @@ -0,0 +1,11 @@ +# prd-drafts/ — proposals, isolated (catalog) + +Draft PRDs not yet approved; a `../prd/` contract may never cite one (`doc-lint` enforces the isolation). **To write or modify one, follow [`../guides/docs-prd.md`](../guides/docs-prd.md).** + +## Drafts — maintained by hand + +Add a row when you add a draft; `doc-lint` fails the build if one is missing. + +| Draft | Proposes | Reserved namespace | +|---|---|---| +| _(no drafts yet)_ | | | diff --git a/.knowledge/prd/README.md b/.knowledge/prd/README.md new file mode 100644 index 00000000..26f306ca --- /dev/null +++ b/.knowledge/prd/README.md @@ -0,0 +1,31 @@ +# prd/ — tested contracts (catalog) + +The ratified, test-backed PRDs — one per built system. **To write or modify one, follow [`../guides/docs-prd.md`](../guides/docs-prd.md).** + +## Components — authored + +The project's ontology, in order (`prefix — gloss`). The owner's call; an agent stops and asks. + +``` +1. base- — config, service provider, HTTP transport, driver base +2. provider- — per-vendor clients and shared driver behavior +3. modality- — text, image, audio, voice, embeddings, rerank, moderation, batch +4. flow- — tool loop, streaming, sub-agents, similarity search +``` + +## Contents — maintained by hand + +Add a row when you add a PRD; `doc-lint` fails the build if one is missing. Component, then file, each row +a link to the contract followed by a one-line gloss of what it is. Read top to bottom, this list is the +product's high-level map. + +No contracts are ratified yet — the ontology above is declared, but `prd/` holds no PRD files. + +- **Base** + - _(no PRDs yet)_ +- **Provider** + - _(no PRDs yet)_ +- **Modality** + - _(no PRDs yet)_ +- **Flow** + - _(no PRDs yet)_ diff --git a/.knowledge/references/README.md b/.knowledge/references/README.md new file mode 100644 index 00000000..4106013e --- /dev/null +++ b/.knowledge/references/README.md @@ -0,0 +1,17 @@ +# references/ — visual targets (catalog) + +Screenshots, competitor captures, and UI to match. Sibling of [`../research/`](../research/) (analysis vs. targets). **To add one: see below.** + +## Sets — maintained by hand + +Add a row when you add a set; `doc-lint` fails the build if one is missing. One row per set (a subfolder). + +| Set | Competitor / platform | What we're comparing | +|---|---|---| +| _(no reference sets yet)_ | | | + +## How to add one + +- **Organize by competitor or platform first** — a folder per source (`stripe/`, `ios/`, `figma/`), then by + what you're comparing inside it (a screen, a flow, a style). +- A `NOTES.md` in each set: what inspires, what to avoid, provenance. PII-free; respect licensing. diff --git a/.knowledge/research/README.md b/.knowledge/research/README.md new file mode 100644 index 00000000..b168ac24 --- /dev/null +++ b/.knowledge/research/README.md @@ -0,0 +1,11 @@ +# research/ — prior art (catalog) + +Dated notes on how others solved a problem, each answering one question. **To write or modify one, follow [`../guides/docs-research.md`](../guides/docs-research.md).** Visual targets: sibling [`../references/`](../references/). + +## Notes — maintained by hand + +Add a row when you add a note; `doc-lint` fails the build if one is missing. Newest `Last updated` first. + +| Last updated | Note | Question it answers | +|---|---|---| +| _(no notes yet)_ | | | diff --git a/.knowledge/scripts/README.md b/.knowledge/scripts/README.md new file mode 100644 index 00000000..c2411fbd --- /dev/null +++ b/.knowledge/scripts/README.md @@ -0,0 +1,42 @@ +# scripts/ — tooling & your workspace + +Code, not prose — stack-neutral, standard-library only, runs with nothing installed. + +The **shipped tooling** below is versioned by `knowledge-template`: don't rewrite it per project, it updates +by version bump. Everything else you drop in here is yours. + +## Contents + +| Script | What it does | +|---|---| +| [`doc-lint`](./doc-lint) | Enforces the standard on `.knowledge/` — namespaces, IDs, citations, glyph tables, catalogs, research notes, and the payload's own integrity. A red lint is a broken doc. | +| [`test_doc_lint.py`](./test_doc_lint.py) | The linter's teeth-test: a valid project passes, each mutation fails on its own rule. If this fails, the linter has lost a tooth. | + +## Usage + +``` +python3 .knowledge/scripts/doc-lint .knowledge # lint this project's docs +python3 .knowledge/scripts/test_doc_lint.py # prove the linter still works +``` + +Wire both into CI. A rule that matters is a check in `doc-lint`, teeth-tested beside it — prose that isn't +enforced is teaching, not law. + +## Payload integrity + +`../.payload-manifest` records a `sha256` for every file this project must never edit — the `docs-*.md` +standards and the two scripts above. `doc-lint` re-hashes them on every run, so a repo can **prove** it is +running the version stamped in `../.version` instead of taking it on trust. A drifted file is named in the +failure: restore it from that release, or upgrade the whole `.knowledge/` and re-stamp the version. + +**Do not hand-edit the manifest.** Only the upstream release step rewrites it, after changing the payload: + +``` +python3 .knowledge/scripts/doc-lint --write-manifest .knowledge # re-record the checksums (upstream only) +``` + +## Your scripts + +This folder is also a workspace. Add project-specific helper scripts here — generators, build steps, one-off +checks, anything that helps an agent work in this repo. Keep the shipped `doc-lint` and `test_doc_lint.py` +unchanged (they're versioned); everything else here is yours to add and name. diff --git a/.knowledge/scripts/doc-lint b/.knowledge/scripts/doc-lint new file mode 100755 index 00000000..87e31fe1 --- /dev/null +++ b/.knowledge/scripts/doc-lint @@ -0,0 +1,590 @@ +#!/usr/bin/env python3 +"""doc-lint — enforces the .knowledge/ documentation standard (see guides/docs-prd.md). + +Ships inside the copied payload (.knowledge/scripts/), so every project runs the same version. +Zero dependencies: standard library only. + + python3 .knowledge/scripts/doc-lint [PATH_TO_.knowledge] # lint (defaults to the parent .knowledge/) + python3 .knowledge/scripts/doc-lint --payload # an unadopted payload: skip the AGENTS.md check + python3 .knowledge/scripts/doc-lint --write-manifest # re-record the payload integrity manifest + +Its teeth-test lives beside it: python3 .knowledge/scripts/test_doc_lint.py +Exits non-zero if any rule is broken. A red lint is a broken doc, not a style note. +""" +import hashlib +import os +import re +import sys + +R_ID = re.compile(r"R-[A-Z0-9]+-\d+") +R_ID_NS = re.compile(r"R-([A-Z0-9]+)-\d+") +R_ID_NUM = re.compile(r"R-[A-Z0-9]+-(\d+)") +WORD = re.compile(r"\S+") +NUMERIC = re.compile(r"(?", "_(none)_"], "CODEMAP.md": ["", "_(none)_"], + "OVERVIEW.md": ["", " [prefix,...] in order.""" + path = os.path.join(self.root, "prd", "README.md") + if not os.path.isfile(path): + return [] + text = self.read(path) + m = re.search(r"## Components.*?```(.*?)```", text, re.S) + prefixes = [] + if m: + for line in m.group(1).splitlines(): + cm = re.match(r"\s*\d+\.\s+(\S+?)\s+—\s+.+", line) + if cm: + prefixes.append(cm.group(1)) + elif line.strip() and not line.strip().startswith("#"): + self.err("prd/README.md", f"Components line not `N. prefix — gloss`: {line.strip()}") + return prefixes + + # ---- checks ----------------------------------------------------------- + def check_manifest(self): + """Every shipped directory and file must still exist — catch removals/renames.""" + for d in REQUIRED_DIRS: + if not os.path.isdir(os.path.join(self.root, d)): + self.err("manifest", f"required directory `{d}/` is missing (removed or renamed?)") + for fp in REQUIRED_FILES: + if not os.path.isfile(os.path.join(self.root, fp)): + self.err("manifest", f"required file `{fp}` is missing (removed or renamed?)") + + def stamped_version(self): + path = os.path.join(self.root, ".version") + return self.read(path).strip() if os.path.isfile(path) else "unknown" + + def check_integrity(self): + """Every versioned file still hashes to what the shipped manifest records. + + This is the only check an adopting repo can run without a copy of the upstream to diff against: + the checksums travel inside the payload, so a project can prove it runs the version it claims.""" + path = os.path.join(self.root, MANIFEST) + version = self.stamped_version() + if not os.path.isfile(path): + self.err(MANIFEST, f"the payload integrity manifest is missing, so nothing can prove this " + f"`.knowledge/` is the version `.version` claims ({version}) — restore it " + f"from that release, or regenerate it with `--write-manifest`") + return + recorded = {} + for n, raw in enumerate(self.read(path).splitlines(), 1): + line = raw.strip() + if not line or line.startswith("#"): + continue + parts = line.split(None, 1) + if len(parts) != 2 or len(parts[0]) != 64: + self.err(MANIFEST, f"line {n} is not ` `: {line}") + continue + recorded[parts[1].strip()] = parts[0] + drifted = [] + for rel in sorted(recorded): + target = os.path.join(self.root, rel) + if not os.path.isfile(target): + self.err(MANIFEST, f"records `{rel}`, which is not in this `.knowledge/` " + "(a versioned file was deleted or renamed)") + elif sha256_file(target) != recorded[rel]: + drifted.append(rel) + for rel in VERSIONED_FILES: + if rel not in recorded: + self.err(MANIFEST, f"does not cover `{rel}` — a versioned file with no recorded checksum " + "is unguarded (regenerate the manifest with `--write-manifest`)") + if drifted: + self.err(MANIFEST, "these versioned files no longer match the checksums recorded for version " + f"{version}: " + ", ".join(f"`{r}`" for r in drifted) + + " — the docs tooling has drifted from the version `.version` claims. They " + "are owned by knowledge-template and are never edited in place: restore " + "them from that release, or upgrade the whole `.knowledge/` and re-stamp " + "`.version`.") + + def check_overview(self): + """OVERVIEW.md is schema-closed, same as a PRD: a fixed set of headings, in a fixed order, under + the framing line that says the file describes the design rather than what is built today.""" + path = os.path.join(self.root, "OVERVIEW.md") + if not os.path.isfile(path): + return # manifest already flags a missing OVERVIEW.md + text = self.read(path) + preamble = text.split("\n## ", 1)[0] + lower = preamble.lower() + if not ("prd/" in preamble and "prd-drafts/" in preamble + and any(w in lower for w in OVERVIEW_FRAMING_WORDS)): + self.err("OVERVIEW.md", "the framing line under the title is gone — say once that this " + "describes the platform as designed, and name both `prd/` and " + "`prd-drafts/` so a reader can see where the line falls") + found = self.headings(text) + for h in found: + if h not in OVERVIEW_HEADINGS: + self.err("OVERVIEW.md", f"`## {h}` is not in the closed schema") + for h in OVERVIEW_HEADINGS: + if h not in found: + self.err("OVERVIEW.md", f"missing required section `## {h}`") + kept = [h for h in found if h in OVERVIEW_HEADINGS] + if sorted(kept) == sorted(OVERVIEW_HEADINGS) and kept != OVERVIEW_HEADINGS: + self.err("OVERVIEW.md", "sections are out of order — the schema reads " + + " -> ".join(OVERVIEW_HEADINGS)) + + def check_trio(self): + """BRIEF/CODEMAP/MEMORY must end with their edit-gated standard pointer.""" + for fname, guide in TRIO.items(): + path = os.path.join(self.root, fname) + if not os.path.isfile(path): + continue # manifest already flags a missing trio file + nonempty = [l for l in self.read(path).splitlines() if l.strip()] + last = nonempty[-1] if nonempty else "" + if "Editing this file" not in last or guide not in last: + self.err(fname, f"must end with its standard pointer to guides/{guide} (last line)") + + def check_root(self): + """Only the known top-level files and homes may sit at the .knowledge/ root.""" + for name in sorted(os.listdir(self.root)): + if name not in ALLOWED_ROOT: + self.err(name, "unexpected entry at the .knowledge/ root (only shipped files and homes allowed)") + + def check_guide_links(self): + """Each home catalog must link to the guide for writing or modifying its docs.""" + for rel, guide in CATALOG_GUIDE.items(): + path = os.path.join(self.root, rel) + if not os.path.isfile(path): + continue # manifest flags a missing catalog + if f"guides/{guide}" not in self.read(path): + self.err(rel, f"catalog must link to `../guides/{guide}` (write or modify)") + + def check_flat(self): + """prd/ and prd-drafts/ are one flat directory — a subdirectory hides its files from every check.""" + for home in ("prd", "prd-drafts"): + d = os.path.join(self.root, home) + if not os.path.isdir(d): + continue # manifest flags a missing home + for name in sorted(os.listdir(d)): + if os.path.isdir(os.path.join(d, name)): + self.err(f"{home}/README.md", + f"`{name}/` is a subdirectory — {home}/ is flat, and nothing inside it is linted") + + def check_filled(self): + """The orientation docs must actually be written — shipped placeholders may not survive adoption.""" + for fname, markers in PLACEHOLDERS.items(): + path = os.path.join(self.root, fname) + if not os.path.isfile(path): + continue # manifest flags a missing trio file + text = self.read(path) + for marker in markers: + if marker in text: + self.err(fname, f"still contains the shipped placeholder `{marker}` — fill this in " + "(or delete the section) before calling the project adopted") + + def check_agents(self): + """The repo root must have an AGENTS.md that still routes an agent to the orientation trio.""" + path = os.path.join(os.path.dirname(self.root), "AGENTS.md") + if not os.path.isfile(path): + self.err("AGENTS.md", "the repo root has no AGENTS.md (the entry point into .knowledge/)") + return + text = self.read(path) + missing = [f for f in AGENTS_TRIO if f".knowledge/{f}" not in text] + if missing: + self.err("AGENTS.md", "must load the orientation trio — no reference to " + ", ".join(missing)) + if "guides/docs-agents.md" not in text: + self.err("AGENTS.md", "must point at `.knowledge/guides/docs-agents.md` so a revision is " + "routed to the standard first (same edit gate as the trio)") + + def check_contiguous(self, path, numbers): + """A namespace's numbers run from 1 with no gaps. A gap means a row was deleted without + renumbering, or mis-numbered from the start — either way an ID that resolves nowhere.""" + for ns, seen in sorted(numbers.items()): + missing = [n for n in range(1, max(seen) + 1) if n not in seen] + if missing: + gaps = ", ".join(f"R-{ns}-{n}" for n in missing) + self.err(path, f"namespace `{ns}` skips {gaps} (a namespace numbers as an unbroken run " + "— erasing a requirement means renumbering to close the gap)") + + def check(self): + self.check_manifest() + self.check_integrity() + self.check_overview() + self.check_trio() + self.check_root() + self.check_flat() + self.check_guide_links() + if not self.payload: + self.check_agents() + self.check_filled() + prefixes = self.components() + rank = {p: i for i, p in enumerate(prefixes)} + prd = self.collect_prd("prd") + drafts = self.collect_prd("prd-drafts") + + defined = {} + draft_ids = set() + ns_owner = {} + + for scope, files in (("prd", prd), ("prd-drafts", drafts)): + for path, (fm, body) in files.items(): + fid = fm.get("id") + if not fid: + self.err(path, "missing frontmatter `id:`") + if not fm.get("name"): + self.err(path, "missing frontmatter `name:`") + if fid: + if fid in ns_owner: + self.err(path, f"namespace `{fid}` is already owned by {ns_owner[fid]} " + "(one file per namespace)") + else: + ns_owner[fid] = path + rows = list(self.table_rows(body)) + has_pass = False + numbers = {} # namespace -> the requirement numbers this file assigns + + base = os.path.basename(path) + if scope == "prd" and prefixes and not any(base.startswith(p) for p in prefixes): + self.err(path, f"filename prefix not a listed component: {base}") + + for glyph, rid, req, evid in rows: + ns = R_ID_NS.match(rid).group(1) + if fid and ns != fid: + self.err(path, f"{rid} does not match file namespace `{fid}` (one namespace per file)") + if rid in defined: + self.err(path, f"{rid} already defined in {defined[rid]} (no ID in two files)") + else: + defined[rid] = path + if scope == "prd-drafts": + draft_ids.add(rid) + numbers.setdefault(ns, set()).add(int(R_ID_NUM.match(rid).group(1))) + if glyph == "✅": + has_pass = True + if not evid or evid == "—": + self.err(path, f"{rid} is ✅ but names no test in Evidence") + elif glyph != "❌": + self.err(path, f"{rid} has invalid glyph `{glyph}` (only ✅/❌)") + words = WORD.findall(req) + if len(words) > 25: + self.err(path, f"{rid} requirement is {len(words)} words (max 25)") + if NUMERIC.search(req): + self.err(path, f"{rid} requirement has a numeric literal (name the tunable instead)") + if ";" in req: + self.err(path, f"{rid} requirement joins two assertions with a semicolon (split it)") + + self.check_contiguous(path, numbers) + + allowed = {"What this is", "Why it exists", "Requirements", "Open questions"} + for h in self.headings(body): + if h not in allowed: + self.err(path, f"`## {h}` is not in the closed schema") + if "/" in (fid or "") or " " in (fid or ""): + self.err(path, "compound/invalid namespace in `id:`") + + lv = "last_verified" in fm + if scope == "prd": + if has_pass and not lv: + self.err(path, "has a ✅ row but no `last_verified:`") + if lv and not has_pass: + self.err(path, "has `last_verified:` but no ✅ row") + + edges = {} + for path, (fm, body) in prd.items(): + fid = fm.get("id") + cited = {rid for rid in R_ID.findall(body) if R_ID_NS.match(rid).group(1) != fid} + for rid in sorted(cited): + if rid in draft_ids: + self.err(path, f"cites {rid} which is a prd-drafts/ proposal (contracts never cite drafts)") + elif rid not in defined: + self.err(path, f"cites {rid} which resolves nowhere") + else: + dep = defined[rid] + if dep != path: + edges.setdefault(path, set()).add(dep) + if prefixes: + dp = next((p for p in prefixes if os.path.basename(dep).startswith(p)), None) + sp = next((p for p in prefixes if os.path.basename(path).startswith(p)), None) + if dp is not None and sp is not None and rank[dp] > rank[sp]: + self.err(path, f"cites {rid} in a later layer `{dp}` (citations only go up the stack)") + + cycle = find_cycle(edges) + if cycle: + self.err(cycle[0], "citation cycle: " + " -> ".join(cycle) + " (the graph must be a DAG)") + + self.check_sections() + self.check_catalog_complete() + self.check_catalog_links() + self.check_research() + return not self.errors + + def check_sections(self): + """Every shipped README keeps its required sections — a gutted catalog is drift.""" + for rel, sections in REQUIRED_SECTIONS.items(): + path = os.path.join(self.root, rel) + if not os.path.isfile(path): + continue # manifest flags a missing README + headings = [l for l in self.read(path).splitlines() if l.startswith("## ")] + for sec in sections: + if not any(h.startswith(sec) for h in headings): + self.err(rel, f"README missing required section `{sec}`") + + def check_catalog_complete(self): + """Every doc in a home is listed in that home's catalog — a catalog behind the tree misleads.""" + for home, kind in CATALOG_CONTENTS.items(): + d = os.path.join(self.root, home) + readme = os.path.join(d, "README.md") + if not os.path.isdir(d) or not os.path.isfile(readme): + continue # manifest flags a missing home or catalog + text = self.read(readme) + for name in sorted(os.listdir(d)): + if name == "README.md" or name.startswith("."): + continue + is_dir = os.path.isdir(os.path.join(d, name)) + if is_dir != (kind == "dir") or (kind == "md" and not name.endswith(".md")): + continue + if name not in text: + self.err(f"{home}/README.md", f"catalog omits {name}") + + def check_catalog_links(self): + """Every link in every home catalog must resolve to a real file or directory.""" + for home in ["prd", "prd-drafts", "research", "references", "guides"]: + readme = os.path.join(self.root, home, "README.md") + if not os.path.isfile(readme): + continue # manifest flags a missing catalog + for target in re.findall(r"\]\(([^)]+)\)", self.read(readme)): + raw = target.strip() + if raw.startswith(("http", "#", "mailto:")) or "://" in raw: + continue + rel = raw.split()[0].split("#", 1)[0] # drop any title / anchor + if not rel: + continue + if not os.path.exists(os.path.normpath(os.path.join(self.root, home, rel))): + self.err(f"{home}/README.md", f"catalog links to missing `{rel}`") + + def check_research(self): + d = os.path.join(self.root, "research") + if not os.path.isdir(d): + return + for name in sorted(os.listdir(d)): + if not name.endswith(".md") or name == "README.md": + continue + path = os.path.join("research", name) + text = self.read(os.path.join(d, name)) + if "**Last updated:**" not in text: + self.err(path, "research note missing `**Last updated:**`") + + for h in self.headings(text): + if h not in RESEARCH_HEADINGS: + self.err(path, f"`## {h}` is not in the closed schema") + + cites = set(re.findall(r"\[(\d+)\]", text)) + body = text.split("## What they do", 1) + if len(body) > 1 and not re.search(r"\[\d+\]", body[1].split("\n## ", 1)[0]): + self.err(path, "`What they do` cites no source — reporting with no citation is assertion") + sources = {} + if "## Sources" in text: + tail = text.split("## Sources", 1)[1].split("\n## ", 1)[0] + sources = {n: line for n, line in re.findall(r"^\s*(\d+)\.\s*(.*)$", tail, re.M)} + for n in sorted(cites - set(sources), key=int): + self.err(path, f"citation [{n}] has no Sources entry") + for n, line in sorted(sources.items(), key=lambda kv: int(kv[0])): + if "http" not in line and "(internal" not in line: + self.err(path, f"source [{n}] carries no link — give the URL you opened, " + "or mark it `(internal)` if it isn't a public page") + if n not in cites: + self.err(path, f"source [{n}] is never cited (every source earns its place)") + + + +def main(): + args = [a for a in sys.argv[1:] if not a.startswith("--")] + flags = [a for a in sys.argv[1:] if a.startswith("--")] + if [f for f in flags if f not in ("--payload", "--write-manifest")] or len(args) > 1: + print("usage: doc-lint [--payload | --write-manifest] [PATH_TO_.knowledge]") + return 2 + here = os.path.dirname(os.path.abspath(__file__)) # .knowledge/scripts + root = os.path.abspath(args[0]) if args else os.path.dirname(here) + if "--write-manifest" in flags: + return write_manifest(root) + lint = Lint(root, payload="--payload" in flags) + if lint.check(): + print(f"doc-lint: OK ({root})") + return 0 + for where, msg in lint.errors: + print(f"doc-lint: {where}: {msg}") + print(f"\ndoc-lint: {len(lint.errors)} problem(s)") + return 1 + + +if __name__ == "__main__": + sys.exit(main()) diff --git a/.knowledge/scripts/test_doc_lint.py b/.knowledge/scripts/test_doc_lint.py new file mode 100755 index 00000000..bf99c0fa --- /dev/null +++ b/.knowledge/scripts/test_doc_lint.py @@ -0,0 +1,483 @@ +#!/usr/bin/env python3 +"""Teeth-test for doc-lint — ships in the payload beside the linter. + +The valid base is a *copy of the real payload* with two sample PRDs added, so the +structural checks (every shipped dir/guide present, the trio's standard pointer) are +exercised against the actual shipped tree. Each mutation must fail on its own rule. + + python3 .knowledge/scripts/test_doc_lint.py +""" +import os +import shutil +import subprocess +import sys +import tempfile + +HERE = os.path.dirname(os.path.abspath(__file__)) +DOC_LINT = os.path.join(HERE, "doc-lint") +PAYLOAD = os.path.dirname(HERE) # the real .knowledge/ this script ships in + +CATALOG = """# prd/ — catalog + +To write or modify one, follow [../guides/docs-prd.md](../guides/docs-prd.md). + +## Components + +``` +1. base- — the substrate +2. entity- — a placed thing +``` + +## Contents + +- **Base** + - [base-core](./base-core.md) +- **Entities** + - [entity-widget](./entity-widget.md) +""" +BASE = """--- +id: CORE +name: Core +last_verified: 2026-07-16 +--- + +## What this is + +The substrate. + +## Requirements + +| | ID | Requirement | Evidence | +|:--:|---|---|---| +| ✅ | R-CORE-1 | The substrate persists between runs | `coreHolds` | +""" +WIDGET = """--- +id: WIDGET +name: Widget +--- + +## What this is + +A widget, built on R-CORE-1. + +## Requirements + +| | ID | Requirement | Evidence | +|:--:|---|---|---| +| ❌ | R-WIDGET-1 | A widget reports its state on demand | — | +""" + + +# Minimal catalogs for the homes we clear — a real project's catalogs link to the files we just +# deleted, so they must be reset too or the fixture fails on its own dangling links. +# The fixture is a *populated* project, not an empty one: a draft, a research note, a reference set and +# a project how-to. Rules that only bite on a populated home stayed untested for a long time — both bugs +# found by piloting on a real repo were of exactly that shape. +DRAFT = """--- +id: FUTURE +name: Future thing +--- + +## What this is + +Something proposed, not yet ratified. + +## Requirements + +| | ID | Requirement | Evidence | +|:--:|---|---|---| +| \u274c | R-FUTURE-1 | A future thing is proposed | \u2014 | +""" +NOTE = """# Research: how others do it + +**Last updated:** 2026-07-16 +**Question it answers:** what shape does the rest of the world use + +## What they do + +They do a thing [1], and we have seen it ourselves [2]. + +## Verdict for us + +Borrow the shape. + +## Sources + +1. A public page — https://example.com/a-page +2. Our own walkthrough (internal) +""" +HOWTO = """# How to do the recurring thing + +The steps, in order, with the commands. + +1. Run the first command. +2. Check it worked. +""" +STUB_CATALOGS = { + "prd-drafts/README.md": """# prd-drafts/ — proposals (catalog) + +Isolated until approved. To write one, follow [../guides/docs-prd.md](../guides/docs-prd.md). + +## Drafts + +| Draft | Proposes | Reserved namespace | +|---|---|---| +| [entity-future](./entity-future.md) | a future thing | FUTURE | +""", + "research/README.md": """# research/ — prior art (catalog) + +To write one, follow [../guides/docs-research.md](../guides/docs-research.md). + +## Notes + +| Last updated | Note | Question it answers | +|---|---|---| +| 2026-07-16 | [how-others-do-it](./how-others-do-it.md) | what shape the world uses | +""", + "references/README.md": """# references/ — visual targets (catalog) + +## Sets + +| Set | Competitor / platform | What we're comparing | +|---|---|---| +| [acme](./acme/) | Acme | their onboarding | + +## How to add one + +- One folder per source. +""", +} + +BRIEF = """# Brief — Sample Project + +## Story + +A sample project, filled in as an adopted repo would be. + +--- +*Editing this file? Follow the standard first: [`guides/docs-brief.md`](./guides/docs-brief.md).* +""" +CODEMAP = """# Codemap — Sample Project + +## Entry Points + +- `src/main.ext` — where it starts. + +--- +*Editing this file? Follow the standard first: [`guides/docs-codemap.md`](./guides/docs-codemap.md).* +""" + +FRAMING = """*This describes the platform as designed. What is proven today is recorded row by row in the +contracts — [`prd/`](./prd/) for the ratified ones, [`prd-drafts/`](./prd-drafts/) for those in proposal.* +""" +WHAT_THIS_IS = """## What this is + +A sample project for sample customers, sold by the seat. +""" +OVERVIEW = """# Overview — Sample Project + +""" + FRAMING + """ +""" + WHAT_THIS_IS + """ +## The platform + +```mermaid +flowchart LR + a["Core"] --> b["Widget"] +``` + +## How it works + +- **Core** — the substrate everything stands on. [contract](./prd/base-core.md) +- **Widget** — a placed thing, reports its state. [contract](./prd/entity-widget.md) + +## What you use + +- **An operator** — one workspace, where widgets are placed and watched. + +## What governs it + +- **What a widget may report** — the owner sets it. See [`prd/entity-widget.md`](./prd/entity-widget.md). + +--- +*Editing this file? Follow the standard first: [`guides/docs-overview.md`](./guides/docs-overview.md).* +""" + +AGENTS = """# AGENTS + +Always read first: `.knowledge/BRIEF.md`, `.knowledge/CODEMAP.md`, `.knowledge/MEMORY.md`. + +Never modify this file without approval; when approved, follow `.knowledge/guides/docs-agents.md`. +""" + + +def w(aidir, rel, content): + path = os.path.join(aidir, rel) + os.makedirs(os.path.dirname(path), exist_ok=True) + with open(path, "w", encoding="utf-8") as fh: + fh.write(content) + + +def w_repo(aidir, rel, content): + """Write beside .knowledge/, at the repo root — where AGENTS.md lives.""" + w(os.path.dirname(aidir), rel, content) + + +def build_base(aidir): + shutil.copytree(PAYLOAD, aidir) # the real shipped payload — structure exactly as shipped + # This script ships *inside* a project's .knowledge/, so the payload above may already hold that + # project's real PRDs, notes, and reference sets. Clear the content homes: the fixture below is the + # whole of the valid base, and the test stays deterministic no matter what the project has written. + for home in ("prd", "prd-drafts", "research", "references"): + d = os.path.join(aidir, home) + for name in os.listdir(d): + if name == "README.md": + continue + path = os.path.join(d, name) + shutil.rmtree(path) if os.path.isdir(path) else os.remove(path) + for rel, body in STUB_CATALOGS.items(): # ...and reset their catalogs, which linked to those files + w(aidir, rel, body) + mp = os.path.join(aidir, "MAP.md") # generated from content we just replaced — stale by construction + if os.path.exists(mp): + os.remove(mp) + w(aidir, "prd-drafts/entity-future.md", DRAFT) # a populated project, not an empty one + w(aidir, "research/how-others-do-it.md", NOTE) + w(aidir, "references/acme/NOTES.md", "What inspires: the layout. Provenance: public marketing page.\n") + w(aidir, "guides/do-the-thing.md", HOWTO) + w(aidir, "BRIEF.md", BRIEF) # an adopted project has these filled in + w(aidir, "CODEMAP.md", CODEMAP) + w(aidir, "OVERVIEW.md", OVERVIEW) + gr = os.path.join(aidir, "guides/README.md") # ...and list it in the guides catalog + with open(gr, encoding="utf-8") as fh: + text = fh.read() + # append, don't replace a placeholder row — a real project's catalog already lists its own how-tos + w(aidir, "guides/README.md", + text.rstrip() + "\n| [do-the-thing.md](./do-the-thing.md) | do the recurring thing |\n") + w(aidir, "prd/README.md", CATALOG) # add two sample PRDs + a matching catalog + w(aidir, "prd/base-core.md", BASE) + w(aidir, "prd/entity-widget.md", WIDGET) + w_repo(aidir, "AGENTS.md", AGENTS) # the repo root the payload is adopted into + + +def run(aidir, *flags): + p = subprocess.run([sys.executable, DOC_LINT, *flags, aidir], capture_output=True, text=True) + return p.returncode, p.stdout + + +def case(name, mutate, expect_ok, sub=None, flags=()): + with tempfile.TemporaryDirectory() as d: + aidir = os.path.join(d, ".knowledge") + build_base(aidir) + mutate(aidir) + code, out = run(aidir, *flags) + ok = (code == 0) + passed = (ok == expect_ok) and (sub is None or sub in out) + detail = "" if passed else f" (exit={code}, wanted_ok={expect_ok}, sub={sub!r})\n{out}" + print(f"[{'PASS' if passed else 'FAIL'}] {name}{detail}") + return passed + + +def main(): + cases = [ + # --- the shipped structure is intact --- + ("valid full payload passes", lambda a: None, True), + ("a shipped guide was removed", + lambda a: os.remove(os.path.join(a, "guides/docs-brief.md")), + False, "required file `guides/docs-brief.md` is missing"), + ("a home directory was removed", + lambda a: shutil.rmtree(os.path.join(a, "references")), + False, "required directory `references/` is missing"), + ("the linter script was renamed away", + lambda a: os.remove(os.path.join(a, "scripts/doc-lint")), + False, "required file `scripts/doc-lint` is missing"), + # --- payload integrity: `.version` is only a claim until the checksums agree with it --- + ("a versioned guide was edited after adoption", + lambda a: w(a, "guides/docs-prd.md", + open(os.path.join(a, "guides/docs-prd.md"), encoding="utf-8").read() + + "\nOur project decided otherwise about all of the above.\n"), + False, "no longer match the checksums recorded"), + ("the manifest no longer covers a versioned file", + lambda a: w(a, ".payload-manifest", "".join( + line + "\n" + for line in open(os.path.join(a, ".payload-manifest"), encoding="utf-8").read().splitlines() + if not line.endswith("scripts/doc-lint"))), + False, "does not cover `scripts/doc-lint`"), + ("the payload manifest is missing", + lambda a: os.remove(os.path.join(a, ".payload-manifest")), + False, "integrity manifest is missing"), + # --- the stakeholder-facing overview is schema-closed, same as a contract --- + ("an overview heading outside the schema", + lambda a: w(a, "OVERVIEW.md", OVERVIEW + "\n## Why it is different\n\nInvented prose.\n"), + False, "`## Why it is different` is not in the closed schema"), + ("an overview missing a required section", + lambda a: w(a, "OVERVIEW.md", OVERVIEW.replace(WHAT_THIS_IS, "")), + False, "missing required section `## What this is`"), + ("overview sections out of order", + lambda a: w(a, "OVERVIEW.md", OVERVIEW.replace("## What you use", "## PLACE") + .replace("## What governs it", "## What you use").replace("## PLACE", "## What governs it")), + False, "sections are out of order"), + ("an overview without its framing line", + lambda a: w(a, "OVERVIEW.md", OVERVIEW.replace(FRAMING, "")), + False, "framing line under the title is gone"), + ("a trio file lost its standard pointer", + lambda a: w(a, "BRIEF.md", "# Brief\n\nJust a project, no pointer at the bottom.\n"), + False, "must end with its standard pointer"), + # --- PRD contract rules --- + ("two namespaces in one file", + lambda a: w(a, "prd/entity-widget.md", WIDGET.replace("R-WIDGET-1", "R-OTHER-1")), + False, "does not match file namespace"), + ("duplicate ID across files", + lambda a: w(a, "prd/base-core.md", BASE.replace("id: CORE", "id: WIDGET").replace("R-CORE-1", "R-WIDGET-1")), + False, "already defined"), + ("requirement over 25 words", + lambda a: w(a, "prd/base-core.md", BASE.replace( + "persists between runs", + "persists between runs and also across every conceivable restart cycle no matter how many " + "times the machine happens to reboot itself over and over again without any exception")), + False, "max 25"), + ("numeric literal in requirement", + lambda a: w(a, "prd/base-core.md", BASE.replace("between runs", "for 30 runs")), + False, "numeric literal"), + ("two assertions joined by a semicolon", + lambda a: w(a, "prd/base-core.md", BASE.replace("persists between runs", + "persists between runs; it also survives a crash")), + False, "semicolon"), + ("a namespace skips an id", + lambda a: w(a, "prd/base-core.md", + BASE + "| ❌ | R-CORE-3 | The substrate reports its own version | — |\n"), + False, "namespace `CORE` skips R-CORE-2"), + ("checkmark with no test named", + lambda a: w(a, "prd/base-core.md", BASE.replace("| `coreHolds` |", "| — |")), + False, "names no test"), + ("contract cites a draft", + lambda a: (w(a, "prd-drafts/entity-future.md", + "---\nid: FUTURE\nname: Future\n---\n\n## Requirements\n\n" + "| | ID | Requirement | Evidence |\n|:--:|---|---|---|\n" + "| ❌ | R-FUTURE-1 | A future thing exists | — |\n"), + w(a, "prd/entity-widget.md", WIDGET.replace("built on R-CORE-1.", "built on R-CORE-1 and R-FUTURE-1."))), + False, "prd-drafts/ proposal"), + ("citation resolves nowhere", + lambda a: w(a, "prd/entity-widget.md", WIDGET.replace("R-CORE-1.", "R-GHOST-9.")), + False, "resolves nowhere"), + ("catalog omits a PRD file", + lambda a: w(a, "prd/README.md", CATALOG.replace(" - [entity-widget](./entity-widget.md)\n", "")), + False, "catalog omits"), + ("catalog omits a research note", + lambda a: w(a, "research/market.md", + "# Research: market\n\n**Last updated:** 2026-07-16\n\n## What they do\n\nA claim.\n"), + False, "research/README.md: catalog omits market.md"), + ("two files claim one namespace", + lambda a: (w(a, "prd/base-core.md", BASE.replace("id: CORE", "id: WIDGET").replace("R-CORE-1", "R-WIDGET-9")), + w(a, "prd/entity-widget.md", WIDGET.replace("built on R-CORE-1.", "standalone."))), + False, "one file per namespace"), + ("a catalog links to a missing file", + lambda a: w(a, "guides/README.md", + open(os.path.join(a, "guides/README.md"), encoding="utf-8").read() + + "\n[ghost](./docs-ghost.md)\n"), + False, "links to missing"), + ("a README is missing a required section", + lambda a: w(a, "research/README.md", "# research/ — catalog\n\nGutted, no sections.\n"), + False, "missing required section"), + ("last_verified without a checkmark", + lambda a: w(a, "prd/entity-widget.md", WIDGET.replace("name: Widget\n---", "name: Widget\nlast_verified: 2026-07-16\n---")), + False, "no ✅ row"), + ("citation into a later layer", + lambda a: w(a, "prd/base-core.md", BASE.replace("The substrate.", "The substrate. Uses R-WIDGET-1.")), + False, "up the stack"), + ("filename prefix not a component", + lambda a: w(a, "prd/gadget-foo.md", + "---\nid: GADGET\nname: Gadget\n---\n\n## Requirements\n\n" + "| | ID | Requirement | Evidence |\n|:--:|---|---|---|\n" + "| ❌ | R-GADGET-1 | A gadget does a thing | — |\n"), + False, "filename prefix not a listed component"), + ("heading outside the schema", + lambda a: w(a, "prd/entity-widget.md", WIDGET + "\n## Notes\n\nextra prose.\n"), + False, "not in the closed schema"), + ("a PRD hides in a subdirectory", + lambda a: w(a, "prd/legacy/entity-hidden.md", + "---\nid: HIDDEN\nname: Hidden\n---\n\n## Requirements\n\n" + "| | ID | Requirement | Evidence |\n|:--:|---|---|---|\n" + "| ❌ | R-HIDDEN-1 | A hidden thing exists | — |\n"), + False, "is a subdirectory"), + ("a citation cycle inside one layer", + lambda a: (w(a, "prd/entity-widget.md", WIDGET.replace("A widget, built on R-CORE-1.", "A widget. See R-OTHER-1.")), + w(a, "prd/entity-other.md", + "---\nid: OTHER\nname: Other\n---\n\n## What this is\n\nAn other. See R-WIDGET-1.\n\n" + "## Requirements\n\n| | ID | Requirement | Evidence |\n|:--:|---|---|---|\n" + "| ❌ | R-OTHER-1 | An other thing exists | — |\n"), + w(a, "prd/README.md", CATALOG + " - [entity-other](./entity-other.md)\n")), + False, "citation cycle"), + ("research heading outside the schema", + lambda a: (w(a, "research/market.md", + "# Research: market\n\n**Last updated:** 2026-07-16\n\n## What they do\n\nStuff.\n" + "\n## Extra Thoughts\n\nMine.\n"), + w(a, "research/README.md", + open(os.path.join(a, "research/README.md"), encoding="utf-8").read() + "\n[market](./market.md)\n")), + False, "not in the closed schema"), + ("a source with no link", + lambda a: (w(a, "research/market.md", + "# Research: market\n\n**Last updated:** 2026-07-16\n\n## What they do\n\nA claim [1].\n" + "\n## Sources\n\n1. Some vendor's documentation\n"), + w(a, "research/README.md", + open(os.path.join(a, "research/README.md"), encoding="utf-8").read() + "\n[market](./market.md)\n")), + False, "carries no link"), + ("an internal source needs no link", + lambda a: (w(a, "research/market.md", + "# Research: market\n\n**Last updated:** 2026-07-16\n\n## What they do\n\nA claim [1].\n" + "\n## Sources\n\n1. Our own auction walkthrough (internal)\n"), + w(a, "research/README.md", + open(os.path.join(a, "research/README.md"), encoding="utf-8").read() + "\n[market](./market.md)\n")), + True), + ("a source that is never cited", + lambda a: (w(a, "research/market.md", + "# Research: market\n\n**Last updated:** 2026-07-16\n\n## What they do\n\nA claim [1].\n" + "\n## Sources\n\n1. First — https://example.com/a\n2. Unused — https://example.com/b\n"), + w(a, "research/README.md", + open(os.path.join(a, "research/README.md"), encoding="utf-8").read() + "\n[market](./market.md)\n")), + False, "is never cited"), + ("research note missing its date", + lambda a: w(a, "research/competitor.md", + "# Research: competitor\n\n**Question it answers:** what\n\n## What they do\n\nStuff.\n"), + False, "**Last updated:**"), + ("research citation without a source", + lambda a: w(a, "research/market.md", + "# Research: market\n\n**Last updated:** 2026-07-16\n\n## What they do\n\nA claim [1].\n"), + False, "no Sources entry"), + ("a catalog is missing its guide link", + lambda a: w(a, "prd/README.md", + CATALOG.replace("To write or modify one, follow [../guides/docs-prd.md](../guides/docs-prd.md).\n\n", "")), + False, "must link to"), + ("an unexpected entry at the root", + lambda a: w(a, "SCRATCH.md", "not allowed here\n"), + False, "unexpected entry at the .knowledge/ root"), + ("an unfilled orientation doc", + lambda a: w(a, "BRIEF.md", "# Brief — \n\nUnfilled.\n\n---\n" + "*Editing this file? Follow the standard first: [`guides/docs-brief.md`]" + "(./guides/docs-brief.md).*\n"), + False, "shipped placeholder"), + ("reporting with no citation", + lambda a: w(a, "research/how-others-do-it.md", + NOTE.replace("They do a thing [1], and we have seen it ourselves [2].", + "They do a thing, and we have seen it ourselves.") + .replace("1. A public page — https://example.com/a-page\n", "") + .replace("2. Our own walkthrough (internal)\n", "")), + False, "cites no source"), + # --- the entry point that routes agents into .knowledge/ at all --- + ("the repo has no AGENTS.md", + lambda a: os.remove(os.path.join(os.path.dirname(a), "AGENTS.md")), + False, "no AGENTS.md"), + ("AGENTS.md no longer points at its own standard", + lambda a: w_repo(a, "AGENTS.md", + "# AGENTS\n\nRead `.knowledge/BRIEF.md`, `.knowledge/CODEMAP.md`, " + "`.knowledge/MEMORY.md`.\n"), + False, "must point at"), + ("AGENTS.md no longer loads the trio", + lambda a: w_repo(a, "AGENTS.md", "# AGENTS\n\nBuild well. Nothing about the knowledge base.\n"), + False, "must load the orientation trio"), + ] + results = [case(*c) for c in cases] + print(f"\n{sum(results)}/{len(results)} checks passed") + return 0 if all(results) else 1 + + +if __name__ == "__main__": + sys.exit(main()) diff --git a/.knowledge/tmp/.gitignore b/.knowledge/tmp/.gitignore new file mode 100644 index 00000000..86b87379 --- /dev/null +++ b/.knowledge/tmp/.gitignore @@ -0,0 +1,7 @@ +# .knowledge/tmp/ — local scratch space for AI-generated temporary docs, assets, and experiment output. +# The folder is kept in git (via .gitkeep); everything you put inside it is ignored and stays local. +# Use it for throwaway work; never for durable knowledge (that goes in a home like prd/ or MEMORY.md). +* +!.gitkeep +!.gitignore +!README.md diff --git a/.knowledge/tmp/README.md b/.knowledge/tmp/README.md new file mode 100644 index 00000000..56a47e61 --- /dev/null +++ b/.knowledge/tmp/README.md @@ -0,0 +1,12 @@ +# tmp/ — local scratch + +Throwaway space for AI-generated temporary files: draft output, generated assets, experiment results, +anything you need while working but nobody should keep. **Everything here is git-ignored** (except this +README and the folder markers) — it stays on your machine and never lands in a commit. + +## Rules + +- **Never durable knowledge.** A fact worth keeping goes to its home (`prd/`, `research/`, `MEMORY.md`, …), + never here. If something in `tmp/` matters tomorrow, it's in the wrong place. +- **Assume it vanishes.** Don't reference a `tmp/` file from any committed doc — the link will rot. +- **Yours only.** Nothing here is shared; it's not part of the copied payload's meaning, just a workbench. diff --git a/AGENTS.md b/AGENTS.md index c571d7f0..41f8f562 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -1,64 +1,74 @@ # AGENTS -Rules for every AI coding agent working in this repository. These rules are law; where they conflict with your general habits, this file wins. - -This is an **open-source Laravel package**. Everything here ships into strangers' applications: the public API is a contract, every dependency is an imposition on every consumer, and the documentation is the product's front door. - ---- - -## Before You Work - -1. Read [`BRIEF.md`](./.ai/BRIEF.md) (scope), [`CODEMAP.md`](./.ai/CODEMAP.md) (structure), and [`MEMORY.md`](./.ai/MEMORY.md) (lessons) before your first edit. -2. Read the relevant VitePress documentation before working on any module. **Documentation is the source of truth — it overrides all assumptions.** -3. Read every file before editing it. -4. Search the codebase before writing new logic. If it exists — reuse, extend, or refactor. Never duplicate. -5. When the user raises a concern, investigate before contradicting. Contradict only with evidence from the codebase. - -## Hard Gates — Require Explicit User Approval - -- **Public API changes.** Renaming or removing a public method, changing a signature, changing a config key, or changing documented behavior is a breaking change for every consumer. Never do it without approval. -- **Dependencies.** Never add, remove, or major-version-bump a Composer dependency without approval — a package dependency is imposed on every consuming application. -- **Migrations.** Any change to shipped migrations or persisted schema must be confirmed before proceeding — consumers have already run the old ones. -- **Deletions.** Do not delete files or directories outside the immediate scope of the task without approval. -- **This file.** Never modify `AGENTS.md` without approval. If a rule seems wrong or missing, raise it. +Rules for every agent working in this repository. These rules are law; where they conflict with your general +habits, this file wins. + +This is an **open-source Laravel package — a unified AI SDK (Atlas v3) that owns its own provider layer**. +Everything here ships into strangers' applications: the public API is a contract, every dependency is an +imposition on every consumer, and the documentation is the product's front door. The *what & why* lives in +`.knowledge/BRIEF.md`; the knowledge map is `.knowledge/README.md`. This file defines how you build here. + +## Before you work + +Load light; pull depth only when the task needs it. + +1. **Always read first:** `.knowledge/BRIEF.md` (what & why), `.knowledge/CODEMAP.md` (where things are), + `.knowledge/MEMORY.md` (current friction). `.knowledge/README.md` maps the rest. +2. **On demand, when the task enters an area:** `.knowledge/prd/` (ratified contracts — source of truth), + `.knowledge/prd-drafts/` (proposals), `.knowledge/research/` + `.knowledge/references/` (prior art, visual + targets), `.knowledge/guides/` (how to write each doc + project how-tos). +3. **How work flows:** `research/` -> `prd-drafts/` -> `prd/`; a `prd/` contract never cites a draft. New + guaranteed behavior is a `prd/` row backed by a test — cite its `R--` in the code. Follow a doc's + guide before writing or modifying it, and keep docs true in the same task. Run + `python3 .knowledge/scripts/doc-lint .knowledge` before finishing; scratch -> `.knowledge/tmp/`. +4. Read every file before editing it; search before writing new logic — reuse, extend, refactor. +5. When the user raises a concern, investigate before contradicting — evidence, not a hunch. +6. Read the relevant VitePress documentation before working on any module. **Consumer docs are the source of + truth — they override all assumptions.** + +## Hard gates — require explicit approval + +- **Persisted state.** Any change to schema, stored data, or migrations is confirmed first. +- **Dependencies.** Do not add, remove, or major-version-bump a package without approval. +- **Deletions.** Do not delete files outside the task's immediate scope without approval. +- **Commits.** Do not commit or push unless told to. +- **This file.** Never modify `AGENTS.md` without approval; when approved, follow + `.knowledge/guides/docs-agents.md`. If a rule seems wrong or missing, raise it. + +**This is a public open-source package — the gates above carry extra weight, plus:** + +- **Public API changes.** Renaming or removing a public method, changing a signature, changing a config key, or + changing documented behavior is a breaking change for every consumer. Never do it without approval. +- **Dependencies are imposed on every consumer.** A Composer dependency ships into every consuming application — + the approval gate above is absolute here, major-version bumps included. +- **Shipped migrations.** Consumers have already run the old ones — any change to a shipped migration or + persisted schema is confirmed before proceeding. ## Never -- Never commit credentials, tokens, or real API keys — not in code, tests, fixtures, docs examples, or sandbox files. This repository is public. +- Never touch secrets or commit credentials. +- Never leave debug output or commented-out code in completed work. +- Never commit credentials, tokens, or real API keys — not in code, tests, fixtures, docs examples, or sandbox + files. This repository is public. - Never leave `dd()`, `dump()`, `ray()`, `var_dump()`, or commented-out code in completed work. -- Never depend on a consuming application. All code is **stateless, framework-aware, and application-agnostic** — the package must be fully self-contained. +- Never depend on a consuming application. All code is **stateless, framework-aware, and application-agnostic** — + the package must be fully self-contained. - Never use `use function` imports. -- Never call a real provider API from an automated test. Unit and Feature tests use the fakes in `src/Testing/`; real-API validation happens only in the sandbox. - -## Documentation Duties - -This repository has two documentation surfaces with different audiences. Both are your responsibility, not the user's. - -**Runtime docs (`.ai/`, agent-facing):** - -- Restructured directories or moved files → update `CODEMAP.md` in the same task. -- Learned something that would have saved you time (a trap, a non-obvious constraint, a tooling quirk) → append it to `MEMORY.md`. -- Do not add rationale, maintainer commentary, or history to these files. They address the next agent doing work, nothing else. - -**Consumer docs (`docs/`, VitePress, published at atlasphp.org):** - -| Code change | Documentation update | -|---|---| -| Adding a feature | Update the relevant docs in the same task | -| Changing behavior | Update docs immediately | -| Adding a module | Add docs to the appropriate section | -| Fixing a bug | None, unless behavior was misdocumented | -| Deprecating | Mark deprecated, add migration notes | -| Removing | Remove from docs completely — no "removed" comments | +- Never call a real provider API from an automated test. Unit and Feature tests use the fakes in `src/Testing/`; + real-API validation happens only in the sandbox. -- Every code example in the docs must be syntactically correct and runnable. -- Cross-references use relative links. No duplicate content across files — for Prism-level features, link to Prism docs instead of restating them. +## Tech stack ---- +- **Package:** Atlas v3 — a unified AI SDK for Laravel. It owns its own provider layer; there is **no external AI + package dependency**. Modern PHP 8.2+, PSR-12 formatted by Pint, statically analyzed by PHPStan. +- **Testing:** Pest — Feature tests for workflows, Unit tests for services — built on the fakes in `src/Testing/`. +- **Sandbox:** a full Laravel app (`sandbox/`) with real persistence and Horizon, for validation against + real provider APIs. +- **Consumer docs:** VitePress, published at atlasphp.org. ## Architecture -Atlas v3 is a unified AI SDK for Laravel. It owns its own provider layer — no external AI package dependency. +Atlas owns its own provider layer — no external AI package dependency. **Runtime flow** (a request travels down, never sideways or up): @@ -79,11 +89,16 @@ HttpClient (sends HTTP, fires transport events) - **Drivers are stateless** — one request → one response; loops belong to the executor. - **One shared HttpClient** — all providers use the same transport with consistent event dispatching. -## Layer Boundaries +### Layer boundaries -Dependencies flow **downward only**. A lower layer never imports from a higher one, and no two services depend on each other circularly. +Dependencies flow **downward only**. A lower layer never imports from a higher one, and no two services depend +on each other circularly. -**Model services** (`Persistence/Services/`, named `{Model}ModelService`) are the single point of truth for persistence: create/update/delete, model-specific query helpers, pre-persistence normalization. They never orchestrate workflows, call other services, call providers, or dispatch events. All Eloquent access in the package goes through them — no direct queries elsewhere, no business logic in models beyond accessors, mutators, and scopes. +**Model services** (`Persistence/Services/`, named `{Model}ModelService`) are the single point of truth for +persistence: create/update/delete, model-specific query helpers, pre-persistence normalization. They never +orchestrate workflows, call other services, call providers, or dispatch events. All Eloquent access in the +package goes through them — no direct queries elsewhere, no business logic in models beyond accessors, mutators, +and scopes. ```php // ❌ The most common violation — orchestration smuggled into the model layer @@ -99,7 +114,9 @@ class AgentModelService } ``` -**Domain services** (named by intent: `CreateAgentService`, `ProcessToolCallService`) implement business logic: orchestrating model services, managing transactions, dispatching events and jobs, calling providers through contracts. They contain no direct Eloquent queries and no provider implementation details. +**Domain services** (named by intent: `CreateAgentService`, `ProcessToolCallService`) implement business logic: +orchestrating model services, managing transactions, dispatching events and jobs, calling providers through +contracts. They contain no direct Eloquent queries and no provider implementation details. ```php // ✅ Orchestration lives here — persistence delegated, dependencies injected @@ -122,13 +139,19 @@ class CreateAgentService } ``` -**Provider clients** (`Providers/{Provider}/`) are low-level external API clients: HTTP calls, authentication, request/response transformation, retries. They return DTOs or primitives and contain no business decisions, no database access, and no dependencies on domain services. `OpenAiClient::complete()` returning a `CompletionResponse` is a provider client; a client that also writes the response to a conversation table is not. +**Provider clients** (`Providers/{Provider}/`) are low-level external API clients: HTTP calls, authentication, +request/response transformation, retries. They return DTOs or primitives and contain no business decisions, no +database access, and no dependencies on domain services. `OpenAiClient::complete()` returning a +`CompletionResponse` is a provider client; a client that also writes the response to a conversation table is not. -**Support** (`Support/`) holds pure utilities only: no database, no HTTP, no service dependencies, no state, no side effects. `TokenCounter::count()` computing from its input is Support; a `TokenCounter` that caches results through a `CacheContract` is not — caching is a side effect and belongs a layer up. +**Support** (`Support/`) holds pure utilities only: no database, no HTTP, no service dependencies, no state, no +side effects. `TokenCounter::count()` computing from its input is Support; a `TokenCounter` that caches results +through a `CacheContract` is not — caching is a side effect and belongs a layer up. -## Contracts & Dependency Injection +### Contracts & dependency injection -Inject dependencies through the constructor and let Laravel's container resolve them. Type-hint the interface when one exists. +Inject dependencies through the constructor and let Laravel's container resolve them. Type-hint the interface +when one exists. ```php // ❌ All three forbidden acquisition patterns @@ -145,11 +168,13 @@ class ProcessAgentResponseService (`app()` is permitted inside service providers only.) -**Earn your abstractions.** Every layer of indirection needs a concrete justification. Create a contract only when multiple implementations exist or are planned, a test needs a substitution seam, or the dependency crosses a package boundary — a single-implementation class that no test mocks gets no interface. The same discipline forbids: speculative generalization for requirements that don't exist, proxy services that pass through to another service, wrapper classes that add no behavior, and DTOs that mirror an Eloquent model 1:1. - ---- +**Earn your abstractions.** Every layer of indirection needs a concrete justification. Create a contract only +when multiple implementations exist or are planned, a test needs a substitution seam, or the dependency crosses +a package boundary — a single-implementation class that no test mocks gets no interface. The same discipline +forbids: speculative generalization for requirements that don't exist, proxy services that pass through to +another service, wrapper classes that add no behavior, and DTOs that mirror an Eloquent model 1:1. -## Naming Conventions +## Naming conventions | Type | Pattern | Example | |---|---|---| @@ -169,11 +194,46 @@ class ProcessAgentResponseService | Traits (resolver) | `Resolves*` | `ResolvesProvider` | | Traits (action) | `{Verb}s*` | `TracksExecution`, `StoresMedia` | -Handler interfaces (`Providers/Handlers/`) use `*Handler` — they define modality capabilities (what a provider can do). Resolver contracts (`Providers/Contracts/`) use `*Contract` — they define composition seams (how provider internals plug together). Both are PHP interfaces; the naming reflects the architectural role. +Handler interfaces (`Providers/Handlers/`) use `*Handler` — they define modality capabilities (what a provider +can do). Resolver contracts (`Providers/Contracts/`) use `*Contract` — they define composition seams (how +provider internals plug together). Both are PHP interfaces; the naming reflects the architectural role. + +Methods are short, descriptive, and predictable: booleans prefixed `is`/`has`/`can`, actions named with verbs +(`create`, `execute`, `process`), and every name must match documented terminology. -Methods are short, descriptive, and predictable: booleans prefixed `is`/`has`/`can`, actions named with verbs (`create`, `execute`, `process`), and every name must match documented terminology. +## Best practices — do / don't -## Package Structure +- **Do** `declare(strict_types=1)` in every PHP file, PSR-12 formatted by Pint, modern PHP 8.2+ syntax. +- **Do** give every class a PHPDoc block summarizing its purpose: + +```php +✅ /** + * Class UserWebhookService + * + * Handles webhook registration, processing, and retry logic for user-related events. + */ + class UserWebhookService { /* ... */ } +❌ class UserWebhookService { /* ... */ } // no doc block — intent left to the reader to reverse-engineer +``` + +- **Do** raise custom exceptions for expected failures; keep config files in `config/` with sensible defaults. +- **Do** cover new features with full Pest coverage — Feature tests for workflows, Unit tests for services, + built on `src/Testing/` fakes, **never real APIs**. + +### Quality thresholds + +- Methods stay under **20–30 lines**; nesting stays under **3 levels** — use early returns and extraction. +- A class with **10+ public methods** splits by responsibility. A class whose tests mock **5+ dependencies** is + doing too much. +- No hidden dependencies — if a method needs something, the constructor receives it. No global state or + singletons. No complex logic buried in untestable private methods — extract a class. +- Eager-load relationships accessed in loops; never query inside a loop — batch or pre-fetch. Chunk large + dataset operations. Cache only what is *measured* slow, never preemptively. +- The same validation or transformation in **3+ places** gets consolidated. Duplication is acceptable when + isolation or clarity outweighs DRY — but intentional duplication carries a brief comment saying why. + "Almost identical" code is a bug signal — inspect the difference. + +## Directory structure ``` package-root/ @@ -186,7 +246,6 @@ package-root/ │ ├── Events/ │ ├── Exceptions/ │ ├── Executor/ -│ ├── Facades/ │ ├── Input/ │ ├── Messages/ │ ├── Middleware/ @@ -206,7 +265,9 @@ package-root/ └── sandbox/ ``` -Each top-level `src/` directory is a **domain concern, not a generic pattern**; namespacing follows structure (`Atlasphp\Atlas\Messages\UserMessage`); cross-domain imports are allowed; and no subdirectory is created until there are enough files to justify it. +Each top-level `src/` directory is a **domain concern, not a generic pattern**; namespacing follows structure +(`Atlasphp\Atlas\Messages\UserMessage`); cross-domain imports are allowed; and no subdirectory is created until +there are enough files to justify it. Where a new file goes: @@ -229,41 +290,30 @@ Where a new file goes: | Fluent builder | `src/Pending/` | | Test fake | `src/Testing/` | -Contracts and concerns live with their domain — never in a shared dumping ground — except the genuinely cross-cutting ones in top-level `src/Concerns/`. - ---- - -## Code Rules +Contracts and concerns live with their domain — never in a shared dumping ground — except the genuinely +cross-cutting ones in top-level `src/Concerns/`. -- `declare(strict_types=1)` in every PHP file. PSR-12, formatted by Pint. Modern PHP 8.2+ syntax. -- Every class carries a PHPDoc block summarizing its purpose: +## Build, test & run -```php -/** - * Class UserWebhookService - * - * Handles webhook registration, processing, and retry logic for user-related events. - */ +```bash +composer check # the gate: Pint, PHPStan, Pest, doc-lint — must pass before a task is done +composer lint # Pint (format) +composer lint:test # Pint --test (format check only) +composer analyse # PHPStan +composer test # Pest +composer lint:docs # python3 .knowledge/scripts/doc-lint .knowledge ``` -- Custom exceptions for expected failures. Config files live in `config/` with sensible defaults. -- New features require full Pest coverage: Feature tests for workflows, Unit tests for services — built on `src/Testing/` fakes, never real APIs. - -### Quality thresholds - -- Methods stay under **20–30 lines**; nesting stays under **3 levels** — use early returns and extraction. -- A class with **10+ public methods** splits by responsibility. A class whose tests mock **5+ dependencies** is doing too much. -- No hidden dependencies — if a method needs something, the constructor receives it. No global state or singletons. No complex logic buried in untestable private methods — extract a class. -- Eager-load relationships accessed in loops; never query inside a loop — batch or pre-fetch. Chunk large dataset operations. Cache only what is *measured* slow, never preemptively. -- The same validation or transformation in **3+ places** gets consolidated. Duplication is acceptable when isolation or clarity outweighs DRY — but intentional duplication carries a brief comment saying why. "Almost identical" code is a bug signal — inspect the difference. +Use `composer lint` / `lint:test` / `analyse` / `test` for faster iteration, but the full `composer check` +must pass before done. Documentation-only changes are exempt from the code gate but must still pass `lint:docs`. ---- +## Sandbox testing -## Sandbox Testing +The sandbox (`sandbox/`) validates package features against real provider APIs and real persistence — see +`sandbox/README.md`. Use it for provider integration, real database behavior, and end-to-end validation. -The sandbox (`sandbox/`) validates package features against real provider APIs and real persistence — see `sandbox/README.md`. Use it for provider integration, real database behavior, and end-to-end validation. - -**Horizon must be running** for queued features to process, and **must be restarted after code changes** to pick up new code: +**Horizon must be running** for queued features to process, and **must be restarted after code changes** to pick +up new code: ```bash cd sandbox @@ -272,25 +322,69 @@ php artisan horizon # blocks the terminal; append & to background it If sandbox tests hang or return empty responses, check Horizon first. ---- - ## Changelog -`CHANGELOG.md` is consumer-facing: write for someone deciding whether to upgrade, never for someone reading the diff. +`CHANGELOG.md` is consumer-facing: write for someone deciding whether to upgrade, never for someone reading the +diff. - **One line per change**, leading with the user-visible effect, never the internal mechanism. -- **Section order:** `### Added` → `### Changed` → `### Fixed` → `### Migration`. Omit empty sections — except `### Migration`, which is always present and always last. Drop-in releases state: `No breaking changes — drop-in upgrade. No consumer action required.` Breaking releases name what changed and the smallest steps a consumer must take. -- **Consumer-facing only.** No housekeeping, refactors, test cleanup, or dependency bumps. No class names, file paths, or stack traces in `### Fixed`. Mention a config key or signature only when the consumer needs it. -- **Header:** `## [vX.Y.Z](https://github.com/atlas-php/atlas/releases/tag/vX.Y.Z) - YYYY-MM-DD`, with a trailing `---` between releases. +- **Section order:** `### Added` → `### Changed` → `### Fixed` → `### Migration`. Omit empty sections — except + `### Migration`, which is always present and always last. Drop-in releases state: `No breaking changes — + drop-in upgrade. No consumer action required.` Breaking releases name what changed and the smallest steps a + consumer must take. +- **Consumer-facing only.** No housekeeping, refactors, test cleanup, or dependency bumps. No class names, file + paths, or stack traces in `### Fixed`. Mention a config key or signature only when the consumer needs it. +- **Header:** `## [vX.Y.Z](https://github.com/atlas-php/atlas/releases/tag/vX.Y.Z) - YYYY-MM-DD`, with a + trailing `---` between releases. + +## Documentation duties + +Keep docs true in the same task that changes reality. Before creating or editing a doc, read its home +`README.md` and follow its `guides/docs-*.md`. + +- Moved/restructured files -> update `.knowledge/CODEMAP.md`. +- Hit friction — **anything that cost you a failed attempt**: an env var or flag you had to discover, + a guard you had to satisfy, a command that only worked the second way you tried it, an error whose + message didn't say what to do -> **write the line into `.knowledge/MEMORY.md` the moment you find + the workaround, before you carry on** — by the end of the task it will feel too small to mention, + which is exactly how the next agent loses the same hour. Delete it once solved. +- Owner ratifies a draft (**the whole file**, not one row) -> `git mv` it into `prd/`; IDs carry over and + the conformance review then sets glyphs. **Approval moves a draft, not proof** — proof is the glyph + column. Behavior and its requirement row change in the same commit. +- Scratch -> `.knowledge/tmp/` (git-ignored). +- Keep the runtime `.knowledge/` orientation docs terse and agent-facing — no rationale, maintainer commentary, + or change history in `BRIEF.md` / `CODEMAP.md` / `MEMORY.md`; they address the next agent doing work, nothing + else. + +**Consumer docs (`docs/`, VitePress, published at atlasphp.org):** a second surface with a different audience, +equally your responsibility, not the user's. + +| Code change | Documentation update | +|---|---| +| Adding a feature | Update the relevant docs in the same task | +| Changing behavior | Update docs immediately | +| Adding a module | Add docs to the appropriate section | +| Fixing a bug | None, unless behavior was misdocumented | +| Deprecating | Mark deprecated, add migration notes | +| Removing | Remove from docs completely — no "removed" comments | ---- +- Every code example in the docs must be syntactically correct and runnable. +- Cross-references use relative links. No duplicate content across files — for Prism-level features, link to + Prism docs instead of restating them. -## Definition of Done +## Definition of done A task is done when the change is verified against its stated requirement — never based on effort — and: -1. `composer check` passes: Pint, PHPStan, Pest. Use `composer lint` / `lint:test` / `analyse` / `test` for faster iteration — but the full `composer check` must pass before done. Documentation-only changes are exempt. -2. Documentation reflects the change, per Documentation Duties. -3. Every rule in this file was upheld. This file is the checklist — re-read it, do not restate it. +1. `composer check` passes (Pint, PHPStan, Pest, doc-lint). Use `composer lint` / `lint:test` / `analyse` / + `test` for faster iteration, but the full `composer check` must pass. Documentation-only changes are exempt + from the code gate but must still pass `lint:docs`. +2. Documentation reflects the change, per Documentation duties (both the runtime `.knowledge/` docs and the + consumer `docs/`). +3. Every rule here held. This file is the checklist — re-read it, do not restate it. +4. New guaranteed behavior is proven by a `.knowledge/prd/` requirement and its test. +5. **Friction you hit is in `.knowledge/MEMORY.md`, not only in your reply** — the next agent reads the file, + not this conversation. Hit none? Say that in your reply. **Never write "no friction" into the file** — + `MEMORY.md` records traps, never their absence. **When creating task lists or plans, the final step is always:** _"Re-read `AGENTS.md` and verify Definition of Done."_ diff --git a/composer.json b/composer.json index e70194d7..7d9eb6c9 100644 --- a/composer.json +++ b/composer.json @@ -44,10 +44,13 @@ "lint:test": "pint --test", "analyse": "phpstan analyse --memory-limit=512M", "test": "@php -d memory_limit=512M ./vendor/bin/pest", + "lint:docs": "python3 .knowledge/scripts/doc-lint .knowledge", + "test:docs": "python3 .knowledge/scripts/test_doc_lint.py", "check": [ "@lint:test", "@analyse", - "@test" + "@test", + "@lint:docs" ] }, "extra": {