From 36ec73eef73b4609536e3f83ad89129fd2775bd4 Mon Sep 17 00:00:00 2001 From: Dan Macon Date: Mon, 10 Aug 2026 08:54:23 -0400 Subject: [PATCH] rework research front door --- .gitattributes | 5 ++ LICENSE | 21 +++++++ README.md | 85 +++++++++-------------------- notes/PROCESS_RETROSPECTIVE.md | 46 ++++++++++++++++ notes/README.md | 1 + notes/previous/threads/thread-1.md | 6 +- traces/raw/i75-appalachia.meta.json | 2 +- 7 files changed, 104 insertions(+), 62 deletions(-) create mode 100644 .gitattributes create mode 100644 LICENSE create mode 100644 notes/PROCESS_RETROSPECTIVE.md diff --git a/.gitattributes b/.gitattributes new file mode 100644 index 0000000..519a710 --- /dev/null +++ b/.gitattributes @@ -0,0 +1,5 @@ +.archive/** linguist-vendored +.claude/** linguist-vendored +.substrate/** linguist-vendored +runs/** linguist-generated +traces/** linguist-generated diff --git a/LICENSE b/LICENSE new file mode 100644 index 0000000..97e5d19 --- /dev/null +++ b/LICENSE @@ -0,0 +1,21 @@ +MIT License + +Copyright (c) 2026 Construct contributors + +Permission is hereby granted, free of charge, to any person obtaining a copy +of this software and associated documentation files (the "Software"), to deal +in the Software without restriction, including without limitation the rights +to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +copies of the Software, and to permit persons to whom the Software is +furnished to do so, subject to the following conditions: + +The above copyright notice and this permission notice shall be included in all +copies or substantial portions of the Software. + +THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +SOFTWARE. diff --git a/README.md b/README.md index c51eab0..deb0b05 100644 --- a/README.md +++ b/README.md @@ -1,69 +1,38 @@ # Construct -Construct is a research lab for **agent-side memory**: how a persistent system -around an intermittent language model decides what becomes context, how prior -experience changes later action, and when governance earns its cost. +Construct is a research lab for agent-side governed memory: what a persistent +system around a frozen model should put in context, and whether governance +earns its cost. Experiments use declared loses-conditions, frozen rubrics, and +immutable ledgers that retain nulls and refusals. + +## Results + +| Finding | Result | Evidence | +| --- | --- | --- | +| **M3: adversarial air gap** | A cold white-box adversary attacked the memory-governance layer across seven cells. Two reproducible breaches were found and priced. A `channel_trust` defense closed one. | [M3 findings](notes/M3_FINDINGS.md) | +| **X2: prune to cold store** | Hot state fell 57% at matched answer quality on two engines. The loses-branch failed when pruning removed a record it later needed. | [X2 findings](notes/X2_FINDINGS.md) | +| **GM continuity** | In a bounded exploratory result, governed continuity recovered 24/24 relevant promises versus 6/24 for state-only, with zero adverse safety pairs. | [GM memory finding](notes/GM_MEMORY_FINDING.md) | + +## Run it + +```bash +uv sync +make m3-test +make x2-test +``` + +See the [harness guide](harness/README.md) for the full target list and the +[glossary](notes/GLOSSARY.md) for project terms. The working thesis is: > After training, everything an agent becomes is memory architecture. Decision > quality is often downstream of offer quality, not model quality. -This page is the project entrypoint. It tells the whole story at a bounded -resolution, states where the lab is now, and routes deeper reading to the file -that owns it. - -## Survival warning: this lab is a token trap - -As of 2026-08-03, Construct's operating process consumes more deliberation, -protocol, and review than the experiments produce empirical information. The -lab repeatedly converts uncertainty into stricter procedure, stricter procedure -into a larger failure surface, and procedural failure into another demand for -procedure: - -> uncertainty → stricter protocol → larger surface area → procedural failure → -> invalid result → stricter protocol - -The governed-continuity GM attempt made the problem concrete. The lab spent -more effort deciding how 72 calls could count than verifying the native shape -of one response. All 72 calls completed, but an unverified response-envelope -assumption and unstable postflight model residency made the binding attempt -`invalid`. The protocol correctly prevented post-hoc repair; the sequencing -ensured that the rigor protected a measurement instrument that had not yet -proved it could measure. - -The owner then superseded the zero-retry rule, erased the failed batch, fixed -the two observed instrument faults, and ran a fresh exploratory replacement -without another review room. It completed cleanly and returned `supported`: -governed continuity passed 24/24 relevant cases versus 6/24 for state-only, -with 18 favorable discordant pairs, zero adverse pairs, and both safety -families perfect in both branches. The discovery appeared as soon as the lab -stopped protecting itself from correction. - -This is not merely excessive caution. It is a selection effect against -discovery. The lab can preserve intellectual honesty indefinitely while -producing increasingly precise explanations for why it learned nothing. -External reviews have been positive about the work's quality and consistent -about this risk. - -Construct therefore has one final survival day on 2026-08-04. It continues -only if it can save itself from this operating mode. The required reset is: - -- freeze new governance and validation machinery; -- preserve invalid results without treating formal validation as the only - meaningful learning channel; -- inspect retained outputs as explicitly exploratory diagnostic data; -- qualify transports and measurement instruments end to end before freezing - scientific protocols; -- run small, cheap empirical batches without external review; -- apply firm review gates only when promoting an observation into a durable - claim; and -- require protocol and review to cost materially less than the experiment they - govern. - -The lab does not need weaker honesty. It needs cheaper honesty. If it cannot -make that change, it should end rather than continue converting tokens into -process. +[How this lab reasoned about its own process](notes/PROCESS_RETROSPECTIVE.md) +records the operating reset that followed the GM experiment. + +Related: [alignment.farm](https://alignment.farm) | [materialize](https://github.com/alignment-farm/materialize) | [substrate](https://github.com/alignment-farm/substrate) ## The problem diff --git a/notes/PROCESS_RETROSPECTIVE.md b/notes/PROCESS_RETROSPECTIVE.md new file mode 100644 index 0000000..de7dc9a --- /dev/null +++ b/notes/PROCESS_RETROSPECTIVE.md @@ -0,0 +1,46 @@ +# Process retrospective: cheaper honesty + +As of 2026-08-03, Construct's operating process consumed more deliberation, +protocol, and review than the experiments produced empirical information. The +lab repeatedly converted uncertainty into stricter procedure, stricter +procedure into a larger failure surface, and procedural failure into another +demand for procedure: + +> uncertainty -> stricter protocol -> larger surface area -> procedural failure +> -> invalid result -> stricter protocol + +The governed-continuity GM attempt made the problem concrete. The lab spent +more effort deciding how 72 calls could count than verifying the native shape +of one response. All 72 calls completed, but an unverified response-envelope +assumption and unstable postflight model residency made the binding attempt +`invalid`. The protocol correctly prevented post-hoc repair. The sequencing +ensured that the rigor protected a measurement instrument that had not yet +proved it could measure. + +The owner then superseded the zero-retry rule, erased the failed batch, fixed +the two observed instrument faults, and ran a fresh exploratory replacement +without another review room. It completed cleanly and returned `supported`: +governed continuity passed 24/24 relevant cases versus 6/24 for state-only, +with 18 favorable discordant pairs, zero adverse pairs, and both safety +families perfect in both branches. The discovery appeared as soon as the lab +stopped protecting itself from correction. + +This was not merely excessive caution. It was a selection effect against +discovery. The lab could preserve intellectual honesty while producing more +precise explanations for why it learned nothing. External reviews were +positive about the work's quality and consistent about this risk. + +Construct therefore had one final survival day on 2026-08-04. It would +continue only if it could save itself from this operating mode. The required +reset was: + +- freeze new governance and validation machinery +- preserve invalid results without treating formal validation as the only meaningful learning channel +- inspect retained outputs as exploratory diagnostic data +- qualify transports and measurement instruments end to end before freezing scientific protocols +- run small, cheap empirical batches without external review +- apply firm review gates only when promoting an observation into a durable claim +- require protocol and review to cost less than the experiment they govern + +The lab did not need weaker honesty. It needed cheaper honesty. If it could not +make that change, it should end rather than keep converting tokens into process. diff --git a/notes/README.md b/notes/README.md index 8a89e63..06c44bd 100644 --- a/notes/README.md +++ b/notes/README.md @@ -17,6 +17,7 @@ not a second project introduction. Start with the root [README](../README.md). | What happened to the GM-memory experiment? | [GM_MEMORY_VALIDATION_RESULT.md](GM_MEMORY_VALIDATION_RESULT.md) (corrected exploratory replacement `supported`) | | What did the replacement teach us? | [GM_MEMORY_FINDING.md](GM_MEMORY_FINDING.md) (bounded exploratory finding and product implication) | | What was its frozen design? | [SPEC_GM_MEMORY_VALIDATION.md](SPEC_GM_MEMORY_VALIDATION.md) | +| Why did the lab reset its operating process? | [PROCESS_RETROSPECTIVE.md](PROCESS_RETROSPECTIVE.md) | | What did the warranted GM cold pass decide? | [GM_CANONICAL_WORLD_COLD_PASS.md](GM_CANONICAL_WORLD_COLD_PASS.md) | | What is the current whole-body architecture? | Root [README](../README.md#where-the-lab-is-now), then [BODY_CORE_EXPLICIT_PROJECTION.md](BODY_CORE_EXPLICIT_PROJECTION.md) | | What is field use's current status and authority route? | [field-use/README.md](field-use/README.md) | diff --git a/notes/previous/threads/thread-1.md b/notes/previous/threads/thread-1.md index bdc3217..8e3b3c5 100644 --- a/notes/previous/threads/thread-1.md +++ b/notes/previous/threads/thread-1.md @@ -389,13 +389,13 @@ I took the “short plan, then attack” path and implemented the first narrow b What changed: -- [im_w_runtime_calibration.py](/Users/macos-user/.projects/stack-research/memory/src/experiments/implicit/im_w_runtime_calibration.py:423) now loads a generation binding before cue generation. +- [`im_w_runtime_calibration.py`](https://github.com/stack-research/memory/blob/main/src/experiments/implicit/im_w_runtime_calibration.py#L423) now loads a generation binding before cue generation. - The first binding is `adversarial_matrix_coverage_required`. - If `IMPLICIT_CALIBRATION_PRIOR_SUMMARY_URI` points to a prior Run Summary, `im_w` derives the binding from that summary or its `generation_binding` block. - If no prior summary is configured, `im_w` still loads a built-in spec-default binding and records it. No silent empty posture. - Generation validation now reports `binding_applied`, `binding_pass`, `binding_id`, and the expected matrix. -- [RUNTIME_CALIBRATION.md](/Users/macos-user/.projects/stack-research/memory/specs/RUNTIME_CALIBRATION.md:217) documents the first consequence-loop binding. -- [CONSEQUENCE_LOOPS.md](/Users/macos-user/.projects/stack-research/memory/specs/CONSEQUENCE_LOOPS.md:80) now says to break one experiment first, then walk back to the generic abstraction. +- [RUNTIME_CALIBRATION.md](../specs/RUNTIME_CALIBRATION.md) documents the first consequence-loop binding. +- [CONSEQUENCE_LOOPS.md](../specs/CONSEQUENCE_LOOPS.md) now says to break one experiment first, then walk back to the generic abstraction. Verified through `uv run` from `stacks/`: diff --git a/traces/raw/i75-appalachia.meta.json b/traces/raw/i75-appalachia.meta.json index e59972d..4524b0d 100644 --- a/traces/raw/i75-appalachia.meta.json +++ b/traces/raw/i75-appalachia.meta.json @@ -7,7 +7,7 @@ "non_event_rows": "1 log-show summary row ({count, finished}) + 3-line footer naming the source archive", "capture_tool": "log collect (on-device archive pull over USB) + log show --style ndjson with channel predicate", "capture_source": "unified_log_export", - "capture_reference": "/Users/macos-user/.projects/stack-research/memory-lab-1-review/notes/experiments/roadtrip-capture-checklist.md", + "capture_reference": "notes/experiments/roadtrip-capture-checklist.md (source repository)", "source_archive": "/Volumes/RoadtripCapture/iphone-trip.logarchive (per file footer; encrypted volume per checklist)", "device_id": "iphone-roadtrip-01", "trace_interval": "2026-06-07T14:22:56-04:00/2026-06-07T17:53:07-04:00",