NOESIS: Empirical Findings & Case Study on Hidden-State Memory Injection

NOTICE: SCIENTIFIC CASE STUDY / NEGATIVE RESULT This repository documents the architecture, implementation, and empirical evaluation of NOESIS — a memory-centric, inference-time learning architecture combining Test-Time Training (TTT) and Titans Neural Long-Term Memory (LMM).

Key Conclusion: Injecting un-aligned vector memory embeddings into a frozen LLM's hidden-state representation manifold fails to convey verbal semantic information without task-supervised token-level projection training.

Read the complete chronological breakdown below and in FINDINGS.md.

---

Executive Summary

NOESIS explores whether long-term associative memory (Titans LMM) and sequence-local fast-weight adaptation (TTT) can enable persistent context-free memory recall in a frozen LLM backbone (Qwen/Qwen3.5-0.8B) without relying on prompt text injection or KV-cache history.

Through rigorous systematic controls (KV-Cache Purging, Negative-Control Noise Injections, and $N=5$ Multi-Fact Discrimination), we disproved three common false-positive illusions (KV-cache Echo, Confident-Tone Perturbation, and $N=1$ Coincidence) and established an objective baseline accuracy of 0.00% (0/22) for unguided vector-to-hidden memory injection.

---

Chronological Findings & Methodology

Phase 1: Core Mechanics Fixes (Deterministic Verification)

1. TTT Look-Ahead Leakage: Discovered that fast weights $W_{fast}$ were being applied retroactively across the entire sequence. Replaced with step-by-step causal prediction where token prediction at step $t$ depends strictly on $W_{fast, t-1}$.
2. Titans Gate Hindsight Target: Fixed hindsight target leakage where gate targets depended on post-update representations. Replaced target with pre-update reconstruction loss and surprise signal.
3. Cross-Space Projection Bug: Discovered invalid key-space projection $W_{key}(v_r)$ applied to value-space vectors in Titans.retrieve(). Replaced with normalized signal-strength ratio $|v_r| / (|k_q| + \epsilon)$.
4. Deterministic Validation: Hash-backed deterministic embeddings verified causal surprise discrimination (0.49 vs 1.45 random) and gate retention learning (decay resistance improved from 42.7% to 10.0% loss).

Phase 2: Real Backbone Integration & Scale Stability

5. Numerical Explosion: Raw pre-head hidden states ($d_{model}=1024$, Öklid norm $|x| \approx 139$) caused TTT gradients to explode to $10^{23}$ and Titans loss to $10^{32}$.
6. Representation Normalization Adapter: Implemented CanonicalRepresentation enforcing scale-invariant unit norm: $$\text{Canonical}(x) = \frac{\text{LayerNorm}(x)}{\sqrt{d_{model}}}$$ Achieved perfect numerical stability across TTT ($0.00096$ surprise) and Titans ($9.16 \times 10^{-8}$ loss).

Phase 3: Systematic Disproof of False-Positive Illusions

7. The KV-Cache Echo Trap: Initial "100% recall" outputs (e.g., answering "mor" after 15 noise turns) were proved to originate entirely from the backbone's transformer KV-cache history, not Titans vector memory. Purging KV-cache (backbone_state = None) caused the model to revert to "I don't remember".
8. Norm-Threshold Heuristic Bug: Replaced fragile norm < 2.0 heuristic in compute_logits with an explicit is_canonical contract, eliminating silent double-scaling bugs.
9. The "Confident-Tone" Illusion: Un-aligned hidden vector injection produced confident responses like "Yes, I remember". A Negative Control (injecting pure random Gaussian noise) produced identical confident outputs (e.g., "Kael's favorite color is black"), proving that hidden perturbations induce confident tone shifts without carrying semantic facts.
10. The $N=1$ Coincidence Trap: A single conceptual hit ("kırmızı" $\rightarrow$ "fiery/warm") was disproved by an $N=5$ multi-fact test (yeşil, mavi, sarı, mor, turuncu), which yielded $0/5$ semantic alignment.

Phase 4: Automated Benchmark & Final Result

11. $W_{bridge}$ Auto-Encoder Alignment: Trained a linear bridge $W_{bridge}: \text{Value Space} \rightarrow \text{Backbone Space}$ achieving 92.6% geometric cosine similarity.
12. Automated Protocol Benchmark: Evaluated 50 synthetic facts across 5 categories with an automated train/test split under complete KV-cache purging.
13. Final Result: 0.00% Accuracy (0/22) across both in-category held-out facts and out-of-category domains.

---

Key Scientific Insight

Geometrical auto-encoding is insufficient for semantic interpretation.

A frozen LLM's hidden-state manifold cannot interpret vectors injected from an external un-aligned space (such as Titans Value space), even when geometrically normalized and auto-encoded. Alignment between independent vector spaces cannot be assumed; it requires task-supervised, token-level end-to-end training.

---

Installation & Usage

git clone https://github.com/kaelvalen/noesis.git
cd noesis
pip install -r requirements.txt

Running Smoke Tests

PYTHONPATH=. python -m pytest tests/test_smoke.py -v

Running the Automated Memory Bridge Benchmark

PYTHONPATH=. python tests/benchmark_bridge.py

---

Repository Structure

noesis/
├── noesis/
│   ├── core/         # Engine, backbone, representation contract
│   ├── memory/       # Titans LMM, sparse cache, vector DB
│   ├── adaptation/   # TTT, MoE, consolidator
│   └── ingestion/    # Web and local file ingestion
├── tests/
│   ├── test_smoke.py         # Functional verification tests
│   └── benchmark_bridge.py   # Automated 50-fact KV-cache purge benchmark
├── FINDINGS.md       # Full detailed scientific case study (English)
├── README.md         # Architecture overview & findings summary
└── setup.py