implemet plan

[drzo]

You asked

Implement the plan:

CogDiod — Exhaustive Feature Implementation Plan

The current codebase has a well-structured C kernel with Dis VM, PLN rules, DisTyx 9P dispatch, an Elbo compiler, an LLM bridge, 3 atom packages, 4 unit-test suites, a CLI, a JSON bridge, and 5 language integrations. The following plan identifies every gap and maps each to a concrete deliverable.

---

Phase 1 — Core Kernel Hardening

1.1 STI-Priority Run Queue (mentioned in kernel comment but unimplemented)

• Replace the circular-buffer run queue with a binary min-heap sorted by av.sti (descending)
• Implement rq_push/rq_pop with sift-up/sift-down in cogdiod_kernel.c
• Atoms with higher STI are scheduled first; ties broken by UUID

1.2 AtomSpace Persistence (save/load)

• Add cogdiod_save(CogDiodKernel* k, const char* path) — iterates the atom pool and serialises all atoms (uuid, type, name, TV, AV, outgoing link UUIDs) plus the package list to a binary or JSON file
• Add cogdiod_load(const char* path) — restores a complete kernel from a saved file; re-links channels after all atoms are re-spawned
• Add snapshot and restore commands to the CLI and bridge

1.3 Garbage Collection / LTI-Eviction

• Add periodic LTI-based GC thread: atoms with lti < LTI_THRESHOLD and sti == 0.0 for more than N ECAN cycles are marked ATOM_DYING and destroyed
• Implement cogdiod_gc_sweep(CogDiodKernel* k) called by the ECAN thread
• Send MSG_DESTROY to all incoming-channel neighbours before reclamation

1.4 STI Normalization and Rent Collection

• Add cogdiod_ecan_collect_rent(CogDiodKernel* k): deducts a small periodic STI from every atom to prevent indefinite accumulation
• Add cogdiod_ecan_normalize(CogDiodKernel* k): rescales all STI values so total_sti matches sti_funds; called when total drifts by >10%
• STI funds replenishment: add sti_funds restoration from LTI reservoir

1.5 Episodic TV History (ring buffer already exists, use it consistently)

• Call push_tv_history(atom) in every cogdiod_set_tv before overwriting
• Expose /ai/atoms/<uuid>/history/<version> in DisTyx (returns {strength, confidence} for a given version index)
• Add episodic <uuid> <version> to the CLI

---

Phase 2 — New Atom Types (.elbo source + .c stub + package registration)

Each new type needs three files: packages/<type>/type.elbo, packages/<type>/type_pkg.c, and a concept_node-style stub builder. All must be wired into the Makefile.

2.1 PredicateNode — holds a predicate name and TV; responds to MSG_INFER by evaluating grounded predicates

2.2 NumberNode — stores a numeric value in registers; arithmetic messages (MSG_CUSTOM with opcode field)

2.3 VariableNode — placeholder in pattern templates; responds to MSG_QUERY with its binding or UNBOUND

2.4 ListLink — ordered list of child atom UUIDs; MSG_SOURCE_CHANGED rebroadcasts to all children

2.5 InheritanceLink — subtype/supertype relation; triggers PLN_DED when either end changes TV

2.6 SimilarityLink — symmetric relation using PLN_REV-based TV combination

2.7 MemberLink — set membership; participates in abduction (OP_PLN_ABD)

2.8 EquivalenceLink — bidirectional implication; merges TV with PLN_REV in both directions

2.9 AndLink / OrLink / NotLink — Boolean connectives using TruthValue min/max/complement formulas

2.10 ExecutionLink — triggers GroundedSchemaNode.execute() on MSG_INFER

2.11 ContextLink — scoped evaluation context; wraps a child link with a context atom

2.12 PredictiveImplicationLink — temporal implication (uses OP_PLN_TMP with lag parameter)

2.13 AtTimeLink / TimeNode — represents an event at a specific timestamp; uses av.lti to encode time

2.14 BindLink — pattern match root; see Phase 3

2.15 GroundedSchemaNode / GroundedPredicateNode — callable atoms; ep_init stores a function pointer via kernel_ref and a registration table

---

Phase 3 — Pattern Matching Engine

3.1 Unification Engine (src/pm/unify.c)

• Implement unify(AtomIsolate* pattern, AtomIsolate* graph, Bindings* out) — recursive Prolog-style unification with variable binding
• Handle VariableNode wildcards, typed variables, and deep link traversal
• Return a binding list {variable_uuid → matched_uuid}

3.2 Hypergraph Pattern Traversal (src/pm/pattern_match.c)

• Implement cogdiod_match(CogDiodKernel* k, uint64_t bind_link_uuid, MatchCallback cb, void* ctx) — traverses all atoms reachable from the kernel's pool and calls cb for each binding satisfying the BindLink pattern
• Use the VariableNode-binding engine from 3.1

3.3 BindLink Package

• ep_on_message for MSG_QUERY: runs pattern match and sends MSG_CUSTOM with results on outgoing channels

3.4 DisTyx Query Path

• Add /ai/query as a write-then-read path: Twrite a BindLink uuid → triggers match; Tread returns JSON array of binding maps

3.5 Bridge match operation

• {"op":"match","bind_link_uuid":N} → {"bindings":[{"var":uuid,"val":uuid},...]}

---

Phase 4 — Elbo Compiler Completeness

4.1 let and define forms

• Parser: recognise (let ((x expr) ...) body) and (define name expr)
• Emit OP_STORE into named heap slots; maintain a compile-time symbol table
• Round-trip: elm_save already writes a symbol table section; ensure elbo_compile populates it

4.2 Conditionals (if / cond / when)

• Emit OP_JEQ/OP_JNE with forward patch addresses; fill in target after compiling then/else branches
• Add (if test then else), (cond (test expr) ...), (when test expr...) to the grammar

4.3 Module import forms

• (import PLN "pln.m" (revise)) lowers to a reference in the symbol table with a stub dispatch opcode
• At link time, elm_load_file resolves import symbols against other loaded packages

4.4 Lambda and Closures (basic)

• (defun name (args...) body) already partially supported; extend to support anonymous (fn (args) body)
• Emit a call-return frame using OP_CALL/OP_RET

4.5 Arithmetic literals and float immediates

• Allow (set-tv 0.8 0.9) to inline float constants via OP_LOAD from a constant pool embedded after the bytecode section in the .elm file

4.6 Compile .elbo files to real .elm packages

• Add elbo_compile_file(const char* src_path, const char* out_path) that reads source, compiles with elbo_compile(), and writes with elm_save()
• Wire into the Makefile so packages/*/type.elbo → packages/*/type.elm is a build rule

---

Phase 5 — DisTyx Protocol Completeness

5.1 LLM Namespace (/ai/llm/<uuid>/*)

• Add path-parser branch in distyx_dispatch for /ai/llm/
• model write → calls cogdiod_llm_attach
• prompt write → calls cogdiod_llm_infer(stream=1)
• response read → drains the token ring buffer
• params read/write → JSON CogLlmParams
• stats read → JSON {tokens_per_sec, total_tokens, model_name}

5.2 Missing Atom Paths

• /ai/atoms/<uuid>/name — read: atom name string
• /ai/atoms/<uuid>/type — read: atom type string
• /ai/atoms/<uuid>/lti — read/write: LTI value
• /ai/atoms/<uuid>/history/<n> — read: nth TV in ring buffer
• /ai/atoms/<uuid>/links/in — read: JSON array of incoming channel source UUIDs
• /ai/atoms/<uuid>/ctl — write "gc\n" to destroy, "infer\n" to enqueue

5.3 Package Path

• /ai/packages/<type>.elm — read: returns raw package bytecode for download
• /ai/packages/<type>.elm — write: loads a new .elm package into the cache

5.4 DT_OP_STAT

• Implement DT_OP_STAT in distyx_dispatch returning a qid/size/mtime struct for any valid path

5.5 TCP Accept Loop Completion

• distyx_start_tcp currently starts the server thread; verify the accept loop handles multiple concurrent clients and implements proper 9P session framing (Tversion/Rversion, Tattach/Rattach, fid table per session)

---

Phase 6 — ECAN and Hebbian Learning Enhancements

6.1 Hebbian Weight Decay

• In cogdiod_hebbian_update, also apply weight *= DECAY_FACTOR for all channels that have not fired recently (now - last_fire_time > DECAY_WINDOW)
• Add cogdiod_hebbian_decay_all(CogDiodKernel* k) called from the ECAN thread

6.2 Hebbian-Guided Channel Pruning

• Channels with weight < PRUNE_THRESHOLD are candidates for cogdiod_unlink
• Add cogdiod_prune_weak_channels(CogDiodKernel* k, float threshold)

6.3 ECAN Importance Diffusion Tuning

• Make the 50% spread factor (currently hardcoded in OP_ECAN_SP) a kernel-level parameter k->ecan_spread_factor
• Add cogdiod_set_ecan_params(CogDiodKernel* k, float spread, float decay, float rent)

6.4 Forgetting Curve (LTI Decay)

• LTI decreases by lti *= LTI_DECAY each ECAN cycle unless the atom receives an MSG_ATTEND
• Atoms with lti < LTI_EVICT_THRESHOLD are GC candidates (see Phase 1.3)

---

Phase 7 — Language Integration Completeness

7.1 Racket #lang elbo macro tower (cogdiod.rkt)

• Implement a real #lang elbo reader extension using Racket's syntax/module-reader
• Define macros: atom, implication, evaluation, before, during, after (temporal)
• Each macro expands to a bridge spawn + link call sequence

7.2 Clojure Persistent AtomSpace (cogdiod.clj)

• Replace the flat-map AtomSpace with a persistent HAM-Trie (use Clojure's native {}—it's already a HAMT)
• Version every write: (set-tv space uuid tv) returns a new space; keep a vector of all versions
• Implement (get-tv-at space version uuid) for episodic recall

7.3 Guile Environment Chaining (cogdiod.scm)

• Implement make-child-env as a proper Guile module using make-module and module-add!
• eval-in-env env expr should correctly shadow bindings without mutating parents
• Add env->json env to serialise a hypothesis environment for DisTyx transport

7.4 Maude Meta-Rule Rewriting (cogdiod.maude)

• Complete the COGDIOD-META module: implement the counter-based k >= 5 rule application tracking
• Add a BOOST operation that edits the rule formula from pln-deduction to pln-boosted(factor)
• Add a Clojure executor test that drives the Maude rewriting cycle through 10 iterations and verifies TV improvement

7.5 Perl AUTOLOAD Context (cogdiod.pl)

• Complete the use overload section with '@{}' (array context = outgoing links) and '0+' (numeric = STI)
• Implement AUTOLOAD for any method matching ecan_* or pln_*: parse the method name, map to a bridge call, and dispatch
• Add a test script that exercises all 4 context forms and at least 3 AUTOLOAD-generated methods

---

Phase 8 — Bridge Protocol Completeness

8.1 PLN operations over bridge

• {"op":"pln_deduce","ant_uuid":N,"impl_uuid":M} → computes pln_deduce using the two atoms' TVs and returns the result TV (without mutating either atom)
• {"op":"pln_revise","uuid":N,"strength":s,"confidence":c} → revises atom N's TV with the given evidence and stores it
• {"op":"pln_infer_chain","uuids":[...]} → applies pln_deduce along a chain of ImplicationLinks

8.2 Snapshot and Restore

• {"op":"snapshot"} → returns complete AtomSpace JSON (all atoms, links, TVs, STI, LTI)
• {"op":"restore","atoms":[...]} → rebuilds the bridge's internal atom store from a snapshot

8.3 Rewrite Rule

• {"op":"rewrite_rule","from_pattern":"...","to_pattern":"..."} → adds a pattern-rewrite rule to the bridge's rule table; applied on every subsequent spawn or set_tv

8.4 Episodic Lookup

• {"op":"episodic","uuid":N,"version":V} → returns TV at history version V from the atom's ring buffer

8.5 Attention Query

• {"op":"top_sti","n":10} → returns the top N atoms by STI
• {"op":"sleeping"} → returns all atoms in ATOM_SLEEPING state

---

Phase 9 — Testing and Fuzzing

9.1 PLN Property-Based Tests

• Add tests/test_pln_props.c: for each PLN rule verify: (a) output strength ∈ [0,1], (b) output confidence ≤ min(input confidences), (c) idempotence of revision with itself, (d) deduction with strength=0 gives strength=0

9.2 Elbo Compiler Tests (test_elbo_compiler.c)

• Compile each form (concept, implication, pln-ded, set-tv, halt, send, recv, spawn) and verify bytecode bytes
• Round-trip test: compile → elm_save → elm_load_file → execute

9.3 Pattern Matching Tests (test_pattern_match.c)

• Test BindLink with a single variable, typed variable, and nested link
• Test no-match and multi-match scenarios

9.4 Fuzz DisTyx paths (fuzz_distyx.c — already exists, extend)

• Add corpus entries for LLM paths, package paths, and stat paths
• Verify no crash on malformed UTF-8 paths, oversized fid values, or negative buf_len

9.5 Bridge Protocol Fuzz (fuzz_bridge.c)

• Fuzz the JSON parser in cogdiod_bridge.c with random JSON fragments
• Verify no heap corruption or lock leaks

9.6 Integration Test Extension (test_cogdiod.c)

• Add Phase 3 pattern match scenario (BindLink matching ConceptNode by type)
• Add LLM stub scenario (attach with no model path, verify graceful -1 return)
• Add persistence round-trip: save → destroy → load → verify atom count and TVs

---

Phase 10 — Tooling and Documentation

10.1 Visualization Tool (tools/cogdiod_graph.html — already exists, enhance)

• Add live WebSocket feed from the bridge server: on every spawn/link/set_tv event push a JSON delta to the page
• Render TV as node colour, STI as node size, channels as directed edges with Hebbian weight as edge width

10.2 CLI enhancements (tools/cogdiod_cli.c)

• Add cogdiod-cli pln-infer-chain <uuid1> <uuid2> ...
• Add cogdiod-cli top-sti <n>
• Add cogdiod-cli load-package <path>
• Add cogdiod-cli compile <elbo_file> → compiles to .elm and loads

10.3 cogdiod-init quickstart script

• Shell script that: builds the kernel, loads all packages, starts the bridge, spawns a default ConceptNode and ImplicationLink, and prints the resulting UUIDs

10.4 API Reference (docs/API.md)

• Document every function in cogdiod.h, cogdiod_llm.h, distyx.h, elbo_compiler.h
• Document every bridge operation with request/response examples
• Document every DisTyx path with semantics, mode, and byte format

10.5 Architecture Decision Records (docs/adr/)

• ADR-001: Why Dis VM over WASM
• ADR-002: Why 9P/Styx over REST
• ADR-003: Why djb2 for type hashing
• ADR-004: PLN formula source and derivation

---

Dependency Order and Priority

Priority	Phase	Key Deliverable
P0	1.1	STI-priority run queue (correctness)
P0	1.3	GC sweep (prevents memory growth)
P0	5.5	TCP accept loop (DisTyx is non-functional without it)
P1	4.2	Conditionals in Elbo compiler
P1	4.6	.elbo → .elm build pipeline
P1	2.1–2.9	Core missing atom types
P1	3.1–3.4	Pattern matching engine
P2	5.1–5.4	DisTyx LLM + missing paths
P2	6.1–6.4	ECAN tuning and Hebbian decay
P2	8.1–8.5	Bridge protocol completeness
P3	7.1–7.5	Language integration completeness
P3	9.1–9.6	Full test coverage
P4	10.1–10.5	Tooling and documentation

Show less