Skip to content

matmul/softmax have two implementations selected by EIGENSCRIPT_EXT_MODEL — a build flag forking a numeric kernel that nothing diffs #745

Description

@InauguralPhysicist

Found in a modularity review of main @ 8e8970a. Filed as a structural concern; I tried and failed to demonstrate an actual divergence, so the severity below is deliberately hedged.

What

ne_matmul_buf and ne_softmax_buf have two independent implementations selected by a build flag:

  • naive i‑k‑j loops at src/builtins_tensor.c:21 and :37, guarded by #if !(EIGENSCRIPT_EXT_MODEL)
  • NE_TILE_SIZE-blocked versions at src/model_infer.c:18 and :46 (tiling loop at :52-53)

The matmul / softmax builtins call them unconditionally (builtins_tensor.c:370, 394, 411, 434), so which kernel runs is decided by -DEIGENSCRIPT_EXT_MODEL (Makefile:56 vs :76). The same pattern repeats for json_obj_get (builtins.c:2703 static fallback vs model_internal.h:209), which backs the json_path builtin.

Why it matters in principle

Tiling changes floating-point accumulation order. If the two kernels ever disagree by an ULP, make and make http/make full compute different answers for the same script — and the VM stops being the byte-exact oracle ouroboros diffs against (ouroboros/CLAUDE.md: "the bytecode VM is the byte-exact oracle"). No test compares the two kernels against each other.

What I actually measured — no divergence found

I built both variants and ran an 80×80 matmul (larger than NE_TILE_SIZE) over values chosen to be non-exactly-representable, summing all 6400 output cells:

make        → 373059.0300000004
make http   → 373059.0300000004     (identical)

So either the tiling preserves summation order for this shape, or my inputs weren't adversarial enough to expose a difference. I am not claiming the builds currently disagree. The structural fork is real; the numeric consequence is unproven.

Suggested fix

Regardless of whether they currently agree, one of these:

  1. Move the fast kernel into the always-compiled TU so every variant runs the same code, and delete the fallback; or
  2. Keep the fallback but add a test that runs both kernels over the same inputs and asserts byte-equality — which also answers empirically whether the fork has ever mattered.

Option 1 is preferable: a stub standing in for a missing extension should be a slow but identical path, not a second algorithm. A "which build am I on?" dependency in numeric output is exactly the silent-wrong-answer class v0.33.0 targeted, expressed as a build-variant fork rather than a bug.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions