fix(tasks): a task_yield on one thread no longer suspends every other (#739 item 3)

[InauguralPhysicist]

Third of #739's items. Item 1 (trace tape) landed in #761, item 2 (exit request) in #762.

The bug

The cooperative scheduler lives on EigsThread — tasks are single-threaded by construction — but the suspend request that drives it was a plain global (vm.c), set at four sites and polled by every vm_run on every thread at CASE(CALL).

So a task_yield on the main thread drove an unrelated OS worker's next call into vm_suspend_halt. Having no scheduler of its own, the victim's task_save_slice(task_current_running()) was handed NULL and saved nothing, and vm_run returned NULL mid-evaluation with its frames deliberately undrained — the suspend path preserves them on purpose, because a real suspend is going to resume. A spawned worker therefore produced null instead of its result, with no error anywhere.

Reproduced, with a control

Two workers each computing 400000, while main drives a task_yield loop:

program	result
workers + main yielding	w1=null w2=null (3 of 3 runs)
identical program, yields removed	w1=400000 w2=400000 (3 of 3 runs)

The control is load-bearing here: two earlier attempts of mine were bad instruments rather than negative results — one used join of w (the string join builtin, not thread_join), so both arms returned ""; the other never interleaved at all, because main blocked in thread_join and the queued tasks only drained at exit.

The fix, and one deviation from the sketch

The flag is per-thread now. The issue proposes moving it into TaskScheduler; I put it on EigsThread beside the task_sched it drives instead. Same scope and same correctness, but the poll is on the hot CASE(CALL) path — inside the scheduler it would need:

g_task_sched && ((TaskScheduler *)g_task_sched)->suspend_request

two dependent loads plus a branch, paid by every thread that has no scheduler at all, which is nearly all of them. On EigsThread it stays a single load off the already-hot eigs_current, with no NULL check.

Validation

tests/test_tasks.eigs gains the case — the suite had no test where cooperative tasks and OS threads ran at the same time. Validated red without the fix: expected 400000, got null on both workers.

• release 3274/3274; asan+ubsan 3278/3278, leak tally 0
• TSan 11/11, including its seeded-race self-validation (seeded race detected (10 warnings) — the gate is live), so the pass is not a silently-dead gate
• cooperative tasks 79/79; freestanding stage 1+2 OK; jit-smoke; embed stack soak OK

docs/SPEC.md and docs/CONCURRENCY.md now state the scoping rule — the two concurrency models compose, and task_yield hands control to the next task on the calling thread only.

Refs #739 — item 4 (s_osr_threshold, g_sandbox_*, g_stream_file, g_model, g_db_conn) remains open, and now has a live symptom: see the RSS-growth finding in the issue thread.

🤖 Generated with Claude Code

[Copilot]

Pull request overview

This PR fixes a concurrency bug in the EigenScript runtime where a cooperative-task suspension request (task_yield / task joins) was incorrectly scoped process-wide, causing unrelated OS threads to hit the suspend path and silently return null mid-evaluation. The fix makes the suspend request per-OS-thread (alongside the per-thread task scheduler), and documents/tests the composed behavior of cooperative tasks with OS-thread workers.

Changes:

• Move g_task_suspend_request from a process-global variable to a per-thread bridge macro backed by EigsThread::task_suspend_request.
• Add a regression test that runs OS-thread workers while the main thread repeatedly task_yields to ensure workers are unaffected.
• Update SPEC/concurrency docs and changelog to reflect the per-thread scoping rule and prior failure mode.

Reviewed changes

Copilot reviewed 7 out of 7 changed files in this pull request and generated no comments.

Show a summary per file

File	Description
tests/test_tasks.eigs	Adds regression coverage proving task_yield on one thread does not suspend other OS threads.
src/vm.h	Removes the global extern and documents the new per-thread scoping via bridge macro.
src/vm.c	Removes the process-global definition so the runtime uses the per-thread bridge macro instead.
src/eigenscript.h	Adds EigsThread::task_suspend_request and defines g_task_suspend_request as a per-thread bridge macro.
docs/SPEC.md	Specifies that tasks (and yielding) are scoped to the calling OS thread.
docs/CONCURRENCY.md	Documents composition of cooperative tasks with OS threads and the scoping rationale.
CHANGELOG.md	Records the fixed behavior and user-visible symptom (null results on unrelated threads).

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[InauguralPhysicist]

Closing: this commit landed in #764, not on its own merits.

I branched the item-4 work off this branch with git checkout -b, so it inherited this commit and #764's merge carried it to main — tests/test_tasks.eigs's regression case, the vm.h/vm.c changes, and the SPEC/CONCURRENCY scoping notes are all in b70e649. Verified present on main.

The work is unchanged and its gates are unchanged (release 3274/3274, asan 3278/3278 tally 0, TSan 11/11 with the seeded-race self-check live, red-then-green as expected 400000, got null). But it merged under a PR whose description covered only item 4, so it did not get reviewed as its own change. Noted on #764 as well. The rationale for putting the flag on EigsThread rather than inside TaskScheduler — the hot CASE(CALL) poll — is in the description above and in the commit message, both still readable.