Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions packages/senpi-codemode/CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,8 @@

### Added

- JavaScript eval cells now expose CommonJS `require`, `module.exports`, `exports`, `__filename`, and `__dirname`, resolved from the session cwd.

### Changed

### Fixed
Expand Down
3 changes: 2 additions & 1 deletion packages/senpi-codemode/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -46,7 +46,7 @@ task-tool names are known.

| Language | Default | Runtime | Notes |
| --- | --- | --- | --- |
| `js` | enabled | In-process worker on senpi's own runtime (Bun or Node.js 24+) | Supports top-level `await` and `return`; the eval prompt's runtime line follows the kernel. |
| `js` | enabled | In-process worker on senpi's own runtime (Bun or Node.js 24+) | Supports top-level `await`/`return` and CommonJS `require`/`module.exports`; the eval prompt's runtime line follows the kernel. |
| `py` | enabled | `python3` or `python` | Optional interpreter detected at session start. |
| `rb` | disabled | `ruby` | Optional interpreter detected at session start. |
| `jl` | disabled | `julia` | Optional interpreter detected at session start. |
Expand Down Expand Up @@ -122,6 +122,7 @@ options object and asynchronous helpers are `await`-able.
| `output(ids, format?, offset?, limit?)` | Delegates transcript retrieval to the configured active `taskTools.output` tool. |
| `parallel(thunks)` | Runs thunks through the configured bounded pool while preserving input order. |
| `pipeline(items, ...stages)` | Applies stages left to right with a barrier between stages. |
| `require` / `module.exports` | JavaScript only. CommonJS bindings resolved from the session cwd (`__filename` is `eval-cell.cjs`). |
| `log(message)` / `phase(title)` | Emits progress text and starts a status phase. |

When a `tool.<name>()` call fails argument validation, the error delivered back
Expand Down
32 changes: 32 additions & 0 deletions packages/senpi-codemode/changes.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,37 @@
# senpi-codemode fork changes

## JavaScript eval CommonJS bindings (2026-09-02)

### What changed

- `packages/senpi-codemode/src/kernels/js/worker-runtime.js` installs cwd-resolved CommonJS
bindings (`require`, `module`, `exports`, `__filename`, `__dirname`) on the persistent JS
worker via `createRequire`, and resyncs `exports` to `module.exports` before each cell.
The session cwd is `resolve()`d first so a relative or empty cwd still yields an absolute
filename for `createRequire`.
- `packages/senpi-codemode/src/kernels/js/worker-indirect-eval.js` wraps user cells so those
five CommonJS names bind as function parameters, matching Node's CJS wrapper. Top-level
declarations that rebind those names are left unrewritten so they SyntaxError instead of
overwriting the persistent `require`/`module`.
- `packages/senpi-codemode/src/kernels/js/prelude.ts` and `src/prompt/eval-prompt.ts` document
the bindings on both Bun and Node runtime lines.

### Why

- JS cells run through ESM-worker `eval`, so Node never wraps them with CJS parameters.
`typeof require` was `undefined`, which blocked loading CJS packages and local `.cjs` files.

### Why an extension could not handle it

- Cell wrapping and worker globals are owned by the eval kernel, not by a host tool or
extension hook. An external wrapper cannot inject `require` into the persistent worker VM.

### Expected merge conflict zones

- MEDIUM in `src/kernels/js/worker-runtime.js` around `#installGlobals` and cell `run`.
- MEDIUM in `src/kernels/js/worker-indirect-eval.js` around `wrapUserCode`.
- LOW in `src/prompt/eval-prompt.ts` around the JS runtime line.

## Binary skill resolution and stdout-safe miss reporting (2026-09-02)

### What changed
Expand Down
1 change: 1 addition & 0 deletions packages/senpi-codemode/src/kernels/js/prelude.ts
Original file line number Diff line number Diff line change
Expand Up @@ -12,4 +12,5 @@ export const JAVASCRIPT_KERNEL_PRELUDE = [
"agent(prompt, options?): delegate work through the reserved agent bridge.",
"parallel(thunks): run async thunks through the configured bounded pool; preserves order and rethrows the lowest-index error after all settle.",
"pipeline(items, ...stages): map items through staged async transforms with a barrier between stages.",
"require(specifier), module.exports, exports, __filename, __dirname: CommonJS bindings resolved from cwd.",
].join("\n");
Original file line number Diff line number Diff line change
Expand Up @@ -11,14 +11,15 @@ export async function awaitMaybePromise(value) {

export function wrapUserCode(code) {
const persistentCode = persistTopLevelDeclarations(code);
if (/\breturn\b/u.test(persistentCode)) return `(async () => {\n${persistentCode}\n})()`;
return `(async () => {\n${captureLastExpression(persistentCode)}\n})()`;
const body = /\breturn\b/u.test(persistentCode) ? persistentCode : captureLastExpression(persistentCode);
return `(async (exports, require, module, __filename, __dirname) => {\n${body}\n})(globalThis.exports, globalThis.require, globalThis.module, globalThis.__filename, globalThis.__dirname)`;
Comment thread
cubic-dev-ai[bot] marked this conversation as resolved.
}

const IDENTIFIER_START_RE = /[$_\p{ID_Start}]/u;
const IDENTIFIER_CONTINUE_RE = /[$_\p{ID_Continue}\u200c\u200d]/u;
const IDENTIFIER_RE = /^[\p{ID_Start}$_][\p{ID_Continue}\u200c\u200d]*$/u;
const DECLARATION_KEYWORDS = new Set(["const", "let", "var"]);
const COMMONJS_RESERVED_NAMES = new Set(["exports", "require", "module", "__filename", "__dirname"]);
const REGEX_PREFIX_KEYWORDS = new Set([
"await",
"case",
Expand Down Expand Up @@ -265,6 +266,7 @@ function rewriteDeclaration(code, declarationStart, start, end, keyword) {
const bindings = [];
collectPatternNames(pattern, bindings);
if (bindings.length === 0) return undefined;
if (bindings.some((name) => COMMONJS_RESERVED_NAMES.has(name))) return undefined;

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P3: The guard makes a reserved-name declaration "SyntaxError against the CJS wrapper parameters instead of overwriting the persistent bindings", but that holds only for const/let. A top-level var module = ... (or var require/var exports/var __filename) is left in the cell body unchanged, and in sloppy mode var may redeclare a function parameter without error — it silently shadows the wrapper parameter for that cell only. A cell ending with var exports = {}; exports.foo = 1 therefore writes to a local object and never reaches module.exports, silently diverging from the documented CommonJS behavior with no error surfaced. Consider handling var reserved bindings explicitly (error or rewrite them) or documenting the inconsistency, and add a test covering var declarations.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At packages/senpi-codemode/src/kernels/js/worker-indirect-eval.js, line 269:

<comment>The guard makes a reserved-name declaration "SyntaxError against the CJS wrapper parameters instead of overwriting the persistent bindings", but that holds only for `const`/`let`. A top-level `var module = ...` (or `var require`/`var exports`/`var __filename`) is left in the cell body unchanged, and in sloppy mode `var` may redeclare a function parameter without error — it silently shadows the wrapper parameter for that cell only. A cell ending with `var exports = {}; exports.foo = 1` therefore writes to a local object and never reaches `module.exports`, silently diverging from the documented CommonJS behavior with no error surfaced. Consider handling `var` reserved bindings explicitly (error or rewrite them) or documenting the inconsistency, and add a test covering `var` declarations.</comment>

<file context>
@@ -265,6 +266,7 @@ function rewriteDeclaration(code, declarationStart, start, end, keyword) {
 		const bindings = [];
 		collectPatternNames(pattern, bindings);
 		if (bindings.length === 0) return undefined;
+		if (bindings.some((name) => COMMONJS_RESERVED_NAMES.has(name))) return undefined;
 		if (preserveDeclaration) {
 			for (const name of bindings) assignments.push(`globalThis[${JSON.stringify(name)}] = ${name};`);
</file context>

if (preserveDeclaration) {
for (const name of bindings) assignments.push(`globalThis[${JSON.stringify(name)}] = ${name};`);
continue;
Expand Down
31 changes: 30 additions & 1 deletion packages/senpi-codemode/src/kernels/js/worker-runtime.js
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
// allow: SIZE_OK — private runtime state and installed globals must stay in one worker module.
import { mkdir, readFile, stat, writeFile } from "node:fs/promises";
import { createRequire } from "node:module";
import { dirname, isAbsolute, join, normalize, resolve, sep } from "node:path";
import { inspect } from "node:util";
import { awaitMaybePromise, indirectEval, wrapUserCode } from "./worker-indirect-eval.js";
Expand All @@ -17,7 +18,7 @@ export class JsWorkerRuntime {
#hooks = null;

constructor(options) {
this.#cwd = options.cwd;
this.#cwd = resolve(options.cwd);
this.#parallelPoolWidth = options.parallelPoolWidth;
this.#localRoots = { ...(options.localRoots ?? {}) };
if (options.artifactsDir && !this.#localRoots.local) this.#localRoots.local = join(options.artifactsDir, "local");
Expand All @@ -35,13 +36,41 @@ export class JsWorkerRuntime {
({ prelude, code: cellCode } = prepared);
}
if (prelude) indirectEval(prelude, `${cellId}:prelude`);
this.#syncCommonJsExports();
return await awaitMaybePromise(indirectEval(wrapUserCode(cellCode), cellId));
} finally {
this.#hooks = null;
}
}

#installCommonJs() {
const filename = join(this.#cwd, "eval-cell.cjs");
Comment thread
cubic-dev-ai[bot] marked this conversation as resolved.
const require = createRequire(filename);
const module = {
id: filename,
filename,
path: this.#cwd,
exports: {},
loaded: false,
children: [],
paths: require.resolve.paths(".") ?? [],
require,
parent: undefined,
};
globalThis.require = require;
globalThis.module = module;
globalThis.exports = module.exports;
globalThis.__filename = filename;
globalThis.__dirname = this.#cwd;
}

#syncCommonJsExports() {
if (!isPlainObject(globalThis.module)) return;
globalThis.exports = globalThis.module.exports;
}

#installGlobals() {
this.#installCommonJs();
globalThis.print = (...values) => this.#emitText("stdout", `${values.map(formatValue).join(" ")}\n`);
globalThis.display = value => this.#display(value);
globalThis.log = message => this.#hooks?.emit({ type: "log", message: String(message) });
Expand Down
2 changes: 1 addition & 1 deletion packages/senpi-codemode/src/prompt/eval-prompt.ts
Original file line number Diff line number Diff line change
Expand Up @@ -138,7 +138,7 @@ Fields:
A detached cell keeps its language kernel busy while it finishes; another language can continue. Do not re-run a detached cell: the same-language busy error names its cell id and output tail. Completion arrives as one notification with the final value/error and buffered output. Stopping a cell interrupts its kernel; the stop result states whether kernel state survived or the kernel was restarted and its variables lost.

{{#if py}}Live event loop: use top-level \`await\` directly; \`asyncio.run(…)\` raises "cannot be called from a running event loop".{{/if}}
{{#if js}}{{#if jsBun}}JS runs in-process on Bun {{jsVersion}}: top-level \`await\`/\`return\` work; \`Bun.*\` builtins available.{{#if bunSkillPath}} MUST READ the bun-1-4 skill at {{bunSkillPath}} before your first js cell — its builtins replace the npm packages you would otherwise install.{{/if}}{{else}}JS runs under Node.js worker: top-level \`await\`/\`return\` work; \`fetch\`/\`Buffer\` available.{{/if}}{{/if}}
{{#if js}}{{#if jsBun}}JS runs in-process on Bun {{jsVersion}}: top-level \`await\`/\`return\` work; \`Bun.*\` builtins available; CommonJS \`require\`/\`module.exports\`/\`exports\`/\`__dirname\`/\`__filename\` work.{{#if bunSkillPath}} MUST READ the bun-1-4 skill at {{bunSkillPath}} before your first js cell — its builtins replace the npm packages you would otherwise install.{{/if}}{{else}}JS runs under Node.js worker: top-level \`await\`/\`return\` work; \`fetch\`/\`Buffer\` available; CommonJS \`require\`/\`module.exports\`/\`exports\`/\`__dirname\`/\`__filename\` work.{{/if}}{{/if}}
{{#if rb}}Ruby: synchronous; helper options are keyword args{{#if spawns}} (e.g. \`output("id", limit: 2)\`){{/if}}; the last expression auto-displays unless it is \`nil\`, an assignment, or a definition (like IRB).{{/if}}
{{#if jl}}Julia: synchronous; helper options are standard keyword args{{#if spawns}} (e.g. \`output("id", limit=2)\`){{/if}}; the last expression auto-displays unless it is an assignment or a definition (like the Julia REPL).{{/if}}
On error, fix and re-run only the failing step. State usually survives a normal error, but a timeout or stop may have restarted the kernel — its message says which. Before rebuilding state, check a sentinel (a variable you defined earlier); only re-establish what is actually gone, since blind re-runs duplicate side effects.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@ Fields:
A detached cell keeps its language kernel busy while it finishes; another language can continue. Do not re-run a detached cell: the same-language busy error names its cell id and output tail. Completion arrives as one notification with the final value/error and buffered output. Stopping a cell interrupts its kernel; the stop result states whether kernel state survived or the kernel was restarted and its variables lost.

Live event loop: use top-level \`await\` directly; \`asyncio.run(…)\` raises "cannot be called from a running event loop".
JS runs under Node.js worker: top-level \`await\`/\`return\` work; \`fetch\`/\`Buffer\` available.
JS runs under Node.js worker: top-level \`await\`/\`return\` work; \`fetch\`/\`Buffer\` available; CommonJS \`require\`/\`module.exports\`/\`exports\`/\`__dirname\`/\`__filename\` work.
Ruby: synchronous; helper options are keyword args (e.g. \`output("id", limit: 2)\`); the last expression auto-displays unless it is \`nil\`, an assignment, or a definition (like IRB).
Julia: synchronous; helper options are standard keyword args (e.g. \`output("id", limit=2)\`); the last expression auto-displays unless it is an assignment or a definition (like the Julia REPL).
On error, fix and re-run only the failing step. State usually survives a normal error, but a timeout or stop may have restarted the kernel — its message says which. Before rebuilding state, check a sentinel (a variable you defined earlier); only re-establish what is actually gone, since blind re-runs duplicate side effects.
Expand Down Expand Up @@ -148,7 +148,7 @@ Fields:

A detached cell keeps its language kernel busy while it finishes; another language can continue. Do not re-run a detached cell: the same-language busy error names its cell id and output tail. Completion arrives as one notification with the final value/error and buffered output. Stopping a cell interrupts its kernel; the stop result states whether kernel state survived or the kernel was restarted and its variables lost.

JS runs under Node.js worker: top-level \`await\`/\`return\` work; \`fetch\`/\`Buffer\` available.
JS runs under Node.js worker: top-level \`await\`/\`return\` work; \`fetch\`/\`Buffer\` available; CommonJS \`require\`/\`module.exports\`/\`exports\`/\`__dirname\`/\`__filename\` work.

On error, fix and re-run only the failing step. State usually survives a normal error, but a timeout or stop may have restarted the kernel — its message says which. Before rebuilding state, check a sentinel (a variable you defined earlier); only re-establish what is actually gone, since blind re-runs duplicate side effects.
</instruction>
Expand Down Expand Up @@ -245,7 +245,7 @@ Fields:
A detached cell keeps its language kernel busy while it finishes; another language can continue. Do not re-run a detached cell: the same-language busy error names its cell id and output tail. Completion arrives as one notification with the final value/error and buffered output. Stopping a cell interrupts its kernel; the stop result states whether kernel state survived or the kernel was restarted and its variables lost.

Live event loop: use top-level \`await\` directly; \`asyncio.run(…)\` raises "cannot be called from a running event loop".
JS runs under Node.js worker: top-level \`await\`/\`return\` work; \`fetch\`/\`Buffer\` available.
JS runs under Node.js worker: top-level \`await\`/\`return\` work; \`fetch\`/\`Buffer\` available; CommonJS \`require\`/\`module.exports\`/\`exports\`/\`__dirname\`/\`__filename\` work.

On error, fix and re-run only the failing step. State usually survives a normal error, but a timeout or stop may have restarted the kernel — its message says which. Before rebuilding state, check a sentinel (a variable you defined earlier); only re-establish what is actually gone, since blind re-runs duplicate side effects.
</instruction>
Expand Down
125 changes: 124 additions & 1 deletion packages/senpi-codemode/test/js-kernel.test.ts
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
// allow: SIZE_OK — todo 7 parity cases must remain in the plan-listed js-kernel test file.
import { cp, mkdir, mkdtemp, readFile, rm, writeFile } from "node:fs/promises";
import { tmpdir } from "node:os";
import { join } from "node:path";
import { join, relative, resolve } from "node:path";
import { fileURLToPath, pathToFileURL } from "node:url";
import { describe, expect, it } from "vitest";
import type { KernelToHostMessage } from "../src/bridge/protocol.ts";
Expand Down Expand Up @@ -121,6 +121,129 @@ describe("JavaScriptKernel", () => {
});
});

it("requires Node builtins through CommonJS", async () => {
await withKernel(async (kernel) => {
// Given a live JavaScript kernel without an ESM import
// When a cell calls require() for a Node builtin
const run = await runCell(kernel, "return require('node:path').basename('/a/b/c.js')");

// Then the CommonJS binding resolves the builtin
expect(run.result).toMatchObject({ ok: true, valueRepr: '"c.js"' });
});
});

it("requires a relative CommonJS module from the session cwd", async () => {
const root = await mkdtemp(join(tmpdir(), "senpi-codemode-js-cjs-"));
await writeFile(join(root, "module.cjs"), "module.exports = { value: 41 };\n");
const kernel = new JavaScriptKernel({ sessionId: "relative-cjs", cwd: root, parallelPoolWidth: 2 });
try {
// Given a CommonJS module beside the session cell
// When the cell requires it by a relative specifier
const run = await runCell(kernel, 'const mod = require("./module.cjs"); return mod.value + 1');

// Then resolution starts from the session cwd
expect(run.result).toMatchObject({ ok: true, valueRepr: "42" });
} finally {
await kernel.close();
await rm(root, { recursive: true, force: true });
}
});

it("exposes module.exports, exports, __filename, and __dirname", async () => {
const root = await mkdtemp(join(tmpdir(), "senpi-codemode-js-cjs-meta-"));
const kernel = new JavaScriptKernel({ sessionId: "cjs-meta", cwd: root, parallelPoolWidth: 2 });
try {
// Given a live JavaScript kernel
// When a cell inspects CommonJS metadata and writes module.exports
const run = await runCell(
kernel,
[
"exports.mark = true;",
"return {",
" dirname: __dirname,",
" filename: __filename,",
" exportsIsModuleExports: exports === module.exports,",
" requireType: typeof require,",
" exported: module.exports,",
"};",
].join("\n"),
);

// Then cwd-relative CommonJS bindings are available to the cell
expect(run.result.ok).toBe(true);
if (!run.result.ok) return;
expect(JSON.parse(run.result.valueRepr ?? "null")).toEqual({
dirname: root,
filename: join(root, "eval-cell.cjs"),
exportsIsModuleExports: true,
requireType: "function",
exported: { mark: true },
});
} finally {
await kernel.close();
await rm(root, { recursive: true, force: true });
}
});

it("keeps module.exports identity across cells after reassignment", async () => {
await withKernel(async (kernel) => {
// Given a cell that replaces module.exports
await runCell(kernel, "module.exports = { persisted: 7 }");

// When a later cell reads exports and module.exports
const run = await runCell(kernel, "return { same: exports === module.exports, value: exports.persisted }");

// Then exports tracks the replaced module.exports object
expect(run.result).toMatchObject({
ok: true,
valueRepr: JSON.stringify({ same: true, value: 7 }),
});
});
});

it("resolves a relative session cwd before installing CommonJS", async () => {
const root = await mkdtemp(join(tmpdir(), "senpi-codemode-js-cjs-rel-"));
await writeFile(join(root, "module.cjs"), "module.exports = { value: 41 };\n");
const kernel = new JavaScriptKernel({
sessionId: "relative-cwd-cjs",
cwd: relative(process.cwd(), root),
parallelPoolWidth: 2,
});
try {
// Given a kernel whose cwd is not an absolute path
// When a cell requires a sibling CommonJS module and inspects __dirname
const run = await runCell(
kernel,
'const mod = require("./module.cjs"); return { value: mod.value + 1, dirname: __dirname }',
);

// Then createRequire still resolves from the absolute session cwd
expect(run.result.ok).toBe(true);
if (!run.result.ok) return;
expect(JSON.parse(run.result.valueRepr ?? "null")).toEqual({
value: 42,
dirname: resolve(root),
});
} finally {
await kernel.close();
await rm(root, { recursive: true, force: true });
}
});

it("keeps require after a cell redeclares the CommonJS binding", async () => {
await withKernel(async (kernel) => {
// Given a cell that tries to shadow require with a top-level declaration
const first = await runCell(kernel, "const require = 1; return require");

// When the declaration collides with the CommonJS wrapper parameter
expect(first.result.ok).toBe(false);

// Then later cells still have the original require function
const second = await runCell(kernel, "return require('node:path').basename('/a/b/c.js')");
expect(second.result).toMatchObject({ ok: true, valueRepr: '"c.js"' });
});
});

it("imports a relative module from the session cwd", async () => {
const root = await mkdtemp(join(tmpdir(), "senpi-codemode-js-import-"));
await writeFile(join(root, "module.mjs"), "export const value = 41;\n");
Expand Down