Skip to content

Add pnpm materialization profiles#878

Draft
schickling-assistant wants to merge 9 commits into
mainfrom
schickling-assistant/2026-06-29-schickling-assistant-2026-06-29-pnpm-materialization-profiles
Draft

Add pnpm materialization profiles#878
schickling-assistant wants to merge 9 commits into
mainfrom
schickling-assistant/2026-06-29-schickling-assistant-2026-06-29-pnpm-materialization-profiles

Conversation

@schickling-assistant

@schickling-assistant schickling-assistant commented Jun 29, 2026

Copy link
Copy Markdown
Collaborator

Problem

The pnpm install helper had implicit dependency-materialization behavior. That made it hard for downstream repos to prove whether installs use job-local state, split shared files, or a future same-device hardlink profile.

Goal

Expose an explicit materialization profile contract and runtime evidence without switching the default Linux behavior to hardlink imports yet.

Decisions

  • Add materializationProfile ? "auto" with explicit ciJobLocal, splitFilesCas, darwinSplitCas, linuxSharedHardlink, and isolated profiles.
  • Keep Linux auto behavior-compatible via splitFilesCas; linuxSharedHardlink remains opt-in and fail-fast.
  • Keep CI auto job-local, while preserving explicitly requested global profiles in CI.
  • Add local-only overrides: localMaterializationProfile, localStoreDir, and localSharedFilesDir apply only outside CI.
  • Add localOverridesRequireLinux so downstream repos can declare a Linux hardlink CAS boundary without leaking /var/lib Linux paths or hardlink policy into macOS.
  • Validate same-device placement before explicit hardlink installs using Node filesystem stats.
  • Include current dependency-materialization profile evidence in the install-state hash so contract/profile drift cannot silently hit the warm cache.
  • Apply selected profile import policy to pnpm:update and pnpm:dedupe, not only pnpm:install.
  • Record the registered profile trait in repair roots and refuse ambiguous legacy registry rows instead of replaying the caller's current profile for every sibling root.
  • Convert notion-cli dependency FOD to Evergreen-managed per-system hash shape so full verification no longer blocks on an unrepairable literal slot.

Verification

  • CI=1 bash nix/devenv-modules/tasks/shared/tests/pnpm.test.sh
    • Passed 38 helper tests.
  • CI=1 bash nix/devenv-modules/tasks/shared/tests/pnpm-task-smoke.test.sh
    • Passed 30 smoke tests, including explicit hardlink evidence, hardlink import flags for update/dedupe, local-only CI fallback, and localOverridesRequireLinux with a simulated Darwin runtime.
  • nixfmt --check nix/devenv-modules/tasks/shared/pnpm.nix
  • git diff --check -- nix/devenv-modules/tasks/shared/pnpm.nix nix/devenv-modules/tasks/shared/tests/pnpm-task-smoke.test.sh
  • Earlier FOD proof on this branch:
    • nix build .#notion-cli --no-link --print-build-logs on dev3 / x86_64-linux passed.
    • ssh dev4 ... nix build .#notion-cli --no-link --print-build-logs at commit a42a3e495 passed on aarch64-linux.
  • devenv tasks run check:all --no-tui
    • FOD/genie/nix flake surfaces passed after the FOD repair.
    • Still failed in unrelated long integration surfaces: notion-datasource-sync e2e timeouts, one restate-effect endpoint readiness failure, and Vitest fork startup/teardown timeouts.

Complexity

Adds profile-selection helpers because downstream storage observability needs a stable contract and runtime evidence. The complexity is bounded to the pnpm task helpers and keeps hardlink materialization opt-in.

Concerns

  • Linux auto still does not switch to hardlink; that remains gated on production CAS path provisioning, real-workload reuse/link-count evidence, install timing, and native rebuild safety.
  • Darwin real-host proof is still pending. The smoke suite simulates non-Linux runtime gating, but we still need a clean macOS proof path.
  • The downstream dotfiles live graph is currently blocked by /nix disk exhaustion on dev3; dotfiles #1219 records that as the current bottleneck.
  • Evergreen can now reconcile the notion-cli hash slot, but its direct post-write evaluator still cannot import this package because build.nix requires src; real flake builds are the proof surface for this package.

Follow-ups

  • Enable linuxSharedHardlink as the Linux default only after measured production proof.
  • Add targeted real native-package rebuild coverage before hardlink default enablement.
  • Add/repair a clean Darwin proof path that uses bulk temp space without pressuring the internal /nix volume.
  • Continue downstream dotfiles Phase 9 rollout and reuse/amortization tracking.

References

  • Related: schickling/dotfiles#1219
Posted on behalf of @schickling
field value
agent_name 🐧 co1-polar
agent_session_id 61111ff4-aff6-4a16-a4bf-ef47cef3e76a
agent_tool Codex CLI
agent_tool_version 0.141.0
agent_runtime Codex CLI 0.141.0
agent_model unknown
runtime_profile /nix/store/ynb6b5csz0wgbwqw1c43flnp450v5bjv-coding-agent-runtime-profile/share/coding-agents/profile.json
skills_manifest /nix/store/pqhvj5xhdaw62mi7dm62a7i7pwk9wpjj-agent-skills-corpus/share/agent-skills/manifest.json
worktree dotfiles/schickling/2026-06-28-pnpm-overhead
machine dev3
tooling_profile dotfiles@f43f9f1

@schickling-assistant schickling-assistant added origin:agent Filed or primarily produced by an AI agent · Set: AI agent or manual type:feature New user-visible or system capability · Set: manual area:nix Nix flakes, derivations, FOD hashes, builders · Set: manual area:devenv devenv tasks, inputs, and environment configuration · Set: manual labels Jun 29, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:devenv devenv tasks, inputs, and environment configuration · Set: manual area:nix Nix flakes, derivations, FOD hashes, builders · Set: manual origin:agent Filed or primarily produced by an AI agent · Set: AI agent or manual type:feature New user-visible or system capability · Set: manual

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant