feat(profiling): hook the enrich phase into the profiler - #477
Open
Disentinel wants to merge 2 commits into
Open
feat(profiling): hook the enrich phase into the profiler#477Disentinel wants to merge 2 commits into
Disentinel wants to merge 2 commits into
Conversation
The enrich phase was a profiler blind spot. The Rust enrichment plugins (type-inference ~64s, shape-tracker ~34s) ran inside the orchestrator but were silently folded into resolve_ms — ~98s mislabeled as resolution. The TS enrichers (mcp-tool, contract, behavior, package-api, …) run in the CLI after the orchestrator exits and never touched the JSONL at all. Now: - Rust plugins are de-conflated from resolve_ms into a dedicated enrich_ms, emit one `enrich_plugin_complete` event each, and land as `phase=enrich` profile:stage nodes in the subgraph (in-process, before the commit). - TS enrichers append `enrich_step_complete` events to the same analysis-profile.jsonl via a small profileAppend util, timed across success AND RPC-timeout skips (finally), so the tail is visible in the JSONL profiler and the route view. - build_profile_stages maps both event types to phase=enrich; the enrich_step arm is ready for when the subgraph is rebuilt from the full JSONL after enrich. Verified by a real self-analyze: phase_summary now reports resolve_ms=102024 + enrich_ms=97783 (was one conflated 200s resolve); enrich_plugin_complete events for type-inference/shape-tracker; the subgraph gains the 2 enrich stages. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
PRECEDES encodes the ACTUAL (sequential) execution order; this adds REQUIRES — the subset that is a true data dependency. A PRECEDES edge with no REQUIRES backing it means a stage ran sequentially but did not need to: that is parallelism headroom. - REQUIRES edges between profile:stage nodes. Derive model (conservative, so headroom is a floor): language verticals chain internally; shared sinks (depends/method_calls/shape_verifier) require the last pack of every vertical; shape_verifier also requires method_calls; resolvers and TS enrich steps have no REQUIRES (fully parallel); shape-tracker requires type-inference. - Per-phase critical_path_ms + parallelism_headroom_ms METRICs; a run-level parallelism_headroom_ms on profile:run. - "What could run in parallel" = total_work - critical_path is now a graph query. On the grafema monorepo: resolve ~48s headroom (4 independent language resolvers run sequentially), derive ~57s (independent verticals). Pure-function change; 4 unit tests (incl. headroom) green. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Hooks the enrich phase into the analysis profiler — the second profiler blind spot after the derive packs (closed in the profile-subgraph PR this is stacked on).
Enrichment ran in two invisible places:
type-inference~64s,shape-tracker~34s) — run inside the orchestrator but were silently folded intoresolve_ms(~98s mislabeled as resolution).mcp-tool,contract,speced-contract,behavior,package-api,library-callback) — run in the CLI after the orchestrator exits, so they never touched the JSONL.Changes
grafema-orchestrator): de-conflate the user-plugins fromresolve_msinto a dedicatedenrich_ms; emit oneenrich_plugin_completeprofiler event per plugin; addenrichtophase_summary+ the phase METRIC nodes; mapenrich_plugin_complete→phase=enrichprofile:stageinbuild_profile_stages(lands in the subgraph in-process, before the commit).@grafema/cli): newprofileAppend.tsutil that appends toanalysis-profile.jsonlin the orchestrator's schema; anenrichStepwrapper times each enricher — including ones that skip on an RPC timeout (viafinally) — and emitsenrich_step_complete.build_profile_stagesalso mapsenrich_step_complete(ready for when the subgraph is rebuilt from the full JSONL after enrich — the orchestrator currently commits the subgraph from its in-memory stream, which predates the TS phase)._ai/profile-subgraph.md+CHANGELOG.md.Verification (real self-analyze on the grafema monorepo)
resolve_msdropped from a single conflated ~200s to a true 102s resolve + 98s enrich.Note
This run surfaced a pre-existing slow/hanging
mcp-tool-defsenricher (a queryNodesStream that no longer fails fast) — exactly the kind of bottleneck this instrumentation is meant to make visible. Orthogonal to this PR.Base
Stacked on
feat/pipeline-profile-graph(the profile-subgraph PR). Merge that first.🤖 Generated with Claude Code