Fix critical regressions and performance issues

AGENTS.md

@@ -17,6 +17,7 @@
 - `crates/glua_doc_cli`, `crates/schema_to_glua`, and `tools/schema_json_gen`: documentation and schema tooling.
 - `tools/benchmark`: large-workspace benchmark. It requires `BENCH_CODEBASE` and `BENCH_ANNOTATIONS`.
 - `tools/determinism`: diagnostic determinism harness. It requires `DET_CODEBASE` and `DET_ANNOTATIONS`, and answers whether re-analysing a workspace yields the same diagnostics as building it cold. See the module docs for the stage list.
+- `tools/lsp_latency.js`: interactive latency harness. It requires `LSP_CODEBASE` and `LSP_ANNOTATIONS`, and drives a real `glua_ls` binary over stdio using the capabilities and cancellation behaviour VS Code actually uses. Reports completion and diagnostic latency settled versus mid-edit, and asserts that a cancelled diagnostic pull never returns an empty full report (which clears a file's diagnostics in VS Code). Use it before and after any change to reindexing or to the freshness gates — those costs are invisible to unit tests.
 - `docs/mintlify`: user documentation. Follow its nested `AGENTS.md` for changes under that tree.
 
 ## Analysis Architecture
@@ -51,8 +52,9 @@
 - Call-role and annotation-driven tests should load the relevant builtins; otherwise they may pass while bypassing the real metadata path.
 - Typical test commands are `cargo test -p glua_code_analysis <test_name>`, `cargo test -p glua_code_analysis`, and `cargo test`.
 - Use `glua_check` JSON output for before/after corpus diagnostic comparisons. The benchmark measures performance; it is not a diagnostics oracle.
-- Changes to indexes, cached inference, or the unresolve/resolution passes must keep incremental re-analysis equal to a cold build. Verify with `cargo run --release -p determinism` on a real workspace; `repeat`, `order`, `fresh`, `reindex`, `allreindex` and `mainexpand` are expected to report IDENTICAL. The two edit gates are `noopedit` (formerly `edit`) and `realedit`. `noopedit` gates the semantic no-op skip and nothing more: its edit pair is semantically unchanged, so the update path skips the re-index outright and the stage verifies that skipping preserves state — include at least one wide-expansion target (e.g. CityRP `gamemode/core/sh_util.lua`, 1300 files) so the skip is exercised where it matters most. `realedit` is the gate for incremental re-analysis itself, since its edit changes what the file means; set `DET_EDIT_FIND` and `DET_EDIT_REPLACE` or it skips and nothing gates re-analysis. It passes today — CityRP measures IDENTICAL at 11,655 entries with `members dropped=0 gained=0` — so treat any divergence as a regression you introduced, not as a known gap. Note it is far more sensitive than the other gates: changes to the unresolve waves, the infer-cache lifetime, or member ownership can break `realedit` while all seven others still report IDENTICAL. `mainreindex`, `exact`, `split:N` and `editmid` are bisect stages, not gates: the first three run `reindex_files_without_expansion`, which skips production's convergence passes, and `editmid` forces a real offset-shifting re-analysis of the expansion (the batch-composition confluence gap), so they are expected to diverge and only matter for localising a failure the gates already caught. Re-run it before and after, because a change can make a stage identical by *degrading* the cold build rather than by fixing the re-index.
+- Changes to indexes, cached inference, or the unresolve/resolution passes must keep incremental re-analysis equal to a cold build. Verify with `cargo run --release -p determinism` on a real workspace; `repeat`, `order`, `fresh`, `reindex`, `allreindex` and `mainexpand` are expected to report IDENTICAL. A third gate, `indexrepeat`, re-indexes each target with its text untouched and requires the **index** to come back identical; the diagnostic gates cannot see index drift, because re-analysis can attach different members or settle a decl's type differently and still produce the same diagnostics. It does **not** pass today (CityRP: 80 type caches, 2 signatures, 11 class members change on a no-op re-index) and that drift is why incremental work cannot be skipped — every "did this actually change?" test answers yes — so treat any *growth* in those counts as yours. Use `DET_TARGETS=gamemode/core/sh_data.lua` as the working repro: it expands to **4 files** and reproduces the same defect at 4/0/3, which is far cheaper to iterate on than the 1306-file one. The remaining drift sits in three readers, each proven by trace: the sibling-widening cache (`lua/stats.rs`, where `visible_member_count_for_owner_key` is 2 cold and 1 warm, so `lookup_widening_cache` returns `FirstSighting` and `get_widened_member_assignment_type` is never called, so nothing arms the settled retry); first-writer-wins on a decl slot (`common/mod.rs:206-215` deliberately keeps an `any`/`unknown` decl cache, pinned by three tests, so the slot is claimed by whichever writer arrives first and an unrelated later assignment can seed it); and attach-candidate lifetime (`analyzer/mod.rs:317-377`, whose retry list lives in a context that dies when `analyze()` returns, making member *existence* batch-dependent — it owns all 11 class-member drifts). Two dead ends already paid for: arming the settled retry from the `FirstSighting` arm fixes two entries and takes the 1306 gate from 80 to **144**, because it widens members cold previously left alone; and `rederive_contributed_member_assignments` cannot fix the widening class at all, because its `take_while` merges each writer only against *earlier* writers so a first writer is never re-derived. The defect is that analysis output depends on how the workspace was *batched*, not on the source alone: `remove_index(batch)` runs before `update_index(batch)`, so a file sees out-of-batch neighbours complete but in-batch neighbours empty until the walk reaches them. A whole-workspace batch hides everything and so reproduces the cold build exactly (`allreindex` and `mainexpand` are both byte-identical to cold); a four-file batch hides almost nothing and lands somewhere else. It is not edit-specific — `split:4` builds the same workspace cold in four batches and produces 299 different diagnostics against `split:1`. Do not "fix" it by re-indexing everything on an edit: that forces the whole-workspace batch, costs more than a cold build, and freezes the least-informed answer. `editrevert` is the drift gate for the *other* edit path. It applies a real edit through `update_file_by_uri` and then takes it back out; the source ends where it started, so the index and the diagnostics have to as well, and it needs no ground-truth build because the pre-edit index is the truth. It covers what the others cannot — `indexrepeat` re-indexes with the text untouched and so never exercises an edit's invalidation, and `noopedit`'s pair is semantically neutral, so the update path skips the re-index outright. It does **not** pass today: on CityRP an edit-and-revert of `gamemode/core/sh_util.lua` leaves 79 type caches, 3 signatures and 11 class member lists moved, plus 2 `need-check-nil` diagnostics. Treat growth in those counts as yours. Both index gates build their own analysis rather than sharing the caller's, because each re-indexes in place and leaves a converged index behind — sharing one let whichever ran second measure against the other's converged state and report a clean 0, which hid the drift rather than removing it. The two edit gates are `noopedit` (formerly `edit`) and `realedit`. `noopedit` gates the semantic no-op skip and nothing more: its edit pair is semantically unchanged, so the update path skips the re-index outright and the stage verifies that skipping preserves state — include at least one wide-expansion target (e.g. CityRP `gamemode/core/sh_util.lua`, 1300 files) so the skip is exercised where it matters most. `realedit` is the gate for incremental re-analysis itself, since its edit changes what the file means; set `DET_EDIT_FIND` and `DET_EDIT_REPLACE` or it skips and nothing gates re-analysis. It does **not** pass today, and the divergence is a known gap rather than something you introduced — but it is a small, fixed one, so measure it before and after your change and treat any *growth* as yours. On CityRP, editing `gamemode/core/sh_util.lua` (`function cityrp.util.Bind(self, callback)` gaining a parameter) gives `cold_edited -> warm: removed=0 added=2`, both `need-check-nil` in `plugins/cwweapons` (`cw_base/shared.lua:999`, `cw_m249_official/shared.lua:304`). The cause is visible in the index diff: a signature is identified by its position, an edit moves the positions of every signature after it, and `CallSiteParamIndex` contributions that *target* a moved signature are only dropped when the contributing file is itself re-indexed. A contributor outside the reindex expansion keeps pointing at the old position, so `rebuild_derived_state` splits one parameter's inferred type across the stale signature id and the current one (`15167` keeps one union arm, `15174` the other). Losing an arm widens callers' inferences to include `Unknown`, which is what makes `need-check-nil` fire downstream. Fixing it means invalidating contributions by *target* file and pulling those contributors into the reindex expansion, which changes the expansion set, so measure the benchmark as well when you do. Note it is far more sensitive than the other gates: changes to the unresolve waves, the infer-cache lifetime, or member ownership can break `realedit` while all seven others still report IDENTICAL. `mainreindex`, `exact`, `split:N` and `editmid` are bisect stages, not gates: the first three run `reindex_files_without_expansion`, which skips production's convergence passes, and `editmid` forces a real offset-shifting re-analysis of the expansion (the batch-composition confluence gap), so they are expected to diverge and only matter for localising a failure the gates already caught. Re-run it before and after, because a change can make a stage identical by *degrading* the cold build rather than by fixing the re-index.
 - Performance changes require profiling or a targeted before/after benchmark. Use `GLUALS_PROFILE=1` for phase timings and `cargo run --release -p benchmark` for the large-workspace harness.
+- For a sampling profile use `samply` (ETW-based on Windows, so it prompts for admin elevation on every run; the user has to approve it). Three things have to be right or you get a useless profile: build with `CARGO_PROFILE_RELEASE_DEBUG=1 cargo build --release -p benchmark` so the PDB exists, run the binary from `target/release` (samply resolves the PDB by the relative path recorded in the exe, so it only finds it from that directory), and do **not** pass `--main-thread-only` — the tools run analysis on a spawned big-stack thread, so the main thread only shows a join. A working invocation is `cd target/release && BENCH_CODEBASE=<path> samply record --save-only --unstable-presymbolicate -o <out>.json.gz ./benchmark.exe`. That writes `<out>.json.gz` plus a `<out>.json.syms.json` sidecar; the profile itself holds only addresses, so symbol names come from joining the two by `libs[].debugName` and the frame address against each module's `symbol_table` rva ranges.
 - Performance is extremely important; the language server must be quick and responsive on large workspaces without loss of functionality. You are to always optimise at the root cause of performance issues. Things such as budgets, string based prefilters / guards and other similar "hacks" are unacceptable since they will regress functionality in large or complex codebases.
 
 ## Commands

Cargo.toml

@@ -84,7 +84,7 @@ emmylua_codestyle = { path = "vendor/emmylua_codestyle" }
 
 [profile.profiling]
 inherits = "release"
-debug = 1
+debug = 2
 strip = "none"
 
 # Lint configuration for the entire workspace

crates/glua_check/src/bin/glua_check.rs

@@ -6,8 +6,24 @@ use std::error::Error;
 #[global_allocator]
 static GLOBAL: MiMalloc = MiMalloc;
 
-#[tokio::main]
-async fn main() -> Result<(), Box<dyn Error + Sync + Send>> {
+/// Analysis recurses over deeply nested syntax, and a Windows process main
+/// thread has a far smaller stack than a spawned one.
+fn main() -> Result<(), Box<dyn Error + Sync + Send>> {
+    std::thread::Builder::new()
+        .stack_size(256 * 1024 * 1024)
+        .spawn(|| {
+            tokio::runtime::Builder::new_multi_thread()
+                .enable_all()
+                .build()
+                .expect("tokio runtime should build")
+                .block_on(run())
+        })
+        .expect("glua_check worker thread should spawn")
+        .join()
+        .expect("glua_check worker thread should not panic")
+}
+
+async fn run() -> Result<(), Box<dyn Error + Sync + Send>> {
     let cmd_args = CmdArgs::parse();
     run_check(cmd_args).await
 }

crates/glua_code_analysis/Cargo.toml

@@ -18,6 +18,11 @@ include = [
 [lib]
 doctest = false
 
+[features]
+# Cross-checks the type-cache reverse index against a full scan on every query.
+# Off by default: it makes incremental expansion quadratic again.
+verify_type_cache_refs = []
+
 [dev-dependencies]
 googletest.workspace = true