diff --git a/AGENTS.md b/AGENTS.md index 529043de2..5c1a59eb3 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -17,6 +17,7 @@ - `crates/glua_doc_cli`, `crates/schema_to_glua`, and `tools/schema_json_gen`: documentation and schema tooling. - `tools/benchmark`: large-workspace benchmark. It requires `BENCH_CODEBASE` and `BENCH_ANNOTATIONS`. - `tools/determinism`: diagnostic determinism harness. It requires `DET_CODEBASE` and `DET_ANNOTATIONS`, and answers whether re-analysing a workspace yields the same diagnostics as building it cold. See the module docs for the stage list. +- `tools/lsp_latency.js`: interactive latency harness. It requires `LSP_CODEBASE` and `LSP_ANNOTATIONS`, and drives a real `glua_ls` binary over stdio using the capabilities and cancellation behaviour VS Code actually uses. Reports completion and diagnostic latency settled versus mid-edit, and asserts that a cancelled diagnostic pull never returns an empty full report (which clears a file's diagnostics in VS Code). Use it before and after any change to reindexing or to the freshness gates — those costs are invisible to unit tests. - `docs/mintlify`: user documentation. Follow its nested `AGENTS.md` for changes under that tree. ## Analysis Architecture @@ -51,8 +52,9 @@ - Call-role and annotation-driven tests should load the relevant builtins; otherwise they may pass while bypassing the real metadata path. - Typical test commands are `cargo test -p glua_code_analysis `, `cargo test -p glua_code_analysis`, and `cargo test`. - Use `glua_check` JSON output for before/after corpus diagnostic comparisons. The benchmark measures performance; it is not a diagnostics oracle. -- Changes to indexes, cached inference, or the unresolve/resolution passes must keep incremental re-analysis equal to a cold build. Verify with `cargo run --release -p determinism` on a real workspace; `repeat`, `order`, `fresh`, `reindex`, `allreindex` and `mainexpand` are expected to report IDENTICAL. The two edit gates are `noopedit` (formerly `edit`) and `realedit`. `noopedit` gates the semantic no-op skip and nothing more: its edit pair is semantically unchanged, so the update path skips the re-index outright and the stage verifies that skipping preserves state — include at least one wide-expansion target (e.g. CityRP `gamemode/core/sh_util.lua`, 1300 files) so the skip is exercised where it matters most. `realedit` is the gate for incremental re-analysis itself, since its edit changes what the file means; set `DET_EDIT_FIND` and `DET_EDIT_REPLACE` or it skips and nothing gates re-analysis. It passes today — CityRP measures IDENTICAL at 11,655 entries with `members dropped=0 gained=0` — so treat any divergence as a regression you introduced, not as a known gap. Note it is far more sensitive than the other gates: changes to the unresolve waves, the infer-cache lifetime, or member ownership can break `realedit` while all seven others still report IDENTICAL. `mainreindex`, `exact`, `split:N` and `editmid` are bisect stages, not gates: the first three run `reindex_files_without_expansion`, which skips production's convergence passes, and `editmid` forces a real offset-shifting re-analysis of the expansion (the batch-composition confluence gap), so they are expected to diverge and only matter for localising a failure the gates already caught. Re-run it before and after, because a change can make a stage identical by *degrading* the cold build rather than by fixing the re-index. +- Changes to indexes, cached inference, or the unresolve/resolution passes must keep incremental re-analysis equal to a cold build. Verify with `cargo run --release -p determinism` on a real workspace; `repeat`, `order`, `fresh`, `reindex`, `allreindex` and `mainexpand` are expected to report IDENTICAL. A third gate, `indexrepeat`, re-indexes each target with its text untouched and requires the **index** to come back identical; the diagnostic gates cannot see index drift, because re-analysis can attach different members or settle a decl's type differently and still produce the same diagnostics. It does **not** pass today (CityRP: 80 type caches, 2 signatures, 11 class members change on a no-op re-index) and that drift is why incremental work cannot be skipped — every "did this actually change?" test answers yes — so treat any *growth* in those counts as yours. Use `DET_TARGETS=gamemode/core/sh_data.lua` as the working repro: it expands to **4 files** and reproduces the same defect at 4/0/3, which is far cheaper to iterate on than the 1306-file one. The remaining drift sits in three readers, each proven by trace: the sibling-widening cache (`lua/stats.rs`, where `visible_member_count_for_owner_key` is 2 cold and 1 warm, so `lookup_widening_cache` returns `FirstSighting` and `get_widened_member_assignment_type` is never called, so nothing arms the settled retry); first-writer-wins on a decl slot (`common/mod.rs:206-215` deliberately keeps an `any`/`unknown` decl cache, pinned by three tests, so the slot is claimed by whichever writer arrives first and an unrelated later assignment can seed it); and attach-candidate lifetime (`analyzer/mod.rs:317-377`, whose retry list lives in a context that dies when `analyze()` returns, making member *existence* batch-dependent — it owns all 11 class-member drifts). Two dead ends already paid for: arming the settled retry from the `FirstSighting` arm fixes two entries and takes the 1306 gate from 80 to **144**, because it widens members cold previously left alone; and `rederive_contributed_member_assignments` cannot fix the widening class at all, because its `take_while` merges each writer only against *earlier* writers so a first writer is never re-derived. The defect is that analysis output depends on how the workspace was *batched*, not on the source alone: `remove_index(batch)` runs before `update_index(batch)`, so a file sees out-of-batch neighbours complete but in-batch neighbours empty until the walk reaches them. A whole-workspace batch hides everything and so reproduces the cold build exactly (`allreindex` and `mainexpand` are both byte-identical to cold); a four-file batch hides almost nothing and lands somewhere else. It is not edit-specific — `split:4` builds the same workspace cold in four batches and produces 299 different diagnostics against `split:1`. Do not "fix" it by re-indexing everything on an edit: that forces the whole-workspace batch, costs more than a cold build, and freezes the least-informed answer. `editrevert` is the drift gate for the *other* edit path. It applies a real edit through `update_file_by_uri` and then takes it back out; the source ends where it started, so the index and the diagnostics have to as well, and it needs no ground-truth build because the pre-edit index is the truth. It covers what the others cannot — `indexrepeat` re-indexes with the text untouched and so never exercises an edit's invalidation, and `noopedit`'s pair is semantically neutral, so the update path skips the re-index outright. It does **not** pass today: on CityRP an edit-and-revert of `gamemode/core/sh_util.lua` leaves 79 type caches, 3 signatures and 11 class member lists moved, plus 2 `need-check-nil` diagnostics. Treat growth in those counts as yours. Both index gates build their own analysis rather than sharing the caller's, because each re-indexes in place and leaves a converged index behind — sharing one let whichever ran second measure against the other's converged state and report a clean 0, which hid the drift rather than removing it. The two edit gates are `noopedit` (formerly `edit`) and `realedit`. `noopedit` gates the semantic no-op skip and nothing more: its edit pair is semantically unchanged, so the update path skips the re-index outright and the stage verifies that skipping preserves state — include at least one wide-expansion target (e.g. CityRP `gamemode/core/sh_util.lua`, 1300 files) so the skip is exercised where it matters most. `realedit` is the gate for incremental re-analysis itself, since its edit changes what the file means; set `DET_EDIT_FIND` and `DET_EDIT_REPLACE` or it skips and nothing gates re-analysis. It does **not** pass today, and the divergence is a known gap rather than something you introduced — but it is a small, fixed one, so measure it before and after your change and treat any *growth* as yours. On CityRP, editing `gamemode/core/sh_util.lua` (`function cityrp.util.Bind(self, callback)` gaining a parameter) gives `cold_edited -> warm: removed=0 added=2`, both `need-check-nil` in `plugins/cwweapons` (`cw_base/shared.lua:999`, `cw_m249_official/shared.lua:304`). The cause is visible in the index diff: a signature is identified by its position, an edit moves the positions of every signature after it, and `CallSiteParamIndex` contributions that *target* a moved signature are only dropped when the contributing file is itself re-indexed. A contributor outside the reindex expansion keeps pointing at the old position, so `rebuild_derived_state` splits one parameter's inferred type across the stale signature id and the current one (`15167` keeps one union arm, `15174` the other). Losing an arm widens callers' inferences to include `Unknown`, which is what makes `need-check-nil` fire downstream. Fixing it means invalidating contributions by *target* file and pulling those contributors into the reindex expansion, which changes the expansion set, so measure the benchmark as well when you do. Note it is far more sensitive than the other gates: changes to the unresolve waves, the infer-cache lifetime, or member ownership can break `realedit` while all seven others still report IDENTICAL. `mainreindex`, `exact`, `split:N` and `editmid` are bisect stages, not gates: the first three run `reindex_files_without_expansion`, which skips production's convergence passes, and `editmid` forces a real offset-shifting re-analysis of the expansion (the batch-composition confluence gap), so they are expected to diverge and only matter for localising a failure the gates already caught. Re-run it before and after, because a change can make a stage identical by *degrading* the cold build rather than by fixing the re-index. - Performance changes require profiling or a targeted before/after benchmark. Use `GLUALS_PROFILE=1` for phase timings and `cargo run --release -p benchmark` for the large-workspace harness. +- For a sampling profile use `samply` (ETW-based on Windows, so it prompts for admin elevation on every run; the user has to approve it). Three things have to be right or you get a useless profile: build with `CARGO_PROFILE_RELEASE_DEBUG=1 cargo build --release -p benchmark` so the PDB exists, run the binary from `target/release` (samply resolves the PDB by the relative path recorded in the exe, so it only finds it from that directory), and do **not** pass `--main-thread-only` — the tools run analysis on a spawned big-stack thread, so the main thread only shows a join. A working invocation is `cd target/release && BENCH_CODEBASE= samply record --save-only --unstable-presymbolicate -o .json.gz ./benchmark.exe`. That writes `.json.gz` plus a `.json.syms.json` sidecar; the profile itself holds only addresses, so symbol names come from joining the two by `libs[].debugName` and the frame address against each module's `symbol_table` rva ranges. - Performance is extremely important; the language server must be quick and responsive on large workspaces without loss of functionality. You are to always optimise at the root cause of performance issues. Things such as budgets, string based prefilters / guards and other similar "hacks" are unacceptable since they will regress functionality in large or complex codebases. ## Commands diff --git a/Cargo.lock b/Cargo.lock index f58322750..8c6884c4d 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -543,6 +543,7 @@ dependencies = [ name = "determinism" version = "0.1.0" dependencies = [ + "backtrace", "emmy_lsp_types", "glua_code_analysis", "glua_parser", @@ -990,7 +991,9 @@ name = "glua_parser" version = "0.1.5" dependencies = [ "rowan", + "rustc-hash 2.1.1", "serde", + "smol_str", ] [[package]] diff --git a/Cargo.toml b/Cargo.toml index ab7c840a5..6a1fd6613 100644 --- a/Cargo.toml +++ b/Cargo.toml @@ -84,7 +84,7 @@ emmylua_codestyle = { path = "vendor/emmylua_codestyle" } [profile.profiling] inherits = "release" -debug = 1 +debug = 2 strip = "none" # Lint configuration for the entire workspace diff --git a/crates/glua_check/src/bin/glua_check.rs b/crates/glua_check/src/bin/glua_check.rs index 016fc7e51..51dd1e3c8 100644 --- a/crates/glua_check/src/bin/glua_check.rs +++ b/crates/glua_check/src/bin/glua_check.rs @@ -6,8 +6,24 @@ use std::error::Error; #[global_allocator] static GLOBAL: MiMalloc = MiMalloc; -#[tokio::main] -async fn main() -> Result<(), Box> { +/// Analysis recurses over deeply nested syntax, and a Windows process main +/// thread has a far smaller stack than a spawned one. +fn main() -> Result<(), Box> { + std::thread::Builder::new() + .stack_size(256 * 1024 * 1024) + .spawn(|| { + tokio::runtime::Builder::new_multi_thread() + .enable_all() + .build() + .expect("tokio runtime should build") + .block_on(run()) + }) + .expect("glua_check worker thread should spawn") + .join() + .expect("glua_check worker thread should not panic") +} + +async fn run() -> Result<(), Box> { let cmd_args = CmdArgs::parse(); run_check(cmd_args).await } diff --git a/crates/glua_code_analysis/Cargo.toml b/crates/glua_code_analysis/Cargo.toml index fc281c74b..ad519f5d6 100644 --- a/crates/glua_code_analysis/Cargo.toml +++ b/crates/glua_code_analysis/Cargo.toml @@ -18,6 +18,11 @@ include = [ [lib] doctest = false +[features] +# Cross-checks the type-cache reverse index against a full scan on every query. +# Off by default: it makes incremental expansion quadratic again. +verify_type_cache_refs = [] + [dev-dependencies] googletest.workspace = true diff --git a/crates/glua_code_analysis/src/compilation/analyzer/common/migrate_global_member.rs b/crates/glua_code_analysis/src/compilation/analyzer/common/migrate_global_member.rs index 279334dc2..e93b822b2 100644 --- a/crates/glua_code_analysis/src/compilation/analyzer/common/migrate_global_member.rs +++ b/crates/glua_code_analysis/src/compilation/analyzer/common/migrate_global_member.rs @@ -1,6 +1,7 @@ use std::collections::HashSet; use crate::{DbIndex, GlobalId, InFiled, LuaDeclId, LuaMemberId, LuaMemberOwner, LuaTypeOwner}; +use glua_parser::{LuaAstNode, LuaExpr, LuaIndexExpr, PathTrait}; use super::get_owner_id; use crate::compilation::analyzer::lua::is_guarded_table_assignment_member; @@ -216,7 +217,14 @@ pub fn reconcile_parked_global_path_members(db: &mut DbIndex) { if hidden.contains(&member_id) && *alias_file_id != member_id.file_id { continue; } - if Some(alias_owner) != target_owner.as_ref() { + // Re-indexing one file leaves every other file's aliases in + // place, so on an incremental batch nearly all of these are + // already recorded. Skipping those is not an approximation: + // the alias write is a no-op exactly when + // `alias_to_owner_is_recorded` holds. + if Some(alias_owner) != target_owner.as_ref() + && !member_index.alias_to_owner_is_recorded(alias_owner, member_id) + { member_index.add_member_alias_to_owner(alias_owner.clone(), member_id); } } @@ -224,6 +232,171 @@ pub fn reconcile_parked_global_path_members(db: &mut DbIndex) { } } +/// Whether this member is a write through *this* global's path. +/// +/// A candidate table can hold members that arrived through other prefixes (a +/// local alias, a sibling global whose type resolved to the same literal); +/// those must not take part in this global's ownership repair. The path is +/// normally recorded on the member when declaration analysis parks it; a +/// member the lua pass attached directly carries none, so its own syntax +/// decides — the write site is stable under batching either way. +fn member_targets_global_path(db: &DbIndex, member_id: LuaMemberId, global_id: &GlobalId) -> bool { + if let Some(member) = db.get_member_index().get_member(&member_id) + && let Some(recorded) = member.get_global_id() + { + // The record is the write's full access path (`cityrp.LoadedOnce`); + // this repair belongs to the root global's candidates. + let recorded_root = recorded.get_name().split('.').next().unwrap_or_default(); + return recorded_root == global_id.get_name(); + } + + let Some(tree) = db.get_vfs().get_syntax_tree(&member_id.file_id) else { + return false; + }; + let Some(node) = member_id + .get_syntax_id() + .to_node_from_root(&tree.get_red_root()) + else { + return false; + }; + let Some(index_expr) = LuaIndexExpr::cast(node) else { + return false; + }; + let Some(access_path) = index_expr.get_access_path() else { + return false; + }; + let root = access_path.split('.').next().unwrap_or_default(); + if root != global_id.get_name() { + return false; + } + // The root segment must actually read the global: a local (or parameter) + // of the same name writes somewhere else entirely. A read bound to one of + // the root's own global declarations still counts as the global itself. + match index_expr.get_prefix_expr() { + Some(LuaExpr::NameExpr(name_expr)) => db + .get_reference_index() + .get_local_reference(&member_id.file_id) + .and_then(|reference| reference.get_decl_id(&name_expr.get_range())) + .and_then(|decl_id| db.get_decl_index().get_decl(&decl_id)) + .map(|decl| decl.is_global()) + .unwrap_or(false), + // A deeper index (`a.b.c`) belongs to the nested path's own + // reconciliation, not to the root global's. + _ => false, + } +} + +/// Re-homes members that reached a candidate table *directly*. +/// +/// A write whose prefix inferred to one concrete declaration while sibling +/// declarations of the same global were still unresolved attaches straight to +/// that table instead of parking on the global path, so the parked-member +/// reconciliation above never revisits it — and which table won depends on how +/// far the batch had run when the write was analysed. Applying the same target +/// rule the parked path uses (a declaring file keeps its own table, everyone +/// else belongs to the canonical owner) to every member sitting on a candidate +/// owner makes the outcome a function of the declared candidate set alone. +pub fn reconcile_directly_attached_candidate_members(db: &mut DbIndex) { + for global_id in db.get_global_index().sorted_multi_declaration_globals() { + let Some(candidates) = elected_global_owners(db, &global_id) else { + continue; + }; + if candidates.len() < 2 { + continue; + } + let Some((_, canonical_owner)) = candidates.first() else { + continue; + }; + let declaring_files = declaring_files(db, &global_id); + rehome_directly_attached_candidate_members( + db, + &global_id, + &candidates, + &declaring_files, + canonical_owner, + ); + } +} + +fn rehome_directly_attached_candidate_members( + db: &mut DbIndex, + global_id: &GlobalId, + candidates: &[(crate::FileId, LuaMemberOwner)], + declaring_files: &HashSet, + canonical_owner: &LuaMemberOwner, +) { + if candidates.len() < 2 { + return; + } + + let mut seen = HashSet::new(); + let writers = candidates + .iter() + .flat_map(|(_, owner)| db.get_member_index().get_member_history(owner)) + .filter(|member| seen.insert(member.get_id())) + .filter(|member| { + !db.get_member_index() + .has_synthesized_owner(&member.get_id()) + && !file_hands_global_to_scripted_class(db, member.get_file_id(), global_id) + && member_targets_global_path(db, member.get_id(), global_id) + }) + .map(|member| (member.get_id(), member.get_key().clone())) + .collect::>(); + for (member_id, member_key) in writers { + let Some(current) = db.get_member_index().get_member_owner(&member_id) else { + continue; + }; + let target = match candidates + .iter() + .find(|(file_id, _)| *file_id == member_id.file_id) + { + Some((_, owner)) => owner.clone(), + // See the parked-path rule: a file that declares the global but + // has not resolved its table keeps its members parked rather + // than sharing a sibling's overwrite slot. + None if declaring_files.contains(&member_id.file_id) => continue, + None => canonical_owner.clone(), + }; + let needs_move = *current != target && candidates.iter().any(|(_, owner)| owner == current); + let contribution_group_owner = db + .get_member_index() + .member_assignment_contributions() + .contribution_group_of(&member_id) + .map(|(owner, _)| owner); + let needs_contribution_move = contribution_group_owner + .as_ref() + .is_some_and(|owner| *owner != target); + if needs_contribution_move + && let Some(contribution) = db + .get_member_index() + .member_assignment_contributions() + .contribution_of(&member_id) + .cloned() + { + db.get_member_index_mut() + .member_assignment_contributions_mut() + .record(target.clone(), member_key.clone(), member_id, contribution); + } + if needs_move { + restore_non_overwriting_mark(db, member_id); + let member_index = db.get_member_index_mut(); + member_index.set_member_owner(target.clone(), member_id.file_id, member_id); + member_index.add_member_to_owner(target.clone(), member_id); + } + let member_index = db.get_member_index_mut(); + // Aliasing the remaining candidates is what makes a global declared + // once per realm behave like the single table it is at runtime, and it + // has to run even when nothing moved: re-indexing a file rebuilds its + // members from scratch, so the aliases the original migration created + // are gone. + for (_, alias_owner) in candidates { + if *alias_owner != target { + member_index.add_member_alias_to_owner(alias_owner.clone(), member_id); + } + } + } +} + /// Moves a member that landed on a *sibling* file's table literal onto the /// one its own file declares. fn rehome_members_onto_their_own_files_table( diff --git a/crates/glua_code_analysis/src/compilation/analyzer/common/mod.rs b/crates/glua_code_analysis/src/compilation/analyzer/common/mod.rs index c3fe78355..03471c1b2 100644 --- a/crates/glua_code_analysis/src/compilation/analyzer/common/mod.rs +++ b/crates/glua_code_analysis/src/compilation/analyzer/common/mod.rs @@ -2,7 +2,7 @@ mod migrate_global_member; use glua_parser::{LuaAstNode, LuaAstToken, LuaExpr, LuaForRangeStat}; pub(super) use migrate_global_member::{ migrate_global_members_when_type_resolve, migrate_global_path_members_when_owner_resolved, - reconcile_parked_global_path_members, + reconcile_directly_attached_candidate_members, reconcile_parked_global_path_members, }; use rowan::TextRange; diff --git a/crates/glua_code_analysis/src/compilation/analyzer/decl/exprs.rs b/crates/glua_code_analysis/src/compilation/analyzer/decl/exprs.rs index 16502b617..a4ee69a59 100644 --- a/crates/glua_code_analysis/src/compilation/analyzer/decl/exprs.rs +++ b/crates/glua_code_analysis/src/compilation/analyzer/decl/exprs.rs @@ -185,7 +185,7 @@ fn inferred_guard_candidate_references_param(expr: &LuaExpr, param_names: &[Stri && expr.descendants::().any(|name_expr| { name_expr .get_name_text() - .is_some_and(|name| param_names.iter().any(|param| param == &name)) + .is_some_and(|name| param_names.iter().any(|param| *param == name)) }) } @@ -652,7 +652,7 @@ fn dependency_call_has_no_args(expr: &LuaCallExpr) -> bool { fn get_call_name(expr: &LuaCallExpr) -> Option { match expr.get_prefix_expr()? { - LuaExpr::NameExpr(name_expr) => name_expr.get_name_text(), + LuaExpr::NameExpr(name_expr) => name_expr.get_name_text().map(Into::into), LuaExpr::IndexExpr(index_expr) => match index_expr.get_index_key()? { LuaIndexKey::Name(name) => Some(name.get_name_text().to_string()), LuaIndexKey::String(string) => Some(string.get_value()), diff --git a/crates/glua_code_analysis/src/compilation/analyzer/decl/mod.rs b/crates/glua_code_analysis/src/compilation/analyzer/decl/mod.rs index 52a672c8f..f3d098540 100644 --- a/crates/glua_code_analysis/src/compilation/analyzer/decl/mod.rs +++ b/crates/glua_code_analysis/src/compilation/analyzer/decl/mod.rs @@ -18,7 +18,9 @@ use super::{ }, gmod::ensure_scoped_class_type_decl_for_file, }; -use glua_parser::{LuaAst, LuaAstNode, LuaChunk, LuaFuncStat, LuaSyntaxKind, LuaVarExpr}; +use glua_parser::{ + LuaAst, LuaAstNode, LuaChunk, LuaFuncStat, LuaIfStat, LuaSyntaxKind, LuaVarExpr, +}; use rowan::{TextRange, TextSize, WalkEvent}; use crate::{ @@ -85,6 +87,25 @@ impl AnalysisPipeline for DeclAnalysisPipeline { } } +/// Records where each branch of `stat` begins and ends, and which `if` they +/// belong to. Writes in different branches of one `if` are alternatives and all +/// of them stay visible; every other pair of writes is successive, so the later +/// one wins. Recorded on the decl walk, which every file gets, so the answer +/// does not depend on how far inference reached. +fn record_if_branch_ranges(analyzer: &mut DeclAnalyzer, stat: &LuaIfStat) { + let if_range = stat.get_range(); + let file_id = analyzer.get_file_id(); + let branches = stat + .get_block() + .map(|block| block.get_range()) + .into_iter() + .chain(stat.get_all_clause().map(|clause| clause.get_range())); + let member_index = analyzer.db.get_member_index_mut(); + for branch in branches { + member_index.add_conditional_branch_range(file_id, branch, if_range); + } +} + fn walk_node_enter(analyzer: &mut DeclAnalyzer, node: LuaAst) { match node { LuaAst::LuaChunk(chunk) => { @@ -102,6 +123,9 @@ fn walk_node_enter(analyzer: &mut DeclAnalyzer, node: LuaAst) { analyzer.create_scope(stat.get_range(), LuaScopeKind::LocalOrAssignStat); stats::analyze_assign_stat(analyzer, stat); } + LuaAst::LuaIfStat(stat) => { + record_if_branch_ranges(analyzer, &stat); + } LuaAst::LuaForStat(stat) => { analyzer.create_scope(stat.get_range(), LuaScopeKind::Normal); stats::analyze_for_stat(analyzer, stat); diff --git a/crates/glua_code_analysis/src/compilation/analyzer/doc/type_ref_tags.rs b/crates/glua_code_analysis/src/compilation/analyzer/doc/type_ref_tags.rs index 8c71639b8..174797613 100644 --- a/crates/glua_code_analysis/src/compilation/analyzer/doc/type_ref_tags.rs +++ b/crates/glua_code_analysis/src/compilation/analyzer/doc/type_ref_tags.rs @@ -297,14 +297,6 @@ pub fn analyze_outparam(analyzer: &mut DocAnalyzer, tag: LuaDocTagOutparam) -> O return None; }; let field_path = path_segments.collect::>(); - if field_path.is_empty() { - report_invalid_outparam( - analyzer, - &tag, - format!("outparam `{path}` must target at least one field"), - ); - return None; - } let type_ref = tag .get_type() @@ -760,7 +752,7 @@ fn extract_func_name_from_ast(ast: &LuaAst) -> Option { LuaIndexKey::String(string_token) => Some(string_token.get_value()), _ => None, }, - LuaVarExpr::NameExpr(name_expr) => name_expr.get_name_text(), + LuaVarExpr::NameExpr(name_expr) => name_expr.get_name_text().map(Into::into), } } LuaAst::LuaLocalFuncStat(local_func) => local_func diff --git a/crates/glua_code_analysis/src/compilation/analyzer/dynamic_field.rs b/crates/glua_code_analysis/src/compilation/analyzer/dynamic_field.rs index 09ba9f60f..d556e8c85 100644 --- a/crates/glua_code_analysis/src/compilation/analyzer/dynamic_field.rs +++ b/crates/glua_code_analysis/src/compilation/analyzer/dynamic_field.rs @@ -911,7 +911,7 @@ fn param_expr_index(expr: &LuaExpr, param_names: &[String]) -> Option { let name = name_expr.get_name_text()?; param_names .iter() - .position(|param_name| param_name == &name) + .position(|param_name| *param_name == name) } #[derive(Default)] diff --git a/crates/glua_code_analysis/src/compilation/analyzer/flow/bind_analyze/stats.rs b/crates/glua_code_analysis/src/compilation/analyzer/flow/bind_analyze/stats.rs index c963d3752..513884dfd 100644 --- a/crates/glua_code_analysis/src/compilation/analyzer/flow/bind_analyze/stats.rs +++ b/crates/glua_code_analysis/src/compilation/analyzer/flow/bind_analyze/stats.rs @@ -177,8 +177,8 @@ fn collect_assignment_flow_info(binder: &FlowBinder, vars: &[LuaVarExpr]) -> Ass info } -fn push_assignment_index_path(info: &mut AssignmentFlowInfo, path: String) { - let path = internment::ArcIntern::from(smol_str::SmolStr::new(&path)); +fn push_assignment_index_path(info: &mut AssignmentFlowInfo, path: smol_str::SmolStr) { + let path = internment::ArcIntern::from(path); if !info.index_paths.contains(&path) { info.index_paths.push(path); } @@ -227,7 +227,7 @@ fn is_collection_append_write(index_expr: &LuaIndexExpr) -> bool { expr_access_path(&len_expr).is_some_and(|len_path| len_path == prefix_path) } -fn expr_access_path(expr: &LuaExpr) -> Option { +fn expr_access_path(expr: &LuaExpr) -> Option { match expr { LuaExpr::NameExpr(name_expr) => name_expr.get_access_path(), LuaExpr::IndexExpr(index_expr) => index_expr.get_access_path(), diff --git a/crates/glua_code_analysis/src/compilation/analyzer/flow/binder.rs b/crates/glua_code_analysis/src/compilation/analyzer/flow/binder.rs index a39653122..4c24290cf 100644 --- a/crates/glua_code_analysis/src/compilation/analyzer/flow/binder.rs +++ b/crates/glua_code_analysis/src/compilation/analyzer/flow/binder.rs @@ -238,7 +238,7 @@ impl<'a> FlowBinder<'a> { } pub fn get_goto_caches(&mut self) -> Vec { - self.goto_stats.drain(..).collect() + std::mem::take(&mut self.goto_stats) } pub fn get_flow(&self, flow_id: FlowId) -> Option<&FlowNode> { diff --git a/crates/glua_code_analysis/src/compilation/analyzer/gmod/mod.rs b/crates/glua_code_analysis/src/compilation/analyzer/gmod/mod.rs index 20bc483b5..aa7918747 100644 --- a/crates/glua_code_analysis/src/compilation/analyzer/gmod/mod.rs +++ b/crates/glua_code_analysis/src/compilation/analyzer/gmod/mod.rs @@ -142,9 +142,9 @@ fn scan_gmod_keywords( /// Pre-analysis phase: runs BEFORE lua_analyze. /// Collects purely syntactic metadata (hooks, network, realm, scripted class /// type declarations) so that lua_analyze has correct realm keys and scripted -/// class types available from the start. This avoids the previous architecture -/// where flow analysis used `GmodRealm::Unknown` during lua_analyze and had to -/// recompute everything in the unresolve phase with the correct realm. +/// class types available from the start. Without them lua_analyze would see +/// `GmodRealm::Unknown` and the unresolve phase would have to recompute every +/// flow once the realm became known. pub struct GmodPreAnalysisPipeline; impl AnalysisPipeline for GmodPreAnalysisPipeline { @@ -471,14 +471,12 @@ impl GmodPreProfile { /// resolving `net.Start`/`net.Send` reached through a wrapper needs the /// wrapper's signature, its receiver's type, and the members those depend on. /// -/// It used to run inside `GmodPreAnalysisPipeline`, before any of that existed. -/// The collector therefore saw a far poorer index on a cold build than on any -/// later re-index — on CityRP a cold index found 2592 flows where a re-index of -/// the same unchanged source found 2845 — so `gmod-net-*` diagnostics changed -/// across the workspace after the first edit. Nothing in the analysis pipeline -/// reads the network index (only diagnostics do), so collecting it last costs -/// nothing and is the only point at which the input state is the same for a -/// cold build and a partial re-index. +/// Collecting any earlier would let the collector see a poorer index on a cold +/// build than on a re-index, which makes `gmod-net-*` diagnostics change across +/// the workspace after the first edit. Nothing in the analysis pipeline reads +/// the network index (only diagnostics do), so collecting it last costs nothing +/// and is the only point at which the input state is the same for a cold build +/// and a partial re-index. pub struct GmodNetworkAnalysisPipeline; impl AnalysisPipeline for GmodNetworkAnalysisPipeline { @@ -517,6 +515,12 @@ impl AnalysisPipeline for GmodNetworkAnalysisPipeline { let file_ids: Vec = tree_list.iter().map(|tree| tree.file_id).collect(); let reach = HelperStartReachCache::default(); + let helper_call_sites = crate::profile::phase("gmodnet/helper_call_sites", || { + let op_names = net_operation_names(&annotated_global_call_roles); + let mut names = net_producing_function_names(db, &op_names); + names.extend(op_names); + net_helper_call_sites(db, names) + }); let collected = crate::profile::phase("gmodnet/collect_flows", || { super::parallel::map_files_collect(db, &file_ids, |db, file_id| { collect_file_network_flows( @@ -525,10 +529,10 @@ impl AnalysisPipeline for GmodNetworkAnalysisPipeline { &helper_registry, &annotated_global_call_roles, &reach, + &helper_call_sites, ) }) }); - for (file_id, network_data) in file_ids.iter().zip(collected) { if network_data.send_flows.is_empty() && network_data.receive_flows.is_empty() { continue; @@ -545,6 +549,7 @@ fn collect_file_network_flows( helper_registry: &HelperRegistry, annotated_global_call_roles: &AnnotatedGmodGlobalCallRoleMap, reach: &HelperStartReachCache, + helper_call_sites: &NetHelperCallSites, ) -> crate::db_index::FileNetworkData { let Some(root) = db .get_vfs() @@ -557,8 +562,8 @@ fn collect_file_network_flows( let mut local_fns = LocalFnCache::default(); let mut net = NetCallResolver::default(); // One memo for both walks: the receive walk and the three send walks start - // from the same call expressions and reach the same helpers, so resolving - // them twice was pure repeat work. + // from the same call expressions and reach the same helpers, so a shared + // memo resolves each of them once. let mut resolve_memo = ResolveMemo::default(); let (_, _, _, receive_flows) = crate::profile::phase("gmodnet/receive_walk", || { @@ -574,6 +579,7 @@ fn collect_file_network_flows( &mut net, &mut resolve_memo, reach, + helper_call_sites, ) }); @@ -587,6 +593,7 @@ fn collect_file_network_flows( &mut net, &mut resolve_memo, reach, + helper_call_sites, ) } @@ -1128,6 +1135,292 @@ impl HelperRegistryBuilder { } } +/// The final written name of a value expression, for alias discovery. +fn expr_written_name(expr: &LuaExpr) -> Option { + match expr { + LuaExpr::NameExpr(name_expr) => name_expr.get_name_text(), + LuaExpr::IndexExpr(index_expr) => match index_expr.get_index_key()? { + LuaIndexKey::Name(name) => Some(SmolStr::new(name.get_name_text())), + LuaIndexKey::String(string) => Some(SmolStr::new(string.get_value())), + _ => None, + }, + _ => None, + } +} + +/// The names the shipped net operations are declared under (`Start`, +/// `Receive`, and any annotated wrapper of them). +/// +/// Read straight out of the annotated call-role map the pre-analysis pass +/// already built, which is keyed by access path and already tags these calls +/// `NetStart`/`NetReceive`. The pre-pass records op *call sites* by path and so +/// cannot see `local recv = net.Receive`; carrying the op names lets the same +/// reference lookup and per-file binding closure that finds helper calls follow +/// that alias to its call sites. +fn net_operation_names(annotated_roles: &AnnotatedGmodGlobalCallRoleMap) -> HashSet { + annotated_roles + .roles_by_path + .iter() + .filter(|(_, roles)| { + roles.system_roles.iter().any(|(kind, _)| { + matches!( + kind, + GmodSystemCallKind::NetStart | GmodSystemCallKind::NetReceive + ) + }) + }) + .map(|(path, _)| { + SmolStr::new( + path.rsplit_once('.') + .map_or(path.as_str(), |(_, last)| last), + ) + }) + .collect() +} + +/// The written name of a registry entry's declaration, used to decide which +/// call sites could reach it. +fn closure_declared_name(closure: &LuaClosureExpr) -> Option { + if let Some(func_stat) = closure.get_parent::() + && let Some(func_name) = func_stat.get_func_name() + { + return var_expr_written_name(&func_name); + } + if let Some(local_func_stat) = closure.get_parent::() { + return local_func_stat + .get_local_name() + .and_then(|local_name| local_name.get_name_token()) + .map(|token| SmolStr::new(token.get_name_text())); + } + if let Some(assign_stat) = closure.get_parent::() { + let (vars, value_exprs) = assign_stat.get_var_and_expr_list(); + let idx = value_exprs + .iter() + .position(|expr| expr.get_position() == closure.get_position())?; + return var_expr_written_name(vars.get(idx)?); + } + if let Some(table_field) = closure.get_parent::() + && let Some(field_key) = table_field.get_field_key() + { + return match field_key { + LuaIndexKey::Name(name) => Some(SmolStr::new(name.get_name_text())), + LuaIndexKey::String(string) => Some(SmolStr::new(string.get_value())), + _ => None, + }; + } + if let Some(local_stat) = closure.get_parent::() { + let idx = local_stat + .get_value_exprs() + .position(|expr| expr.get_position() == closure.get_position())?; + return local_stat + .get_local_name_list() + .nth(idx) + .and_then(|local_name| local_name.get_name_token()) + .map(|token| SmolStr::new(token.get_name_text())); + } + None +} + +fn var_expr_written_name(var_expr: &LuaVarExpr) -> Option { + match var_expr { + LuaVarExpr::NameExpr(name_expr) => name_expr.get_name_text(), + LuaVarExpr::IndexExpr(index_expr) => match index_expr.get_index_key()? { + LuaIndexKey::Name(name) => Some(SmolStr::new(name.get_name_text())), + LuaIndexKey::String(string) => Some(SmolStr::new(string.get_value())), + _ => None, + }, + } +} + +/// The names a call has to be written with for it to expand into a helper that +/// can reach a `net.Start`. +/// +/// The helpers that can answer yes are a small fixed set, and resolution +/// matches declarations by written name, so a call written with a name no such +/// helper carries cannot expand into one. +/// +/// Seeded from the net operations' own references, then grown outward: each +/// site is walked *up* to the function containing it, and that function's own +/// call sites come from the reference index, whose enclosing functions are the +/// next level. The set settles when a round adds no new site. +/// +/// Nothing is scanned. The cost is proportional to how much net code the +/// workspace actually has, not to its size. +fn net_producing_function_names(db: &DbIndex, op_names: &HashSet) -> HashSet { + let mut names: HashSet = HashSet::new(); + let mut visited_decls: HashSet = HashSet::new(); + // Seeded from the net operations' own references rather than from the + // pre-pass's recorded call sites: that record is only written for files + // that also need hook metadata, so it is not a complete list of net ops. + // The reference index records every reference unconditionally. + let mut frontier: Vec> = op_names + .iter() + .flat_map(|name| name_reference_sites(db, name)) + .collect(); + + while !frontier.is_empty() { + // Grouped so each file's red tree is built once per round rather than + // once per site. + let mut by_file: FxHashMap> = FxHashMap::default(); + for site in frontier.drain(..) { + by_file.entry(site.file_id).or_default().push(site.value); + } + + let mut fresh: Vec = Vec::new(); + let mut next: Vec> = Vec::new(); + for (file_id, syntax_ids) in by_file { + let Some(root) = db + .get_vfs() + .get_syntax_tree(&file_id) + .map(|tree| tree.get_red_root()) + else { + continue; + }; + for syntax_id in syntax_ids { + let Some(node) = syntax_id.to_node_from_root(&root) else { + continue; + }; + let Some(closure) = node.ancestors().find_map(LuaClosureExpr::cast) else { + continue; + }; + let Some(name) = closure_declared_name(&closure) else { + continue; + }; + // A local enters neither name-keyed reference table, so a chain + // through local wrappers only continues if the next level comes + // from the declaration's own references. + if let Some(decl_id) = closure_local_decl_id(file_id, &closure) + && visited_decls.insert(decl_id) + { + next.extend(decl_reference_sites(db, decl_id)); + } + if names.insert(name.clone()) { + fresh.push(name); + } + } + } + + // A newly named function's callers are the next level, and the + // reference index already knows where they are. + for name in fresh { + next.extend(name_reference_sites(db, &name)); + } + frontier = next; + } + + names +} + +/// Every place a name is referenced, from the reference index. +fn name_reference_sites(db: &DbIndex, name: &SmolStr) -> Vec> { + let reference_index = db.get_reference_index(); + let member_key = LuaMemberKey::Name(name.clone()); + reference_index + .get_index_references(&member_key) + .into_iter() + .flatten() + .chain( + reference_index + .get_global_references(name) + .into_iter() + .flatten(), + ) + .collect() +} + +/// Every place a local declaration is read, from the reference index. +fn decl_reference_sites(db: &DbIndex, decl_id: LuaDeclId) -> Vec> { + let Some(references) = db + .get_reference_index() + .get_decl_references(&decl_id.file_id, &decl_id) + else { + return Vec::new(); + }; + references + .cells + .iter() + .filter(|cell| !cell.is_write) + .map(|cell| { + InFiled::new( + decl_id.file_id, + LuaSyntaxId::new(glua_parser::LuaSyntaxKind::NameExpr.into(), cell.range), + ) + }) + .collect() +} + +/// The declaration a closure is bound to, when that binding is a local. +/// +/// A declaration is identified by the position of its declared name, so the two +/// local binding forms yield it without a lookup. +fn closure_local_decl_id(file_id: FileId, closure: &LuaClosureExpr) -> Option { + if let Some(local_func_stat) = closure.get_parent::() { + return Some(LuaDeclId::new( + file_id, + local_func_stat.get_local_name()?.get_position(), + )); + } + let local_stat = closure.get_parent::()?; + let idx = local_stat + .get_value_exprs() + .position(|expr| expr.get_position() == closure.get_position())?; + Some(LuaDeclId::new( + file_id, + local_stat.get_local_name_list().nth(idx)?.get_position(), + )) +} + +/// The call sites that can expand into a helper able to reach a `net.Start`. +/// +/// Every reference to a name is recorded in the reference index while the +/// workspace is indexed, so these call sites are a direct lookup rather than a +/// walk that resolves every call expression in every file. +#[derive(Default)] +struct NetHelperCallSites { + /// Syntax id of the callee reference node, per file. + by_file: FxHashMap>, + /// The helper names themselves, needed per file to pick up locals: a + /// `local function send()` never enters the global reference table, so its + /// call sites are only reachable through its declaration's own references. + names: HashSet, +} + +fn net_helper_call_sites(db: &DbIndex, names: HashSet) -> NetHelperCallSites { + let mut by_file: FxHashMap> = FxHashMap::default(); + let reference_index = db.get_reference_index(); + for name in &names { + let member_key = LuaMemberKey::Name(name.clone()); + for reference in reference_index + .get_index_references(&member_key) + .into_iter() + .flatten() + .chain( + reference_index + .get_global_references(name) + .into_iter() + .flatten(), + ) + { + by_file + .entry(reference.file_id) + .or_default() + .push(reference.value); + } + } + // Source order, so a file's flows are collected in the same order the walk + // produced them. The key is the whole identity rather than just the start + // offset: sites arrive in hash order, and sorting on a partial key leaves + // equal ids non-adjacent, which silently defeats the dedup. + for sites in by_file.values_mut() { + sites.sort_by_key(|syntax_id| { + let range = syntax_id.get_range(); + (range.start(), range.end(), syntax_id.get_kind()) + }); + sites.dedup(); + } + NetHelperCallSites { by_file, names } +} + /// Per-file function definition lookup. Built once and reused for all /// helper-resolution queries against the same file's syntax tree. struct FileFunctionMap { @@ -1187,7 +1480,7 @@ impl FileFunctionMap { }; match func_stat.get_func_name() { Some(LuaVarExpr::NameExpr(name_expr)) => { - if let Some(name) = name_expr.get_name_text() { + if let Some(name) = name_expr.get_name_text().map(String::from) { if bare.insert(name.clone(), block.clone()).is_some() { duplicate_bare.insert(name); } @@ -1211,7 +1504,7 @@ impl FileFunctionMap { if let Some(var) = vars.get(idx) { match var { LuaVarExpr::NameExpr(name_expr) => { - if let Some(name) = name_expr.get_name_text() { + if let Some(name) = name_expr.get_name_text().map(String::from) { if bare.insert(name.clone(), block.clone()).is_some() { duplicate_bare.insert(name); } @@ -1332,9 +1625,8 @@ fn collect_file_gmod_metadata( let collect_non_net_metadata = keywords.needs_hook_metadata(); // Network flows are collected later, by `GmodNetworkAnalysisPipeline`; see - // that pipeline for why. When a file needs no hook metadata either, the - // whole walk can now be skipped — previously it still had to run because - // receive-flow collection rode along with it. + // that pipeline for why. Receive-flow collection therefore no longer rides + // along here, so a file that needs no hook metadata skips the walk whole. let hook_metadata = collect_non_net_metadata.then(|| { let (hook_sites, system_metadata, gm_method_realms, _receive_flows) = collect_hook_and_receive_metadata( @@ -1349,6 +1641,9 @@ fn collect_file_gmod_metadata( &mut net, &mut ResolveMemo::default(), &reach, + // This walk collects hook metadata only; it never expands + // wrapper chains for send flows, so it needs no call sites. + &NetHelperCallSites::default(), ); (hook_sites, system_metadata, gm_method_realms) }); @@ -1398,6 +1693,7 @@ fn collect_hook_and_receive_metadata( net: &mut NetCallResolver, resolve_memo: &mut ResolveMemo, reach: &HelperStartReachCache, + helper_call_sites: &NetHelperCallSites, ) -> ( Vec, GmodSystemFileMetadata, @@ -1414,11 +1710,10 @@ fn collect_hook_and_receive_metadata( root: root.clone(), file_id, }; - // Built once for the whole walk. `resolve_memo` is a pure function of - // `(file_id, call range)` for a fixed registry and index — see the - // field's own doc — but this context used to be constructed *inside* - // the loop, so every call expression started with an empty memo and - // paid a fresh `FxHashMap` allocation. + // Built once for the whole walk rather than per call expression, so the + // memo carries across the walk instead of being reallocated empty each + // time. `resolve_memo` is a pure function of `(file_id, call range)` for a + // fixed registry and index — see the field's own doc. let mut net_ctx = NetCollectCtx { db, helper_registry, @@ -1426,8 +1721,32 @@ fn collect_hook_and_receive_metadata( net, resolve_memo, reach, + helper_call_sites, }; + // Collecting receive flows alone needs no walk: the `net.Receive` sites are + // already recorded by annotation, and the calls that can expand into a + // wrapper that reaches one come from the reference index. + if !collect_non_net_metadata { + if collect_receive_flows { + for call_expr in net_candidate_call_exprs(db, &net_site, helper_call_sites) { + if let Some(receive_flow) = + collect_net_receive_flow(&mut net_ctx, &net_site, &call_expr) + { + receive_flows.push(receive_flow); + } else if call_has_literal_string_arg(&call_expr) { + receive_flows.extend(collect_unannotated_net_wrapper_receive_flows( + &mut net_ctx, + &net_site, + &call_expr, + )); + } + } + receive_flows.sort_by_key(|flow| flow.receive_range.start()); + } + return (hook_sites, system_metadata, gm_method_realms, receive_flows); + } + // Single descendants walk dispatching by node kind. Avoids two separate // O(N) walks for the LuaCallExpr and LuaFuncStat passes. for node in root.syntax().descendants() { @@ -1495,6 +1814,7 @@ fn collect_network_flow_metadata( net: &mut NetCallResolver, resolve_memo: &mut ResolveMemo, reach: &HelperStartReachCache, + helper_call_sites: &NetHelperCallSites, ) -> crate::db_index::FileNetworkData { let site = NetWalkSite { root, file_id }; let mut ctx = NetCollectCtx { @@ -1504,6 +1824,7 @@ fn collect_network_flow_metadata( net, resolve_memo, reach, + helper_call_sites, }; let mut send_flows = crate::profile::phase("gmodnet/send_direct", || { collect_net_send_flows(&mut ctx, &site) @@ -1899,7 +2220,8 @@ fn collect_unannotated_net_wrapper_send_flows( let mut visited = HashSet::new(); let empty_bindings = HashMap::new(); - for call_expr in site.root.descendants::() { + let calls = net_candidate_call_exprs(ctx.db, site, ctx.helper_call_sites); + for call_expr in calls { if ctx.net.role(ctx.db, site.file_id, &call_expr).is_some() { continue; } @@ -1917,6 +2239,107 @@ fn collect_unannotated_net_wrapper_send_flows( flows } +/// The calls in a file that can take part in net-flow collection. +/// +/// A lookup rather than a walk: the call sites that can expand into a +/// net-producing helper come from the reference index, which already records +/// every reference to those helpers' names. +fn net_candidate_call_exprs( + db: &DbIndex, + site: &NetWalkSite, + helper_call_sites: &NetHelperCallSites, +) -> Vec { + let root_syntax = site.root.syntax().clone(); + // Locals never enter the global reference table, so a helper declared + // `local function send()` is reached through its own declaration's + // references instead. + let mut local_sites: Vec = Vec::new(); + if let Some(decl_tree) = db.get_decl_index().get_decl_tree(&site.file_id) { + let names = &helper_call_sites.names; + // `local sendString = MyLib.SendString` calls the helper under a name + // the reference index files under the local binding rather than under + // the helper, so this file's own bindings of a helper name count as + // call sites too. Chains settle by iterating, bounded by the number of + // bindings in the file. + let mut aliases: HashSet = HashSet::new(); + let bindings = decl_tree + .get_decls() + .values() + .filter_map(|decl| { + let source = decl + .get_value_syntax_id()? + .to_node_from_root(&root_syntax) + .and_then(LuaExpr::cast) + .as_ref() + .and_then(expr_written_name)?; + Some((SmolStr::new(decl.get_name()), source)) + }) + .collect::>(); + loop { + let mut added = false; + for (bound, source) in &bindings { + if (names.contains(source) || aliases.contains(source)) + && !names.contains(bound) + && aliases.insert(bound.clone()) + { + added = true; + } + } + if !added { + break; + } + } + + for decl in decl_tree.get_decls().values() { + if !decl.is_local() + || !(names.contains(decl.get_name()) || aliases.contains(decl.get_name())) + { + continue; + } + let Some(references) = db + .get_reference_index() + .get_decl_references(&site.file_id, &decl.get_id()) + else { + continue; + }; + local_sites.extend( + references + .cells + .iter() + .filter(|cell| !cell.is_write) + .map(|cell| { + LuaSyntaxId::new(glua_parser::LuaSyntaxKind::NameExpr.into(), cell.range) + }), + ); + } + } + let mut calls = helper_call_sites + .by_file + .get(&site.file_id) + .map(|syntax_ids| syntax_ids.as_slice()) + .unwrap_or_default() + .iter() + .chain(local_sites.iter()) + .filter_map(|syntax_id| syntax_id.to_node_from_root(&root_syntax)) + .filter_map(|node| { + let call_expr = LuaCallExpr::cast(node.parent()?)?; + // The reference is the callee only when it is the call's prefix; + // `f(SendThing)` passes it as an argument instead. + (call_expr.get_prefix_expr()?.syntax() == &node).then_some(call_expr) + }) + .collect::>(); + + // Source order. The key is the whole range so that equal calls reached + // through both the name and the local-declaration lookup end up adjacent + // and the dedup can see them. + calls.sort_by_key(|call_expr| { + let range = call_expr.get_range(); + (range.start(), range.end()) + }); + calls.dedup_by_key(|call_expr| call_expr.get_range()); + calls +} + #[allow(clippy::too_many_arguments)] fn collect_send_flows_from_helper_call( ctx: &mut NetCollectCtx<'_>, @@ -1943,8 +2366,8 @@ fn collect_send_flows_from_helper_call( // A send flow always starts at a `net.Start` somewhere in the expansion, so // a helper that cannot reach one contributes nothing however it is called. - // Answering that once per helper instead of walking its body once per - // calling file is the difference between ~438k body scans and ~2k. + // The answer depends only on the helper, so it is cached per helper rather + // than recomputed for each calling file. let helper_id = (helper_file_id, helper_key.clone()); let (reaches_start, _) = helper_reaches_net_role( ctx, @@ -2304,7 +2727,11 @@ fn resolve_callback_block( }; let target_name = name_expr.get_name_text()?; - local_fns.get(file_id, root).bare.get(&target_name).cloned() + local_fns + .get(file_id, root) + .bare + .get(target_name.as_str()) + .cloned() } /// Resolve a call expression to a function definition, returning a @@ -2375,7 +2802,11 @@ fn resolve_call_to_function_block( // written bare name identifies exactly one function body in that file. if let Some(LuaExpr::NameExpr(name_expr)) = call_expr.get_prefix_expr() && let Some(name) = name_expr.get_name_text() - && let Some(block) = local_fns.get(root_file_id, root).bare.get(&name).cloned() + && let Some(block) = local_fns + .get(root_file_id, root) + .bare + .get(name.as_str()) + .cloned() { return Some(( format!("unique-local:{name}"), @@ -2427,6 +2858,8 @@ struct NetCollectCtx<'a> { resolve_memo: &'a mut ResolveMemo, /// Shared across files; see [`HelperStartReachCache`]. reach: &'a HelperStartReachCache, + /// See [`NetHelperCallSites`]. + helper_call_sites: &'a NetHelperCallSites, } type ResolvedHelperFn = (String, LuaBlock, LuaChunk, FileId); @@ -2926,10 +3359,36 @@ enum BranchKind { ElseIf, } +/// The node's text up to its first newline. +/// +/// `node.text().to_string()` walks and concatenates every token underneath, so +/// asking an `if` statement for its header used to materialise the statement's +/// whole body — thousands of lines for a large branch — to read one line of it. +/// Network flow analysis does that for every branch around every `net` call in +/// every re-analysed file, which made it one of the more expensive things a +/// keystroke paid for. +fn first_line_text(node: &LuaSyntaxNode) -> String { + let mut line = String::new(); + for token in node + .descendants_with_tokens() + .filter_map(|element| element.into_token()) + { + let text = token.text(); + match text.find('\n') { + Some(end) => { + line.push_str(&text[..end]); + break; + } + None => line.push_str(text), + } + } + line +} + /// Pulls the header text for an `elseif cond then` clause from source. fn extract_branch_header(node: &LuaSyntaxNode, kind: BranchKind) -> Option { const MAX_HEADER_LEN: usize = 80; - let full = node.text().to_string(); + let full = first_line_text(node); let trimmed = full.trim_start(); let nl_idx = trimmed.find('\n').unwrap_or(trimmed.len()); let first_line = &trimmed[..nl_idx]; @@ -2964,7 +3423,9 @@ fn extract_branch_header(node: &LuaSyntaxNode, kind: BranchKind) -> Option Option { const MAX_HEADER_LEN: usize = 80; - let full = stat_node.text().to_string(); + // Only the opener is ever read, and it bails on a multi-line one, so there + // is no reason to materialise the statement's whole body first. + let full = first_line_text(stat_node); let header_raw = match kind { NetFlowKind::Repeat => { // `repeat` itself has no condition until `until` at the end. @@ -3059,6 +3520,13 @@ enum NetCallRole { struct NetCallResolver { caches: HashMap, memo: HashMap<(FileId, LuaSyntaxId), Option>, + /// `role` memoises the *role*, but the signature behind it is asked for + /// twice per call site: once here and once by + /// [`resolve_call_to_function_block`]'s signature path. Resolving a call's + /// signature means resolving its prefix to a semantic decl, which is the + /// single most expensive operation in this pipeline, so the answer is + /// memoised on the same key the role is. + signature_memo: HashMap<(FileId, LuaSyntaxId), Option>, } impl NetCallResolver { @@ -3147,6 +3615,22 @@ impl NetCallResolver { db: &DbIndex, file_id: FileId, call_expr: &LuaCallExpr, + ) -> Option { + let key = (file_id, LuaSyntaxId::from_node(call_expr.syntax())); + if let Some(cached) = self.signature_memo.get(&key) { + return *cached; + } + + let resolved = self.signature_id_uncached(db, file_id, call_expr); + self.signature_memo.insert(key, resolved); + resolved + } + + fn signature_id_uncached( + &mut self, + db: &DbIndex, + file_id: FileId, + call_expr: &LuaCallExpr, ) -> Option { let cache = self .caches @@ -3356,6 +3840,9 @@ fn collect_scripted_scope_type_bindings_with( if decls.is_empty() { return; } + // The class is anchored on the first declaration, so which one that is + // must not depend on the order the decl map happens to iterate in. + decls.sort_by_key(|(_, range)| (range.start(), range.end())); let class_decl_id = ensure_scoped_class_type_decl( db, @@ -3483,19 +3970,9 @@ pub(crate) fn resolve_scoped_authoring_type( .then(|| get_scripted_class_type_decl_id(&info.global_name, &info.class_name)) } -#[derive(Clone)] -enum ResolvedVguiParentSource { - Direct(Vec), - AssignedField { - field_type_ids: Vec, - assignment_parent_type_ids: Vec, - }, - ReceiverField { - field_type_ids: Vec, - receiver_type_ids: Vec, - receiver_field_parent_type_ids: Option>, - }, -} +use crate::{ + GmodVguiParentSourceResolution as ResolvedVguiParentSource, GmodVguiResolvedParentSource, +}; #[derive(Clone)] struct ResolvedVguiParentRelation { @@ -3595,6 +4072,26 @@ fn resolve_vgui_parent_relations( if calls.is_empty() { continue; } + // Every call already resolved means this file was not rebuilt, so + // walking its syntax tree would reproduce what is cached. Skipping it is + // the whole point: only a handful of the workspace's vgui files are + // touched by any one edit. + if let Some(cached) = calls + .iter() + .map(|call| { + call.resolved_source + .as_ref() + .map(|source| ResolvedVguiParentRelation { + syntax_id: call.syntax_id, + child_type_ids: source.child_type_ids.clone(), + parent: source.parent.clone(), + }) + }) + .collect::>>() + { + relations_by_file.push((file_id, cached)); + continue; + } let Some(root) = db .get_vfs() .get_syntax_tree(&file_id) @@ -3638,6 +4135,27 @@ fn resolve_vgui_parent_relations( relations_by_file.push((file_id, relations)); } + let resolved_sources_by_file = relations_by_file + .iter() + .map(|(file_id, relations)| { + let sources = relations + .iter() + .map(|relation| { + ( + relation.syntax_id, + GmodVguiResolvedParentSource { + child_type_ids: relation.child_type_ids.clone(), + parent: relation.parent.clone(), + }, + ) + }) + .collect(); + (*file_id, sources) + }) + .collect::>(); + db.get_gmod_class_metadata_index_mut() + .set_vgui_resolved_parent_sources(&resolved_sources_by_file); + let mut direct_parents_by_child = HashMap::>>::new(); let mut relations_by_child = HashMap::>::new(); for (_, relations) in &relations_by_file { @@ -3897,6 +4415,7 @@ fn collect_vgui_forwarding_parent_calls( child: GmodVguiParentSource::Expr(child.get_syntax_id()), parent: GmodVguiParentSource::LiteralName(parent_type_id.get_name().to_string()), relations: Vec::new(), + resolved_source: None, origin: GmodVguiParentCallOrigin::Forwarded, }); } @@ -4265,7 +4784,7 @@ fn resolve_vgui_field_assignment_parent_type_ids( let owner = field_expr.get_prefix_expr()?; let owner_type_ids = resolve_vgui_parent_expr_type_ids(db, cache, owner); let mut candidates = field_assignment_parents - .get(&field_path)? + .get(field_path.as_str())? .iter() .filter(|assignment| { !field_type_ids.is_empty() && assignment.owner_type_ids == owner_type_ids @@ -4311,12 +4830,13 @@ fn index_vgui_field_assignment_parents( if parent_type_ids.is_empty() { continue; } - assignments.entry(field_path).or_insert_with(Vec::new).push( - VguiFieldAssignmentParent { + assignments + .entry(field_path.to_string()) + .or_insert_with(Vec::new) + .push(VguiFieldAssignmentParent { owner_type_ids: resolve_vgui_parent_expr_type_ids(db, cache, owner), parent_type_ids, - }, - ); + }); } } assignments @@ -4663,7 +5183,8 @@ fn find_and_resolve_getmember_delegations( continue; }; - let Some((target_class, target_method)) = getmember_locals.get(&caller_name) else { + let Some((target_class, target_method)) = getmember_locals.get(caller_name.as_str()) + else { continue; }; if target_method != "SetupDataTables" { @@ -5718,11 +6239,11 @@ fn extract_scoped_base_name(expr: &LuaExpr) -> Option { }, LuaExpr::NameExpr(name_expr) => { let value = name_expr.get_name_text()?; - (!value.trim().is_empty()).then_some(value) + (!value.trim().is_empty()).then(|| value.to_string()) } LuaExpr::IndexExpr(index_expr) => { let value = index_expr.get_access_path()?; - (!value.trim().is_empty()).then_some(value) + (!value.trim().is_empty()).then(|| value.to_string()) } _ => None, } @@ -6015,7 +6536,7 @@ fn resolve_wrapper_arg_mapping( } LuaExpr::NameExpr(name_expr) => { if let Some(name) = name_expr.get_name_text() { - if let Some(idx) = param_names.iter().position(|p| p == &name) { + if let Some(idx) = param_names.iter().position(|p| *p == name) { return (None, Some(idx)); } } @@ -9082,7 +9603,7 @@ impl<'a> AnnotatedGmodCallRoleMap<'a> { let Some(call_path) = func_name.get_access_path() else { continue; }; - role_map.add_local_path_roles(root_decl_id, call_path, roles); + role_map.add_local_path_roles(root_decl_id, call_path.to_string(), roles); } for local_func_stat in root.descendants::() { @@ -9441,7 +9962,11 @@ fn closure_from_signature_id(db: &DbIndex, signature_id: LuaSignatureId) -> Opti .get_vfs() .get_syntax_tree(&signature_id.get_file_id())? .get_red_root(); - root.descendants() + // A signature's position is the offset its closure starts at, so descend to + // that offset rather than scanning every node in the file. + root.token_at_offset(signature_id.get_position()) + .right_biased()? + .parent_ancestors() .filter_map(LuaClosureExpr::cast) .find(|closure| closure.get_position() == signature_id.get_position()) } @@ -9455,7 +9980,7 @@ fn global_call_path_for_signature_closure( if let Some(func_stat) = closure.get_parent::() { let func_name = func_stat.get_func_name()?; return var_expr_has_global_root(db, file_id, &func_name) - .then(|| func_name.get_access_path())?; + .then(|| func_name.get_access_path().map(Into::into))?; } let assign_stat = closure.get_parent::()?; @@ -9464,7 +9989,8 @@ fn global_call_path_for_signature_closure( .iter() .position(|expr| expr.get_position() == closure.get_position())?; let var_expr = vars.get(value_idx)?; - var_expr_has_global_root(db, file_id, var_expr).then(|| var_expr.get_access_path())? + var_expr_has_global_root(db, file_id, var_expr) + .then(|| var_expr.get_access_path().map(Into::into))? } fn var_expr_has_global_root(db: &DbIndex, file_id: FileId, var_expr: &LuaVarExpr) -> bool { @@ -9811,6 +10337,7 @@ fn collect_annotated_scripted_class_call_metadata( child, parent, relations: Vec::new(), + resolved_source: None, origin: GmodVguiParentCallOrigin::Annotated, })); } @@ -12208,7 +12735,7 @@ fn collect_dynamic_wrapper_call_usage( let Some(path) = call_expr.get_access_path() else { return DynamicLoadUsage::default(); }; - let Some(wrapper) = wrappers.get(&path) else { + let Some(wrapper) = wrappers.get(path.as_str()) else { return DynamicLoadUsage::default(); }; let Some(args_list) = call_expr.get_args_list() else { @@ -12272,7 +12799,7 @@ fn collect_dynamic_load_wrappers(root: &LuaChunk) -> HashMap { let path = index_expr.get_access_path()?; aliases - .get(&path) + .get(path.as_str()) .copied() .or_else(|| annotated_roles.load_alias_for_reference_expr(db, file_id, expr)) } @@ -12757,7 +13284,7 @@ fn collect_dynamic_binding_writes(root: &LuaChunk) -> Vec { continue; }; writes.push(DynamicBindingWrite { - name: path, + name: path.to_string(), scope, range, }); @@ -13043,7 +13570,7 @@ fn collect_static_string_bindings(root: &LuaChunk) -> HashMap { continue; }; if let Some(value) = static_string_expr(value, &bindings) { - bindings.insert(name, value); + bindings.insert(name.to_string(), value); } } } @@ -13975,7 +14502,7 @@ fn rebuild_realm_metadata( }; if !detect_filename && !detect_calls { - let realm_metadata = file_ids + let realm_metadata: rustc_hash::FxHashMap = file_ids .into_iter() .map(|file_id| { let ranges = if meta_file_ids.contains(&file_id) { @@ -14001,13 +14528,13 @@ fn rebuild_realm_metadata( }, ) }) - .collect::>(); + .collect(); db.get_gmod_infer_index_mut() .set_all_realm_file_metadata(realm_metadata); return; } - let mut realm_metadata = HashMap::new(); + let mut realm_metadata = rustc_hash::FxHashMap::default(); for file_id in file_ids { let ranges = if meta_file_ids.contains(&file_id) { Vec::new() diff --git a/crates/glua_code_analysis/src/compilation/analyzer/gmod/numeric_range_population.rs b/crates/glua_code_analysis/src/compilation/analyzer/gmod/numeric_range_population.rs index c29df298d..03d29fd1e 100644 --- a/crates/glua_code_analysis/src/compilation/analyzer/gmod/numeric_range_population.rs +++ b/crates/glua_code_analysis/src/compilation/analyzer/gmod/numeric_range_population.rs @@ -72,7 +72,7 @@ pub(super) fn collect_numeric_range_table_populations_for_file( if let LuaVarExpr::NameExpr(name_expr) = var && let Some(name) = name_expr.get_name_text() { - local_helpers.remove(&name); + local_helpers.remove(name.as_str()); } } } @@ -100,7 +100,7 @@ fn simple_func_stat_name(func_stat: &LuaFuncStat) -> Option { let LuaVarExpr::NameExpr(name_expr) = func_stat.get_func_name()? else { return None; }; - name_expr.get_name_text() + name_expr.get_name_text().map(Into::into) } fn numeric_range_populations_from_outer_call( @@ -208,7 +208,7 @@ fn attach_exact_alias_assignment( } for population in populations { if population.table_global == source_root { - population.alias_roots.push(alias_root.clone()); + population.alias_roots.push(alias_root.to_string()); population.alias_roots.sort(); population.alias_roots.dedup(); return true; @@ -259,7 +259,7 @@ fn root_name_from_expr(expr: &LuaExpr) -> Option { prefix = index_expr.get_prefix_expr()?; } match prefix { - LuaExpr::NameExpr(name_expr) => name_expr.get_name_text(), + LuaExpr::NameExpr(name_expr) => name_expr.get_name_text().map(Into::into), _ => None, } } @@ -272,7 +272,7 @@ fn assign_writes_tracked_helper( vars.into_iter().any(|var| { matches!(var, LuaVarExpr::NameExpr(name_expr) if name_expr .get_name_text() - .is_some_and(|name| helpers.contains_key(&name))) + .is_some_and(|name| helpers.contains_key(name.as_str()))) }) } @@ -299,7 +299,7 @@ fn reset_table_names(assign_stat: &LuaAssignStat) -> Vec { let (vars, _) = assign_stat.get_var_and_expr_list(); vars.into_iter() .filter_map(|var| match var { - LuaVarExpr::NameExpr(name_expr) => name_expr.get_name_text(), + LuaVarExpr::NameExpr(name_expr) => name_expr.get_name_text().map(String::from), _ => None, }) .collect() @@ -327,7 +327,7 @@ fn helper_invalidated_by_descendant_write( for var in vars { if let LuaVarExpr::NameExpr(name_expr) = var && let Some(name) = name_expr.get_name_text() - && local_helpers.contains_key(&name) + && local_helpers.contains_key(name.as_str()) { return true; } @@ -360,7 +360,7 @@ fn call_expr_name(call_expr: &LuaCallExpr) -> Option { let LuaExpr::NameExpr(name_expr) = call_expr.get_prefix_expr()? else { return None; }; - name_expr.get_name_text() + name_expr.get_name_text().map(Into::into) } fn call_name_shadowed_in_closure_before_call( @@ -600,7 +600,7 @@ fn protected_pre_loop_names( fn collect_expr_name_texts(expr: &LuaExpr, names: &mut HashSet) { for name_expr in expr.descendants::() { if let Some(name) = name_expr.get_name_text() { - names.insert(name); + names.insert(name.to_string()); } } } @@ -765,7 +765,7 @@ fn pre_loop_helper_body_is_safe( } LuaVarExpr::NameExpr(name_expr) => { if name_expr.get_name_text().is_none_or(|name| { - protected_names.contains(&name) + protected_names.contains(name.as_str()) || !name_expr_resolves_to_local(db, file_id, &name_expr) }) { active_helpers.remove(helper_name); @@ -976,7 +976,8 @@ fn branchy_assignment_is_safe( return false; }; if name_expr.get_name_text().is_none_or(|name| { - protected_names.contains(&name) || !name_expr_resolves_to_local(db, file_id, &name_expr) + protected_names.contains(name.as_str()) + || !name_expr_resolves_to_local(db, file_id, &name_expr) }) { return false; } @@ -1319,7 +1320,7 @@ fn helper_body_mutates_or_shadows_params( if vars.into_iter().any(|var| { matches!(var, LuaVarExpr::NameExpr(name_expr) if name_expr .get_name_text() - .is_some_and(|name| param_names.contains(&name))) + .is_some_and(|name| param_names.contains(name.as_str()))) }) { return true; } diff --git a/crates/glua_code_analysis/src/compilation/analyzer/infer_cache_manager.rs b/crates/glua_code_analysis/src/compilation/analyzer/infer_cache_manager.rs index 1bc0b0493..33ab896ad 100644 --- a/crates/glua_code_analysis/src/compilation/analyzer/infer_cache_manager.rs +++ b/crates/glua_code_analysis/src/compilation/analyzer/infer_cache_manager.rs @@ -80,6 +80,12 @@ impl InferCacheManager { self.current_phase = LuaAnalysisPhase::Force; for infer_cache in self.infer_map.values_mut() { infer_cache.set_phase(LuaAnalysisPhase::Force); + // The force phase answers a failed inference with a floor instead + // of an error, so a failure recorded under the previous phase is + // not the answer this one would give. Replaying it lets whichever + // expression in a chain happened to be walked first decide the + // result, which is the batch's settling order leaking into a type. + infer_cache.clear_deferred_inference_results(); } } diff --git a/crates/glua_code_analysis/src/compilation/analyzer/local_inference/mod.rs b/crates/glua_code_analysis/src/compilation/analyzer/local_inference/mod.rs index ce98efb62..aebb71be4 100644 --- a/crates/glua_code_analysis/src/compilation/analyzer/local_inference/mod.rs +++ b/crates/glua_code_analysis/src/compilation/analyzer/local_inference/mod.rs @@ -19,8 +19,8 @@ use crate::{ SignatureReturnStatus, compilation::analyzer::AnalyzeContext, semantic::{ - expr_may_have_condition_narrowing, infer_bind_value_type, infer_expr, - infer_true_condition_narrowing, resolve_dynamic_field_member, + infer_bind_value_type, infer_expr, infer_true_condition_narrowing, + resolve_dynamic_field_member, }, }; @@ -32,36 +32,38 @@ pub(super) fn stabilize_unknown_locals( ) -> bool { let _profile = crate::profile::Profile::cond_new("local inference stabilize", context.tree_list.len() > 1); - let mut candidates = context - .tree_list - .iter() - .filter_map(|tree| { - db.get_reference_index() - .get_decl_references_map(&tree.file_id) - .map(|references| (tree.file_id, references.clone())) - }) - .flat_map(|(file_id, references)| { - references - .into_iter() - .map(move |(decl_id, references)| (file_id, decl_id, references)) - }) - .filter(|(_, decl_id, _)| { - db.get_decl_index() + // Two index lookups decide this, so they run before the reference list is + // cloned. + let mut candidates = Vec::new(); + for tree in &context.tree_list { + let file_id = tree.file_id; + let Some(references) = db.get_reference_index().get_decl_references_map(&file_id) else { + continue; + }; + for (decl_id, decl_references) in references { + let is_local = db + .get_decl_index() .get_decl(decl_id) - .is_some_and(|decl| matches!(decl.extra, crate::LuaDeclExtra::Local { .. })) - && db - .get_type_index() - .get_type_cache(&(*decl_id).into()) - // `never` is the bottom of the same uninformative band - // as `unknown` (see `LuaTypeCache::supersedes`), and it - // is what an initialiser resolves to when the member it - // reads is not in the index *yet*. - .is_none_or(|cache| { - cache.is_infer() - && matches!(cache.as_type(), LuaType::Unknown | LuaType::Never) - }) - }) - .collect::>(); + .is_some_and(|decl| matches!(decl.extra, crate::LuaDeclExtra::Local { .. })); + if !is_local { + continue; + } + // `never` is the bottom of the same uninformative band as + // `unknown` (see `LuaTypeCache::supersedes`), and it is what an + // initialiser resolves to when the member it reads is not in the + // index *yet*. + let uninformative = db + .get_type_index() + .get_type_cache(&(*decl_id).into()) + .is_none_or(|cache| { + cache.is_infer() && matches!(cache.as_type(), LuaType::Unknown | LuaType::Never) + }); + if !uninformative { + continue; + } + candidates.push((file_id, *decl_id, decl_references.clone())); + } + } candidates.sort_by_key(|(_, decl_id, _)| (decl_id.file_id, decl_id.position)); let mut evidence_by_node = @@ -234,6 +236,32 @@ fn compare_unguarded_child_candidates( .then_with(|| left.stable_cmp(right)) } +fn in_filed_order(value: &InFiled) -> (u32, u32, u32) { + let range = value.value.get_range(); + ( + value.file_id.id, + u32::from(range.start()), + u32::from(range.end()), + ) +} + +/// Declared member types already looked up in this pass. `infer_raw_member_type` +/// reads the member index, so the answer only depends on the pair, and a member +/// path repeats at every use of the declaration it hangs off. +type DeclaredPathBases = FxHashMap<(LuaType, LuaMemberKey), Option<(LuaType, LuaTypeDeclId)>>; + +/// What every member path rooted at one declaration shares, plus the two +/// accumulators the whole pass shares. +struct MemberPathEvidence<'a> { + file_id: crate::FileId, + root_decl_id: crate::LuaDeclId, + declared_root_type: Option<&'a LuaType>, + direct_subtype_members: &'a DirectSubtypeMembers, + candidate_members: &'a FxHashSet, + declared_path_bases: &'a mut DeclaredPathBases, + scores: &'a mut HashMap, +} + /// The evidence sites of one declaration, with whether each sits inside a /// `return`. pub(super) type UnguardedChildSiteCache = @@ -261,6 +289,7 @@ pub(super) fn stabilize_unguarded_children( HashMap::<(LuaDefinitionId, crate::LuaTypeDeclId), InFiled>::new( ); let mut initializer_refinements = HashMap::::new(); + let mut declared_path_bases = DeclaredPathBases::default(); let subtype_index_start = profile.as_ref().map(|_| std::time::Instant::now()); let direct_subtype_members = precompute_direct_subtype_members(db); let nested_candidate_members = direct_subtype_members @@ -278,10 +307,10 @@ pub(super) fn stabilize_unguarded_children( .map(|tree| tree.file_id) .collect::>(); for file_id in file_ids { - let Some(references) = db + let Some(decl_ids) = db .get_reference_index() .get_decl_references_map(&file_id) - .cloned() + .map(|references| references.keys().copied().collect::>()) else { continue; }; @@ -294,38 +323,45 @@ pub(super) fn stabilize_unguarded_children( }; let flow_tree = db.get_flow_index().get_flow_tree(&file_id); - for (decl_id, references) in references { + for decl_id in decl_ids { // The syntactic prerequisites for evidence — a read reference // that is the prefix of an index expression — are pure tree // lookups, while `declaration_base_type` infers a parameter's - // type. + // type. Only a declaration missing from the cache reads its + // reference list, so a second pass re-reads none of them. let all_sites = site_cache.entry((file_id, decl_id)).or_insert_with(|| { - references - .cells - .iter() - .filter(|cell| !cell.is_write) - .filter_map(|cell| { - let name_expr = root - .covering_element(cell.range) - .ancestors() - .find_map(LuaNameExpr::cast) - .filter(|name| name.get_range() == cell.range)?; - let index_expr = name_expr - .syntax() - .ancestors() - .find_map(LuaIndexExpr::cast) - .filter(|index| { - index - .get_prefix_expr() - .is_some_and(|prefix| prefix.syntax() == name_expr.syntax()) - })?; - let in_return = index_expr - .syntax() - .ancestors() - .any(|node| LuaReturnStat::cast(node).is_some()); - Some((name_expr, index_expr, in_return)) + db.get_reference_index() + .get_decl_references_map(&file_id) + .and_then(|references| references.get(&decl_id)) + .map(|references| { + references + .cells + .iter() + .filter(|cell| !cell.is_write) + .filter_map(|cell| { + let name_expr = root + .covering_element(cell.range) + .ancestors() + .find_map(LuaNameExpr::cast) + .filter(|name| name.get_range() == cell.range)?; + let index_expr = name_expr + .syntax() + .ancestors() + .find_map(LuaIndexExpr::cast) + .filter(|index| { + index.get_prefix_expr().is_some_and(|prefix| { + prefix.syntax() == name_expr.syntax() + }) + })?; + let in_return = index_expr + .syntax() + .ancestors() + .any(|node| LuaReturnStat::cast(node).is_some()); + Some((name_expr, index_expr, in_return)) + }) + .collect::>() }) - .collect::>() + .unwrap_or_default() }); let sites = all_sites .iter() @@ -336,7 +372,36 @@ pub(super) fn stabilize_unguarded_children( continue; } - let Some(base_type) = declaration_base_type(db, context, decl_id) else { + let base_type = declaration_base_type(db, context, decl_id); + + // A member path reached from this declaration, e.g. `self.Owner` in + // `self.Owner:ConCommand()`. Climbed from the sites already scanned + // rather than from a walk of the file, so each level of the path + // costs one parent hop. + let mut evidence = MemberPathEvidence { + file_id, + root_decl_id: decl_id, + declared_root_type: base_type.as_ref(), + direct_subtype_members: &direct_subtype_members, + candidate_members: &nested_candidate_members, + declared_path_bases: &mut declared_path_bases, + scores: &mut nested_scores, + }; + for (_, receiver) in &sites { + let mut receiver = receiver.clone(); + while let Some(index_expr) = member_path_parent(&receiver) { + collect_member_path_unguarded_child_evidence( + db, + context, + &mut evidence, + &receiver, + &index_expr, + ); + receiver = index_expr; + } + } + + let Some(base_type) = base_type else { continue; }; let Some(base_id) = unguarded_child_base_id(&base_type) else { @@ -499,22 +564,6 @@ pub(super) fn stabilize_unguarded_children( } } } - - if db - .get_call_site_param_index() - .has_concrete_structural_callback_params(file_id) - { - collect_nested_unguarded_child_evidence( - db, - context, - file_id, - &root, - only_return_evidence, - &direct_subtype_members, - &nested_candidate_members, - &mut nested_scores, - ); - } } if let (Some(profile), Some(start)) = (&mut profile, reference_scan_start) { profile.reference_scan = start.elapsed(); @@ -527,6 +576,10 @@ pub(super) fn stabilize_unguarded_children( let mut updates = Vec::new(); let mut update_sources = Vec::new(); + // Both score maps are hashed, and their entries decide the order facts are + // published and reported in. + let mut scores = scores.into_iter().collect::>(); + scores.sort_by(|(left, _), (right, _)| left.stable_cmp(right)); for (definition, candidates) in scores { let found_type = candidates.parent_type; let candidates = candidates.children; @@ -603,7 +656,9 @@ pub(super) fn stabilize_unguarded_children( )); } - for (_, candidates) in nested_scores { + let mut nested_scores = nested_scores.into_values().collect::>(); + nested_scores.sort_by_key(|candidates| in_filed_order(&candidates.source)); + for candidates in nested_scores { let found_type = candidates.parent_type; let Some(max_score) = candidates.children.values().map(FxHashSet::len).max() else { continue; @@ -638,8 +693,9 @@ pub(super) fn stabilize_unguarded_children( support.sort_by(LuaInferenceNodeId::stable_cmp); support.dedup(); update_sources.push(candidates.source.clone()); - let nodes = candidates - .receivers + let mut receivers = candidates.receivers.into_iter().collect::>(); + receivers.sort_by_key(in_filed_order); + let nodes = receivers .into_iter() .map(|receiver| LuaInferenceNodeId::TypeOwner(crate::LuaTypeOwner::SyntaxId(receiver))); let event_node = @@ -716,104 +772,173 @@ pub(super) fn stabilize_unguarded_children( } } -fn collect_nested_unguarded_child_evidence( +/// The next level of a member path, e.g. `self.Owner:ConCommand` from +/// `self.Owner`. +fn member_path_parent(receiver: &LuaIndexExpr) -> Option { + receiver + .syntax() + .parent() + .and_then(LuaIndexExpr::cast) + .filter(|parent| { + parent + .get_prefix_expr() + .is_some_and(|prefix| prefix.syntax() == receiver.syntax()) + }) +} + +/// Records evidence for one level of a member path, e.g. `receiver` = +/// `self.Owner` and `index_expr` = `self.Owner:ConCommand`. A deeper path calls +/// this once per level. +fn collect_member_path_unguarded_child_evidence( db: &crate::DbIndex, context: &mut AnalyzeContext, - file_id: crate::FileId, - root: &glua_parser::LuaSyntaxNode, - only_return_evidence: bool, - direct_subtype_members: &DirectSubtypeMembers, - candidate_members: &FxHashSet, - scores: &mut HashMap, + evidence: &mut MemberPathEvidence<'_>, + receiver: &LuaIndexExpr, + index_expr: &LuaIndexExpr, ) { - let mut callback_roots = FxHashMap::default(); - for index_expr in root.descendants().filter_map(LuaIndexExpr::cast) { - let Some(LuaExpr::IndexExpr(receiver)) = index_expr.get_prefix_expr() else { - continue; - }; - if only_return_evidence - && !index_expr - .syntax() - .ancestors() - .any(|node| LuaReturnStat::cast(node).is_some()) - { - continue; - } - if is_assignment_target(&index_expr) { - continue; - } - if is_condition_evidence(&index_expr) { - continue; - } - - let cache = context.infer_manager.get_infer_cache(file_id); - let Some(root_decl_id) = nested_receiver_root_decl_id(db, file_id, &receiver) else { - continue; - }; - let callback_inferred = *callback_roots - .entry(root_decl_id) - .or_insert_with(|| is_callback_inferred_structural_root(db, root_decl_id)); - if !callback_inferred { - continue; - } - let Some(index_key) = index_expr.get_index_key() else { - continue; - }; - if LuaMemberKey::index_key_is_dynamic(db, cache, &index_key) { - continue; - } - let Ok(member_key) = LuaMemberKey::from_index_key(db, cache, &index_key) else { - continue; - }; - if !candidate_members.contains(&member_key) { - continue; - } - if !expr_may_have_condition_narrowing(db, cache, LuaExpr::IndexExpr(receiver.clone())) { - continue; + let file_id = evidence.file_id; + let cache = context.infer_manager.get_infer_cache(file_id); + let Some(index_key) = index_expr.get_index_key() else { + return; + }; + if LuaMemberKey::index_key_is_dynamic(db, cache, &index_key) { + return; + } + let Ok(member_key) = LuaMemberKey::from_index_key(db, cache, &index_key) else { + return; + }; + if !evidence.candidate_members.contains(&member_key) { + return; + } + if is_assignment_target(index_expr) { + return; + } + if is_condition_evidence(index_expr) { + return; + } + let Some(receiver_key) = receiver + .get_index_key() + .and_then(|key| LuaMemberKey::from_index_key(db, cache, &key).ok()) + else { + return; + }; + let Some(prefix) = receiver.get_prefix_expr() else { + return; + }; + // At the first level the prefix is the root declaration, whose type is + // already resolved for its whole reference set; reading it off the name + // expression would run a flow walk per use, and the child lookup below rules + // most uses out before narrowing matters. Deeper levels have no such + // shortcut and infer their prefix, which the narrowed type below then reads + // back from the same cache. + let prefix_type = match evidence.declared_root_type { + Some(typ) if matches!(prefix, LuaExpr::NameExpr(_)) => typ.clone(), + _ => match infer_expr(db, cache, prefix) { + Ok(typ) => typ, + Err(_) => return, + }, + }; + // The path is only worth narrowing if the thing it is read from is itself + // settled: a structural object, or a declared class. + if !prefix_type.contains_object_type() + && unguarded_child_base_id(&prefix_type).is_none() + && !prefix_type.is_unknown() + { + return; + } + // `infer_raw_member_type` answers from the member and type indexes alone: it + // takes no file and no offset, and the cache it is handed only memoises a + // lookup keyed by the same type and member. The pair is therefore the whole + // key, and the indexes it reads do not change until this pass publishes. + let declared_base = match evidence + .declared_path_bases + .get(&(prefix_type.clone(), receiver_key.clone())) + { + Some(hit) => hit.clone(), + None => { + let resolved = crate::semantic::infer_raw_member_type_with_cache( + db, + cache, + &prefix_type, + &receiver_key, + ) + .ok() + .and_then(|declared| { + unguarded_child_base_id(&declared).map(|base_id| (declared, base_id)) + }); + evidence + .declared_path_bases + .insert((prefix_type, receiver_key), resolved.clone()); + resolved } - let Some(target) = nested_unguarded_child_target(db, cache, &receiver, root_decl_id) else { - continue; - }; - let Some(receiver_prefix) = receiver.get_prefix_expr() else { - continue; - }; - let receiver_prefix_type = infer_expr(db, cache, receiver_prefix).ok(); - let allow_stable_path = receiver_prefix_type - .as_ref() - .is_some_and(LuaType::contains_object_type); - let allow_opaque_path = receiver_prefix_type - .as_ref() - .is_some_and(LuaType::is_unknown); - if !allow_stable_path && !allow_opaque_path { - continue; - }; - let current = - infer_expr(db, cache, LuaExpr::IndexExpr(receiver.clone())).unwrap_or(LuaType::Unknown); - let Some(base_id) = unguarded_child_base_id(¤t) else { - continue; - }; - let Some(children) = direct_subtype_members - .get(&base_id) - .and_then(|members| members.get(&member_key)) - else { - continue; - }; - if is_matching_short_circuit_guard(db, cache, &index_expr) { - continue; + }; + let (base_id, current) = match declared_base { + // The declaration already names the base, so the child lookup can rule + // the use out before its narrowed type is worth computing. + Some((declared, base_id)) => { + if evidence + .direct_subtype_members + .get(&base_id) + .and_then(|members| members.get(&member_key)) + .is_none() + { + return; + } + let current = infer_expr(db, cache, LuaExpr::IndexExpr(receiver.clone())) + .unwrap_or(LuaType::Unknown); + // A real flow guard wins, exactly as it does for a plain declaration. + if !unguarded_child_current_matches_base(¤t, &declared, &base_id) { + return; + } + (base_id, current) } - if type_has_visible_member_at_use( - db, - context, - &LuaType::Ref(base_id), - &member_key, - file_id, - index_expr.get_position(), - ) { - continue; + // Nothing is declared for this path, so a guard is the only thing that + // could have given it a base. A base read back from this pass's own + // published narrowing is not one: scoring against it would descend one + // more level of the class tree per round, so the receiver's pre-pass + // type is used instead. + None => { + let current = + published_unguarded_child_base(db, file_id, receiver).unwrap_or_else(|| { + infer_expr(db, cache, LuaExpr::IndexExpr(receiver.clone())) + .unwrap_or(LuaType::Unknown) + }); + let Some(base_id) = unguarded_child_base_id(¤t) else { + return; + }; + (base_id, current) } - let source = InFiled::new(file_id, index_expr.get_syntax_id()); - let receiver = InFiled::new(file_id, receiver.get_syntax_id()); - let candidates = scores + }; + let Some(children) = evidence + .direct_subtype_members + .get(&base_id) + .and_then(|members| members.get(&member_key)) + else { + return; + }; + if is_matching_short_circuit_guard(db, cache, index_expr) { + return; + } + if type_has_visible_member_at_use( + db, + context, + &LuaType::Ref(base_id), + &member_key, + file_id, + index_expr.get_position(), + ) { + return; + } + let cache = context.infer_manager.get_infer_cache(file_id); + let Some(target) = nested_unguarded_child_target(db, cache, receiver, evidence.root_decl_id) + else { + return; + }; + let source = InFiled::new(file_id, index_expr.get_syntax_id()); + let receiver = InFiled::new(file_id, receiver.get_syntax_id()); + let candidates = + evidence + .scores .entry(target) .or_insert_with(|| NestedUnguardedChildCandidates { parent_type: current.clone(), @@ -821,58 +946,39 @@ fn collect_nested_unguarded_child_evidence( receivers: FxHashSet::default(), source: source.clone(), }); - if candidates.parent_type != current { - continue; - } - candidates.receivers.insert(receiver); - if source.value.get_range().start() < candidates.source.value.get_range().start() { - candidates.source = source; - } - for child_id in children { - candidates - .children - .entry(child_id.clone()) - .or_default() - .insert(member_key.clone()); - } + if candidates.parent_type != current { + return; + } + candidates.receivers.insert(receiver); + if source.value.get_range().start() < candidates.source.value.get_range().start() { + candidates.source = source; + } + for child_id in children { + candidates + .children + .entry(child_id.clone()) + .or_default() + .insert(member_key.clone()); } } -fn nested_receiver_root_decl_id( +/// What a receiver was before this pass narrowed it, when it did. Recorded on +/// the published step, so it survives the fact that replaced it. +fn published_unguarded_child_base( db: &crate::DbIndex, file_id: crate::FileId, receiver: &LuaIndexExpr, -) -> Option { - let mut current = receiver.clone(); - loop { - match current.get_prefix_expr()? { - LuaExpr::IndexExpr(parent) => current = parent, - LuaExpr::NameExpr(name) => { - return db - .get_reference_index() - .get_local_reference(&file_id)? - .get_decl_id(&name.get_range()); - } - _ => return None, - } - } -} - -fn is_callback_inferred_structural_root( - db: &crate::DbIndex, - root_decl_id: crate::LuaDeclId, -) -> bool { - let Some(decl) = db.get_decl_index().get_decl(&root_decl_id) else { - return false; - }; - let crate::LuaDeclExtra::Param { - idx, signature_id, .. - } = &decl.extra - else { - return false; - }; - db.get_call_site_param_index() - .is_concrete_structural_callback_param(signature_id, *idx) +) -> Option { + let node = LuaInferenceNodeId::TypeOwner(crate::LuaTypeOwner::SyntaxId(InFiled::new( + file_id, + receiver.get_syntax_id(), + ))); + let fact = db.get_inference_fact(&node)?; + let step = fact + .provenance() + .iter() + .find(|step| step.event.kind == LuaInferenceProvenanceKind::UnguardedChild)?; + step.found_type.as_deref().cloned() } fn nested_unguarded_child_target( diff --git a/crates/glua_code_analysis/src/compilation/analyzer/lua/call.rs b/crates/glua_code_analysis/src/compilation/analyzer/lua/call.rs index cd21b6b54..db3047eb2 100644 --- a/crates/glua_code_analysis/src/compilation/analyzer/lua/call.rs +++ b/crates/glua_code_analysis/src/compilation/analyzer/lua/call.rs @@ -210,7 +210,7 @@ fn add_direct_special_call_var_expr( } matcher .access_paths - .entry(access_path) + .entry(access_path.to_string()) .or_default() .push(binding); } @@ -1468,9 +1468,9 @@ fn extract_literal_or_name(expr: &LuaExpr) -> Option { LuaLiteralToken::Nil(_) => Some(GmodClassCallLiteral::Nil), _ => None, }, - LuaExpr::NameExpr(name_expr) => { - name_expr.get_name_text().map(GmodClassCallLiteral::NameRef) - } + LuaExpr::NameExpr(name_expr) => name_expr + .get_name_text() + .map(|name| GmodClassCallLiteral::NameRef(name.to_string())), _ => None, } } diff --git a/crates/glua_code_analysis/src/compilation/analyzer/lua/closure.rs b/crates/glua_code_analysis/src/compilation/analyzer/lua/closure.rs index 38425a381..389cb8311 100644 --- a/crates/glua_code_analysis/src/compilation/analyzer/lua/closure.rs +++ b/crates/glua_code_analysis/src/compilation/analyzer/lua/closure.rs @@ -21,6 +21,31 @@ use crate::{ use super::{LuaAnalyzer, LuaReturnPoint, func_body::analyze_func_body_returns}; +/// The closure's own `return` statements — those not belonging to a closure +/// nested inside it. Cached per closure. +fn closure_own_return_stats( + analyzer: &mut LuaAnalyzer, + closure: &LuaClosureExpr, + block: &LuaBlock, +) -> std::rc::Rc> { + let key = closure.get_syntax_id(); + if let Some(cached) = analyzer.closure_own_returns_cache.get(&key) { + return cached.clone(); + } + let returns = std::rc::Rc::new( + block + .descendants::() + .filter(|return_stat| { + return_stat.ancestors::().next().as_ref() == Some(closure) + }) + .collect::>(), + ); + analyzer + .closure_own_returns_cache + .insert(key, returns.clone()); + returns +} + pub fn analyze_closure(analyzer: &mut LuaAnalyzer, closure: LuaClosureExpr) -> Option<()> { let signature_id = LuaSignatureId::from_closure(analyzer.file_id, &closure); @@ -58,13 +83,7 @@ fn analyze_direct_param_return_alias( // unshadowed and unassigned parameter, with no nested closure that could // capture and replace it before the return executes. if block.descendants::().next().is_some() - || block - .descendants::() - .filter(|returned| { - returned.ancestors::().next().as_ref() == Some(closure) - }) - .count() - != 1 + || closure_own_return_stats(analyzer, closure, &block).len() != 1 { return Some(()); } @@ -154,12 +173,7 @@ fn analyze_class_name_param_return_alias( } let block = closure.get_block()?; - if block - .descendants::() - .filter(|returned| returned.ancestors::().next().as_ref() == Some(closure)) - .count() - != 1 - { + if closure_own_return_stats(analyzer, closure, &block).len() != 1 { return Some(()); } let LuaStat::ReturnStat(return_stat) = block.get_stats().last()? else { @@ -641,12 +655,7 @@ fn falsy_param_nil_free_return_slot( return None; } - let return_stats = block - .descendants::() - .filter(|return_stat| { - return_stat.ancestors::().next().as_ref() == Some(closure) - }) - .collect::>(); + let return_stats = closure_own_return_stats(analyzer, closure, block); let reachable_returns = return_stats .iter() .filter(|return_stat| !return_is_inside_stat(return_stat, if_stat)) @@ -981,11 +990,8 @@ fn non_guard_returns_are_proven_non_nil( guard_return_ranges: &[TextRange], ) -> bool { let mut saw_non_guard_return = false; - let return_stats = block.descendants::().collect::>(); - for return_stat in return_stats { - if return_stat.ancestors::().next().as_ref() != Some(closure) { - continue; - } + let return_stats = closure_own_return_stats(analyzer, closure, block); + for return_stat in return_stats.iter() { if guard_return_ranges.contains(&return_stat.get_range()) { continue; } diff --git a/crates/glua_code_analysis/src/compilation/analyzer/lua/for_range_stat.rs b/crates/glua_code_analysis/src/compilation/analyzer/lua/for_range_stat.rs index eff9cb3b9..782be2eb6 100644 --- a/crates/glua_code_analysis/src/compilation/analyzer/lua/for_range_stat.rs +++ b/crates/glua_code_analysis/src/compilation/analyzer/lua/for_range_stat.rs @@ -247,6 +247,16 @@ fn try_infer_pairs_iter_types_from_table_members( } let table_type = infer_expr(db, cache, table_arg)?; + if matches!(table_type, LuaType::Global) { + // The global table has no enumerable answer: its member types are the very + // thing analysis is computing, and a loop over it can declare further + // globals whose types are then part of the same union. Any snapshot is a + // record of how far inference had progressed, not a fact about the program. + return Ok(Some(VariadicType::Multi(vec![ + LuaType::String, + LuaType::Any, + ]))); + } if let LuaType::TableOf(inner) = &table_type { // Keep the value as T[K] instead of materializing every member type. Large // scripted-class hierarchies can contain hundreds of callable members. @@ -273,12 +283,29 @@ fn try_infer_pairs_iter_types_from_table_members( .map(|(key, member_infos)| (key.clone(), member_infos.clone())) .collect::>(); member_entries.sort_by_key(|(key, _)| member_key_stable_key(key)); + + // A dynamic key aliases every access whose key infers to the same type + // rather than naming a member, so once one is present the literal keys + // beside it are a sample of the indices some file happened to write, not + // the table's key domain. Which samples are in the map depends on how far + // inference had progressed, so the keys are reported by kind. + let keys_are_sampled = member_entries + .iter() + .any(|(key, _)| matches!(key, LuaMemberKey::ExprType(_))); + for (key, member_infos) in member_entries { - let key_type = match key { - LuaMemberKey::Integer(i) => LuaType::IntegerConst(i), - LuaMemberKey::Name(name) => LuaType::StringConst(name.into()), - LuaMemberKey::ExprType(typ) => typ, - LuaMemberKey::None => continue, + let key_type = if keys_are_sampled { + match table_projection_member_key_type(&key) { + Some(typ) => typ, + None => continue, + } + } else { + match key { + LuaMemberKey::Integer(i) => LuaType::IntegerConst(i), + LuaMemberKey::Name(name) => LuaType::StringConst(name.into()), + LuaMemberKey::ExprType(typ) => typ, + LuaMemberKey::None => continue, + } }; keys.push(key_type); diff --git a/crates/glua_code_analysis/src/compilation/analyzer/lua/member_write_policy/collection.rs b/crates/glua_code_analysis/src/compilation/analyzer/lua/member_write_policy/collection.rs index fdea5ce29..0386de204 100644 --- a/crates/glua_code_analysis/src/compilation/analyzer/lua/member_write_policy/collection.rs +++ b/crates/glua_code_analysis/src/compilation/analyzer/lua/member_write_policy/collection.rs @@ -540,7 +540,9 @@ pub(in crate::compilation::analyzer::lua) fn is_collection_append_write( Some(expr_access_path(&prefix_expr) == expr_access_path(&len_expr)) } -pub(in crate::compilation::analyzer::lua) fn expr_access_path(expr: &LuaExpr) -> Option { +pub(in crate::compilation::analyzer::lua) fn expr_access_path( + expr: &LuaExpr, +) -> Option { match expr { LuaExpr::NameExpr(name_expr) => name_expr.get_access_path(), LuaExpr::IndexExpr(index_expr) => index_expr.get_access_path(), diff --git a/crates/glua_code_analysis/src/compilation/analyzer/lua/metatable.rs b/crates/glua_code_analysis/src/compilation/analyzer/lua/metatable.rs index 87ac97234..8a05532bd 100644 --- a/crates/glua_code_analysis/src/compilation/analyzer/lua/metatable.rs +++ b/crates/glua_code_analysis/src/compilation/analyzer/lua/metatable.rs @@ -102,7 +102,7 @@ fn setmetatable_factory_binding( file_id: analyzer.file_id, table_range, metatable_range, - local_name: table_name.get_name_text()?.into(), + local_name: table_name.get_name_text()?, call_position: call_expr.get_position(), function_scope, }) diff --git a/crates/glua_code_analysis/src/compilation/analyzer/lua/mod.rs b/crates/glua_code_analysis/src/compilation/analyzer/lua/mod.rs index 5c373ec48..619dad30f 100644 --- a/crates/glua_code_analysis/src/compilation/analyzer/lua/mod.rs +++ b/crates/glua_code_analysis/src/compilation/analyzer/lua/mod.rs @@ -34,7 +34,8 @@ use stats::{ pub(in crate::compilation::analyzer) use stats::{ get_widened_member_assignment_type, has_multiple_distinct_index_expr_member_owners, is_guarded_table_assignment_index_expr, is_guarded_table_assignment_member, - preserve_guarded_table_assignment_members, + mark_resolved_member_assignment, preserve_guarded_table_assignment_members, + record_resolved_member_assignment_contribution, }; use log::info; @@ -78,7 +79,7 @@ impl AnalysisPipeline for LuaAnalysisPipeline { }; let file_dependency = db.get_file_dependencies_index().get_file_dependencies(); - let order = file_dependency.get_best_analysis_order(&file_ids, &context.metas); + let levels = file_dependency.get_analysis_levels(&file_ids, &context.metas); let stderr_profile_enabled = std::env::var_os("GLUALS_PROFILE").is_some(); let slow_log_enabled = log::log_enabled!(log::Level::Info) || stderr_profile_enabled; let node_profile_enabled = stderr_profile_enabled; @@ -86,69 +87,92 @@ impl AnalysisPipeline for LuaAnalysisPipeline { let mut workspace_profile = node_profile_enabled.then(LuaAnalyzeProfile::default); let mut slow_file_summary = slow_log_enabled.then(SlowLuaAnalyzeSummary::default); let mut file_count: usize = 0; - for file_id in order { - if let Some(root) = tree_map.get(&file_id) { - let file_start = slow_log_enabled.then(Instant::now); - let is_scripted = scripted_scope_files.contains(&file_id); - let mut analyzer = LuaAnalyzer::new( - db, - file_id, - context, - gmod_enabled, - is_scripted, - &special_call_direct_matcher, - ); - let mut profile = node_profile_enabled.then(LuaAnalyzeProfile::default); - for node in root.descendants::() { - if let Some(profile) = profile.as_mut() { - let kind = lua_ast_profile_kind(&node); - let node_start = Instant::now(); - analyze_node(&mut analyzer, node); - profile.record(kind, node_start.elapsed()); - } else { - analyze_node(&mut analyzer, node); - } - } - if let (Some(workspace_profile), Some(profile)) = - (workspace_profile.as_mut(), profile.as_ref()) - { - workspace_profile.merge(profile); + let mut level_shape = node_profile_enabled.then(LevelShape::default); + // Reported inside the level rather than only at level boundaries: the + // widest level can hold most of the workspace. + let progress_total = file_ids.len(); + let progress_step = if crate::progress::is_active() && progress_total > 1 { + (progress_total / 50).max(1) + } else { + 0 + }; + for level in levels { + if let Some(shape) = level_shape.as_mut() { + shape.begin_level(level.len()); + } + for file_id in level { + if progress_step != 0 && file_count.is_multiple_of(progress_step) { + crate::progress::advance_current_phase(file_count, progress_total, "files"); } - analyze_chunk_return(&mut analyzer, root.clone()); - flush_pending_dynamic_key_collection_widenings(&mut analyzer); - file_count += 1; - if let Some(file_start) = file_start { - let file_elapsed = file_start.elapsed(); - if let Some(summary) = slow_file_summary.as_mut() { - summary.record(file_id, file_elapsed); + if let Some(root) = tree_map.get(&file_id) { + let file_start = slow_log_enabled.then(Instant::now); + let is_scripted = scripted_scope_files.contains(&file_id); + let mut analyzer = LuaAnalyzer::new( + db, + file_id, + context, + gmod_enabled, + is_scripted, + &special_call_direct_matcher, + ); + let mut profile = node_profile_enabled.then(LuaAnalyzeProfile::default); + for node in root.descendants::() { + if let Some(profile) = profile.as_mut() { + let kind = lua_ast_profile_kind(&node); + let node_start = Instant::now(); + analyze_node(&mut analyzer, node); + profile.record(kind, node_start.elapsed()); + } else { + analyze_node(&mut analyzer, node); + } } - - // Detailed per-file logging is intentionally reserved for explicit profiling. - // Info logging can be enabled in normal server sessions, and logging every - // >1ms file turns large workspace analysis into a log-I/O hotspot. - let should_log_file = if stderr_profile_enabled { - file_elapsed.as_millis() > 1 - } else { - file_elapsed >= Duration::from_millis(50) - }; - if should_log_file { - let path = db - .get_vfs() - .get_uri(&file_id) - .map(|u| u.to_string()) - .unwrap_or_else(|| format!("{:?}", file_id)); - info!("lua analyze slow file: {} cost {:?}", path, file_elapsed); - if let Some(profile) = profile.as_ref() { - profile.log_slow_file(&path); + if let (Some(workspace_profile), Some(profile)) = + (workspace_profile.as_mut(), profile.as_ref()) + { + workspace_profile.merge(profile); + } + analyze_chunk_return(&mut analyzer, root.clone()); + flush_pending_dynamic_key_collection_widenings(&mut analyzer); + file_count += 1; + if let Some(file_start) = file_start { + let file_elapsed = file_start.elapsed(); + if let Some(summary) = slow_file_summary.as_mut() { + summary.record(file_id, file_elapsed); } - if stderr_profile_enabled { - eprintln!("lua analyze slow file: {} cost {:?}", path, file_elapsed); + if let Some(shape) = level_shape.as_mut() { + shape.record_file(file_elapsed); + } + + // Detailed per-file logging is intentionally reserved for explicit profiling. + // Info logging can be enabled in normal server sessions, and logging every + // >1ms file turns large workspace analysis into a log-I/O hotspot. + let should_log_file = if stderr_profile_enabled { + file_elapsed.as_millis() > 1 + } else { + file_elapsed >= Duration::from_millis(50) + }; + if should_log_file { + let path = db + .get_vfs() + .get_uri(&file_id) + .map(|u| u.to_string()) + .unwrap_or_else(|| format!("{:?}", file_id)); + info!("lua analyze slow file: {} cost {:?}", path, file_elapsed); if let Some(profile) = profile.as_ref() { + profile.log_slow_file(&path); + } + if stderr_profile_enabled { eprintln!( - "lua analyze slow file node profile: {} [{}]", - path, - profile.summary(8) + "lua analyze slow file: {} cost {:?}", + path, file_elapsed ); + if let Some(profile) = profile.as_ref() { + eprintln!( + "lua analyze slow file node profile: {} [{}]", + path, + profile.summary(8) + ); + } } } } @@ -178,11 +202,81 @@ impl AnalysisPipeline for LuaAnalysisPipeline { workspace_profile.summary(8) ); } + if let Some(level_shape) = level_shape.as_ref() { + eprintln!("lua analyze level shape: {}", level_shape.summary()); + } } } } } +/// Measures how much of `lua analyze` could overlap if each dependency level ran +/// concurrently: the critical path is the sum of each level's slowest file. +/// Profiling only (`GLUALS_PROFILE=1`). +#[derive(Default)] +struct LevelShape { + levels: usize, + widths: Vec, + maxes: Vec, + total: Duration, + critical_path: Duration, + level_max: Duration, +} + +impl LevelShape { + fn begin_level(&mut self, width: usize) { + let previous = std::mem::take(&mut self.level_max); + self.critical_path += previous; + if self.levels > 0 { + self.maxes.push(previous); + } + self.levels += 1; + self.widths.push(width); + } + + fn record_file(&mut self, elapsed: Duration) { + self.total += elapsed; + self.level_max = self.level_max.max(elapsed); + } + + fn summary(&self) -> String { + let critical_path = self.critical_path + self.level_max; + let widest = self.widths.iter().copied().max().unwrap_or(0); + let files: usize = self.widths.iter().sum(); + let speedup = if critical_path.is_zero() { + 0.0 + } else { + self.total.as_secs_f64() / critical_path.as_secs_f64() + }; + let mut heaviest: Vec<(usize, usize, Duration)> = self + .widths + .iter() + .zip(self.maxes.iter().chain(std::iter::once(&self.level_max))) + .enumerate() + .map(|(level, (&width, &max))| (level, width, max)) + .collect(); + heaviest.sort_unstable_by_key(|&(_, _, max)| std::cmp::Reverse(max)); + let heaviest: Vec = heaviest + .iter() + .take(6) + .map(|(level, width, max)| format!("L{level}(w={width}) {max:?}")) + .collect(); + format!( + "{} levels over {} files (widest {}, mean width {:.1}); \ + sequential {:?}, critical path {:?}, ideal speedup {:.1}x; \ + heaviest levels: {}", + self.levels, + files, + widest, + files as f64 / self.levels.max(1) as f64, + self.total, + critical_path, + speedup, + heaviest.join(", "), + ) + } +} + #[derive(Default)] struct SlowLuaAnalyzeSummary { files_over_1ms: usize, @@ -334,7 +428,10 @@ struct LuaAnalyzer<'a> { pending_dynamic_key_collection_widenings: FxHashMap, guarded_table_assignment_type_cache: FxHashMap, direct_local_table_member_owner_cache: FxHashMap>, - literal_index_member_owner_cache: FxHashMap, + literal_index_member_owner_cache: FxHashMap, + /// A closure's own `return` statements (excluding nested closures'). + closure_own_returns_cache: + FxHashMap>>, } impl LuaAnalyzer<'_> { @@ -359,6 +456,7 @@ impl LuaAnalyzer<'_> { guarded_table_assignment_type_cache: FxHashMap::default(), direct_local_table_member_owner_cache: FxHashMap::default(), literal_index_member_owner_cache: FxHashMap::default(), + closure_own_returns_cache: FxHashMap::default(), } } diff --git a/crates/glua_code_analysis/src/compilation/analyzer/lua/stats.rs b/crates/glua_code_analysis/src/compilation/analyzer/lua/stats.rs index 31c76dcc1..afdcc36d1 100644 --- a/crates/glua_code_analysis/src/compilation/analyzer/lua/stats.rs +++ b/crates/glua_code_analysis/src/compilation/analyzer/lua/stats.rs @@ -13,7 +13,7 @@ use crate::{ }, db_index::{ LuaDeclId, LuaMember, LuaMemberFeature, LuaMemberId, LuaMemberOwner, LuaType, - MemberAssignmentContribution, + MemberAssignmentContribution, member_id_sort_key, }, semantic::{merge_open_table_types, remove_false_or_nil}, }; @@ -1414,7 +1414,7 @@ fn should_defer_none_infer_expr(expr: &LuaExpr) -> bool { } fn is_call_or_index_expr(expr: &LuaExpr) -> bool { - matches!(expr, LuaExpr::CallExpr(_) | LuaExpr::IndexExpr(_)) + crate::compilation::analyzer::initializer_reads_through_call_or_index(expr) } /// Whether an initializer that inferred to a type carrying no information @@ -1703,21 +1703,25 @@ fn assign_merge_type_owner_and_expr_type( // Where one did not, the merge below is provisional and the // settled pass re-derives it against the complete writer set. let mut skipped_uncached_sibling = false; - if let Some(widened_type) = get_widened_member_assignment_type( + let widened = get_widened_member_assignment_type( analyzer.db, &type_owner, &expr_type, preserve_table_literals, &mut skipped_uncached_sibling, - ) { - if skipped_uncached_sibling && let LuaTypeOwner::Member(member_id) = &type_owner - { - analyzer.context.record_settled_member_widening_candidate( - *member_id, - expr_type.clone(), - preserve_table_literals, - ); - } + ); + // Recorded on the skip, not on the answer: a walk that read no + // sibling type declines to widen at all, and that write needs + // the settled re-derivation just as much as one that widened + // from a partial set. + if skipped_uncached_sibling && let LuaTypeOwner::Member(member_id) = &type_owner { + analyzer.context.record_settled_member_widening_candidate( + *member_id, + expr_type.clone(), + preserve_table_literals, + ); + } + if let Some(widened_type) = widened { expr_type = widened_type; } } @@ -1740,7 +1744,7 @@ fn assign_merge_type_owner_and_expr_type( let guarded_table_assignment = preserve_table_literals || is_guarded_table_assignment_member(analyzer.db, *member_id); let conditional_branch_assignment = - is_member_assignment_in_conditional_branch(analyzer, *member_id); + is_member_assignment_in_conditional_branch(analyzer.db, *member_id); if !dynamic_expr_key_member { record_member_assignment_contribution( analyzer, @@ -1958,8 +1962,25 @@ fn record_member_assignment_contribution( guarded_bootstrap: bool, preserve_table_literals: bool, ) { - let doc_type = analyzer - .db + record_member_assignment_contribution_in( + analyzer.db, + member_id, + bound_type, + source_type, + guarded_bootstrap, + preserve_table_literals, + ); +} + +fn record_member_assignment_contribution_in( + db: &mut DbIndex, + member_id: LuaMemberId, + bound_type: &LuaType, + source_type: Option, + guarded_bootstrap: bool, + preserve_table_literals: bool, +) { + let doc_type = db .get_type_index() .get_type_cache(&member_id.into()) .filter(|cache| cache.is_doc()) @@ -1971,12 +1992,78 @@ fn record_member_assignment_contribution( guarded_bootstrap, preserve_table_literals, }; - analyzer - .db - .get_member_index_mut() + db.get_member_index_mut() .record_member_assignment_contribution(member_id, contribution); } +/// Records the evidence of an assignment whose value only resolved after the +/// walk had moved on. +/// +/// The walk records a contribution as it binds each write, so a write whose +/// right-hand side deferred contributes nothing and the settled merge never +/// sees it. Whether a write deferred is a fact about how far the batch had +/// run - a re-index keeps out-of-batch types standing and resolves inline what +/// a cold build had to defer - so the writer set the merge reads would +/// otherwise differ between the two. +pub(in crate::compilation::analyzer) fn record_resolved_member_assignment_contribution( + db: &mut DbIndex, + member_id: LuaMemberId, + bound_type: &LuaType, +) { + if !is_assignment_file_define_member(db, member_id) { + return; + } + if db + .get_member_index() + .member_assignment_contributions() + .contribution_of(&member_id) + .is_some() + { + return; + } + let guarded_bootstrap = is_guarded_table_assignment_member(db, member_id); + record_member_assignment_contribution_in( + db, + member_id, + bound_type, + None, + guarded_bootstrap, + false, + ); +} + +/// Applies the visibility marks a write earns from its own syntax, for a write +/// the walk did not get to classify. +/// +/// The walk marks each assignment as it binds it — guarded bootstrap, or +/// conditional branch — and those marks decide which writers stay visible for +/// the slot. A write whose right-hand side could not be inferred in time is +/// finished by the unresolve pass instead, which binds the type and stops, so +/// the marks are never applied. Both tests read syntax alone, so the answer is +/// the same either way; only whether anything asks was in question, and that is +/// a fact about how far the batch had run. +pub(in crate::compilation::analyzer) fn mark_resolved_member_assignment( + db: &mut DbIndex, + member_id: LuaMemberId, +) { + if !is_assignment_file_define_member(db, member_id) { + return; + } + if is_guarded_table_assignment_member(db, member_id) { + if !db + .get_member_index() + .is_non_overwriting_assignment_member(member_id) + { + db.get_member_index_mut() + .mark_non_overwriting_assignment_member(member_id); + preserve_guarded_table_assignment_members(db, member_id); + } + } else if is_member_assignment_in_conditional_branch(db, member_id) { + db.get_member_index_mut() + .mark_conditional_branch_assignment_member(member_id); + } +} + fn record_member_assignment_widening_cache( analyzer: &mut LuaAnalyzer, type_owner: &LuaTypeOwner, @@ -2052,11 +2139,8 @@ pub(in crate::compilation::analyzer) fn preserve_guarded_table_assignment_member /// ``` /// /// would silently drop the `Vector` branch and hover `obj.field` as just `nil`. -fn is_member_assignment_in_conditional_branch( - analyzer: &LuaAnalyzer, - member_id: LuaMemberId, -) -> bool { - let Some(tree) = analyzer.db.get_vfs().get_syntax_tree(&member_id.file_id) else { +fn is_member_assignment_in_conditional_branch(db: &DbIndex, member_id: LuaMemberId) -> bool { + let Some(tree) = db.get_vfs().get_syntax_tree(&member_id.file_id) else { return false; }; let root = tree.get_red_root(); @@ -2138,6 +2222,19 @@ pub(in crate::compilation::analyzer) fn get_widened_member_assignment_type( if related_member_id == *member_id { continue; } + // Only writers that come before this one are evidence for it. The walk + // otherwise settles that with "the sibling already has a type cache", + // which reports how far the batch has run rather than anything about + // the source: a re-index clears the batch's caches and leaves the rest + // standing, so the same sibling counts on one run and not on another. + // Reading the order off the source makes the set identical on both, + // and it is the rule the settled re-derivation already applies - a + // later write must not widen the type it is itself checked against. + if !preserve_table_literals + && member_id_sort_key(related_member_id) >= member_id_sort_key(*member_id) + { + continue; + } if !is_member_realm_compatible(db, *member_id, related_member_id) { continue; } @@ -2186,9 +2283,13 @@ pub(in crate::compilation::analyzer) fn get_widened_member_assignment_type( previous_states.iter(), ) } - MemberAssignmentWideningDecision::NoPreviousAssignments => { - widen_related_assignment_type(incoming_type, false) - } + // Only reachable once a preceding writer has been seen but every one of + // them was skipped for having no type yet: siblings exist, and not one + // of them is evidence. Widening the literal here guesses at writers this + // pass has not read, and how many it has read is how far the batch has + // run, not anything about the source. Leave the write as it stands and + // let the settled re-derivation decide against the complete set. + MemberAssignmentWideningDecision::NoPreviousAssignments => return None, }; Some(if preserve_table_literals { @@ -3698,8 +3799,8 @@ mod tests { #[test] fn member_collection_assignment_widening_cache_respects_load_state_masks() { let mut db = DbIndex::new(); - db.get_gmod_infer_index_mut() - .set_all_realm_file_metadata(std::collections::HashMap::from([ + db.get_gmod_infer_index_mut().set_all_realm_file_metadata( + rustc_hash::FxHashMap::from_iter([ ( FileId::new(0), crate::GmodRealmFileMetadata { @@ -3714,7 +3815,8 @@ mod tests { ..Default::default() }, ), - ])); + ]), + ); let owner = LuaMemberOwner::Element(InFiled::new( FileId::new(0), diff --git a/crates/glua_code_analysis/src/compilation/analyzer/mod.rs b/crates/glua_code_analysis/src/compilation/analyzer/mod.rs index 87767ef8b..5d69c292b 100644 --- a/crates/glua_code_analysis/src/compilation/analyzer/mod.rs +++ b/crates/glua_code_analysis/src/compilation/analyzer/mod.rs @@ -30,8 +30,8 @@ use crate::{ semantic::infer_expr_fact_with_cache, }; use glua_parser::{ - LuaAstNode, LuaCallExpr, LuaChunk, LuaClosureExpr, LuaExpr, LuaNameExpr, LuaSyntaxId, - LuaSyntaxNode, + BinaryOperator, LuaAstNode, LuaCallExpr, LuaChunk, LuaClosureExpr, LuaExpr, LuaNameExpr, + LuaSyntaxId, LuaSyntaxNode, }; use infer_cache_manager::InferCacheManager; use lua::LuaReturnPoint; @@ -258,6 +258,16 @@ pub fn analyze(db: &mut DbIndex, need_analyzed_files: Vec>) { common::reconcile_parked_global_path_members(db); } + // Writes that inferred their prefix to one concrete declaration of a + // multi-declaration global attach directly to that table and never + // park, so which table won depends on batch composition. Re-apply the + // ownership rule to them now that every declaration stands. See + // `reconcile_directly_attached_candidate_members`. + { + let _p = Profile::new("reconcile_directly_attached_candidate_members"); + common::reconcile_directly_attached_candidate_members(db); + } + // Runs last of the settled passes: it needs every member to have reached // its final owner, because the writer set it merges is grouped by owner. { @@ -266,6 +276,23 @@ pub fn analyze(db: &mut DbIndex, need_analyzed_files: Vec>) { lua::rederive_contributed_member_assignments(db, &analyzed_files); } + // Every settled pass above refines the types the member attach retry + // reads, so candidates it could not place on the first attempt can be + // placed now. Without this a member's existence depends on how far + // inference had progressed when its file happened to be walked. + { + let _p = Profile::new("attach_settled_index_expr_members (late)"); + attach_settled_index_expr_members(db, &mut context); + } + + // The late attach can still place members straight onto whichever + // candidate table its prefix resolved to, so the direct-attached + // repair has to see its results too. + { + let _p = Profile::new("reconcile_directly_attached_candidate_members (late)"); + common::reconcile_directly_attached_candidate_members(db); + } + // Net flows are collected last: the collector resolves wrappers through // signatures, receiver types and members, none of which exist yet when // the gmod pre-pass runs. See `GmodNetworkAnalysisPipeline`. @@ -312,6 +339,7 @@ fn attach_settled_index_expr_members(db: &mut DbIndex, context: &mut AnalyzeCont } candidates.sort_by_key(|candidate| (candidate.file_id, candidate.value.get_range().start())); candidates.dedup(); + let mut retry = Vec::new(); // Only the candidate files are re-inferred, so only their caches are stale. // Clearing the whole manager would also discard caches the passes that ran @@ -353,8 +381,17 @@ fn attach_settled_index_expr_members(db: &mut DbIndex, context: &mut AnalyzeCont ret_idx: 0, }; let cache = context.infer_manager.get_infer_cache(file_id); - let _ = unresolve::try_resolve_member(db, cache, &mut unresolve_member); + if unresolve::try_resolve_member(db, cache, &mut unresolve_member).is_err() { + // The prefix still has not settled. Dropping it here is what made a + // member's existence depend on analysis order: the passes that run + // after this one go on refining the very types this retry needs, so + // a candidate that fails now can succeed once they have. Keep it + // queued for the next attempt instead. + retry.push(candidate); + } } + + context.settled_member_attach_candidates = retry; } /// Re-resolves inferred returns that settled on `any`/`unknown`. @@ -571,21 +608,64 @@ fn refresh_local_decl_initializer_caches(db: &mut DbIndex, context: &mut Analyze .get_reference_index() .get_decl_references(&decl_id.file_id, decl_id) .is_none_or(|references| !references.mutable); - if !current_is_uninformative && !can_refine_nominal_type && !can_upgrade_authority { - continue; - } - let Some((ret_idx, expr)) = local_initializer_expr(db, &root, *decl_id) else { continue; }; - if !matches!(expr, LuaExpr::CallExpr(_) | LuaExpr::IndexExpr(_)) { + if !initializer_reads_through_call_or_index(&expr) { continue; } + + // Every pass before the dynamic-field one ran without those facts, + // so an initializer that reads a dynamic field was answered blind + // and the answer was cached as if it were final. Re-inferring with + // the index hidden reproduces exactly that blind answer, so where + // it differs from the settled one *and* matches what is cached, the + // cache is provably the guess and the settled read replaces it. + // Whether the field's writer had been walked yet is a property of + // the batch, not of the source: cold cached `false` for + // `local on = LocalPlayer()._flag or false` where re-analysing the + // same unchanged file cached `true`. let inferred_fact = select_result_fact( - infer_expr_fact_with_cache(db, &mut infer_cache, expr), + infer_expr_fact_with_cache(db, &mut infer_cache, expr.clone()), ret_idx, ); let inferred_type = inferred_fact.typ().clone(); + + // Only asked when nothing else would let the settled read through + // and it actually disagrees with the cache, so the second inference + // is paid for the handful of decls whose answer it can change. + let cached_a_blind_dynamic_field_read = !current_is_uninformative + && dynamic_fields_visible + && current_cache + .as_ref() + .is_some_and(|current| current.as_type() != &inferred_type) + && { + let mut blind_cache = crate::LuaInferCache::new( + file_id, + crate::CacheOptions { + analysis_phase, + dynamic_fields_visible: false, + building_dynamic_field_index: false, + }, + ); + let blind_type = select_result_fact( + infer_expr_fact_with_cache(db, &mut blind_cache, expr), + ret_idx, + ) + .typ() + .clone(); + current_cache + .as_ref() + .is_some_and(|current| current.as_type() == &blind_type) + && blind_type != inferred_type + }; + if !current_is_uninformative + && !can_refine_nominal_type + && !can_upgrade_authority + && !cached_a_blind_dynamic_field_read + { + continue; + } if type_is_uninformative(&inferred_type) { // When the cache and the settled re-derivation disagree // over *which* bottom an unresolvable initializer has, both @@ -644,6 +724,7 @@ fn refresh_local_decl_initializer_caches(db: &mut DbIndex, context: &mut Analyze } else if has_stronger_declared_authority || is_nominal_refinement || is_settled_widening + || cached_a_blind_dynamic_field_read { result.updates.push(InitializerCacheUpdate::Overwrite { owner: type_owner, @@ -919,6 +1000,30 @@ fn single_nominal_type_id(typ: &LuaType) -> Option { } } +/// Whether an initializer's type is decided by a call or index read. +/// +/// `or`, `and` and parentheses take their type from an operand, so they inherit +/// exactly the same sensitivity to what the batch has indexed so far while +/// hiding it behind a different syntax node. +pub(crate) fn initializer_reads_through_call_or_index(expr: &LuaExpr) -> bool { + match expr { + LuaExpr::CallExpr(_) | LuaExpr::IndexExpr(_) => true, + LuaExpr::ParenExpr(paren) => paren + .get_expr() + .is_some_and(|inner| initializer_reads_through_call_or_index(&inner)), + LuaExpr::BinaryExpr(binary) => { + matches!( + binary.get_op_token().map(|op| op.get_op()), + Some(BinaryOperator::OpOr | BinaryOperator::OpAnd) + ) && binary.get_exprs().is_some_and(|(left, right)| { + initializer_reads_through_call_or_index(&left) + || initializer_reads_through_call_or_index(&right) + }) + } + _ => false, + } +} + fn local_initializer_expr( db: &DbIndex, root: &LuaSyntaxNode, @@ -1030,6 +1135,13 @@ fn run_analysis(db: &mut DbIndex, context: &mut AnalyzeCont .rsplit("::") .next() .unwrap_or_default(); + if context.tree_list.len() > 1 { + crate::progress::enter_phase( + crate::progress::phase_label(name), + context.tree_list.len(), + "files", + ); + } // Timed through the phase accumulator rather than a `Profile`: several // pipelines already carry their own `Profile`, and an unconditional one // here would add a log line per pipeline per batch on a live server. diff --git a/crates/glua_code_analysis/src/compilation/analyzer/parallel.rs b/crates/glua_code_analysis/src/compilation/analyzer/parallel.rs index 559203803..28ab4ffdb 100644 --- a/crates/glua_code_analysis/src/compilation/analyzer/parallel.rs +++ b/crates/glua_code_analysis/src/compilation/analyzer/parallel.rs @@ -22,10 +22,16 @@ use std::sync::atomic::{AtomicUsize, Ordering}; use crate::db_index::DbIndex; use crate::{FileId, profile::Profile}; +/// Below this many files, `thread::scope` spawn/join and atomic dispatch cost +/// more than the per-file work itself saves, so the batch runs inline. Picked +/// from the profiled cost of one pass over a handful of small files versus +/// spawning/parking a worker pool for it. +const MIN_PARALLEL_FILES: usize = 8; + /// Number of worker threads to use for per-file analysis passes. Capped at 16 to /// match the diagnostics path and avoid oversubscription on large machines. fn worker_count(file_count: usize) -> usize { - if file_count <= 1 { + if file_count < MIN_PARALLEL_FILES { return 1; } let cores = std::thread::available_parallelism() @@ -67,6 +73,11 @@ where // are claimed changes; each still holds its own file's result, so callers // see the same index-aligned `Vec` as before. let dispatch = dispatch_order(db, file_ids); + let report_step = if crate::progress::is_active() { + (n / 50).max(1) + } else { + 0 + }; std::thread::scope(|scope| { for _ in 0..workers { @@ -80,6 +91,9 @@ where if seq >= n { break; } + if report_step != 0 && seq.is_multiple_of(report_step) { + crate::progress::advance_current_phase(seq, n, "files"); + } let idx = dispatch[seq]; let file_id = file_ids[idx]; let value = f(db, file_id); diff --git a/crates/glua_code_analysis/src/compilation/analyzer/unresolve/mod.rs b/crates/glua_code_analysis/src/compilation/analyzer/unresolve/mod.rs index 38cbc3fc1..ad60435a5 100644 --- a/crates/glua_code_analysis/src/compilation/analyzer/unresolve/mod.rs +++ b/crates/glua_code_analysis/src/compilation/analyzer/unresolve/mod.rs @@ -411,7 +411,32 @@ fn try_resolve( ) -> Option { let mut profile = profile_enabled.then(TryResolveProfile::default); let mut cached_sorted_keys: Option> = None; + // Which `(item, reason)` pairs this call has already re-queued. + // + // A wave only continues while something `changed`, and moving an item to a + // *different* reason counts as change. Two items can hand each other the + // same pair of reasons indefinitely — A fails under X naming Y, B fails + // under Y naming X — so `changed` never settles and the wave loop never + // returns. Each wave also purges the inference caches of every file it + // touched, so the same failures are re-derived from scratch every time and + // the loop makes no progress at all. + // + // Re-queueing an item under a reason it has already been re-queued under is + // therefore not progress, and is parked instead. Every wave now either + // resolves an item or retires an `(item, reason)` pair, both of which are + // finite, so the loop terminates. + let mut requeued: HashSet<(UnResolveIdentity, InferFailReason)> = HashSet::new(); + // Waves have no file count to report, so they report what is still deferred. + let initial_outstanding: usize = reason_resolve.values().map(Vec::len).sum(); loop { + if crate::progress::is_active() { + let outstanding: usize = reason_resolve.values().map(Vec::len).sum(); + crate::progress::advance_current_phase( + initial_outstanding.saturating_sub(outstanding), + initial_outstanding, + "deferred", + ); + } let mut changed = false; let mut to_be_remove = Vec::new(); let mut retain_unresolve = Vec::new(); @@ -475,7 +500,9 @@ fn try_resolve( } } Err(reason) => { - if reason != *check_reason { + if reason != *check_reason + && requeued.insert((unresolve_identity(&unresolve), reason.clone())) + { changed = true; retry_file_ids.insert(file_id); retain_unresolve.push((unresolve, reason)); @@ -510,9 +537,15 @@ fn try_resolve( reason_resolve.entry(reason).or_default().push(unresolve); } - // Anything still parked is dropped with the wave: it never joins - // `reason_resolve` here, so it cannot keep a reason group alive into the - // outer round. + // Parking is a within-wave retry, not a hand-back: an item parked on the + // settling wave is dropped with it, and nothing re-adds it, so it never + // reaches the outer round's `set_force` and `resolve_all_reason`. That is + // deliberate. A parked item's reason names a dependency it has already + // failed on, and carrying the reason into the outer round makes + // `resolve_as_unknown` floor that dependency's type cache to `Unknown`. + // The floor is terminal, and it lands on facts the later passes would + // otherwise still derive, so surviving the settle costs more inference + // than the missing floor does. if !changed || reason_resolve.is_empty() { break; } @@ -766,6 +799,39 @@ fn unresolve_kind_rank(unresolve: &UnResolve) -> u8 { } } +/// Separates items whose kind, file and position are shared with a sibling. +/// Closure-argument and call-site-contribution items are all keyed on their +/// call's start position, and a module ref is keyed on the module rather than on +/// the owner receiving it. +#[derive(PartialEq, Eq, Hash)] +enum UnResolveDiscriminator { + None, + ParamIdx(usize), + Owner(LuaSemanticDeclId), +} + +/// Identifies an unresolve item across waves: the same syntax position in the +/// same file for the same item kind, plus the discriminator that separates +/// siblings sharing that position, is the same item. +type UnResolveIdentity = (u8, u32, u32, UnResolveDiscriminator); + +fn unresolve_identity(unresolve: &UnResolve) -> UnResolveIdentity { + let (file_id, position) = unresolve.sort_key(); + let discriminator = match unresolve { + UnResolve::ClosureParams(d) => UnResolveDiscriminator::ParamIdx(d.param_idx), + UnResolve::ClosureReturn(d) => UnResolveDiscriminator::ParamIdx(d.param_idx), + UnResolve::CallSiteContribution(d) => UnResolveDiscriminator::ParamIdx(d.param_idx), + UnResolve::ModuleRef(d) => UnResolveDiscriminator::Owner(d.owner_id.clone()), + _ => UnResolveDiscriminator::None, + }; + ( + unresolve_kind_rank(unresolve), + file_id, + position, + discriminator, + ) +} + fn unresolve_stable_cmp(a: &UnResolve, b: &UnResolve) -> Ordering { unresolve_kind_rank(a) .cmp(&unresolve_kind_rank(b)) diff --git a/crates/glua_code_analysis/src/compilation/analyzer/unresolve/resolve.rs b/crates/glua_code_analysis/src/compilation/analyzer/unresolve/resolve.rs index de9a06e45..63e31efc2 100644 --- a/crates/glua_code_analysis/src/compilation/analyzer/unresolve/resolve.rs +++ b/crates/glua_code_analysis/src/compilation/analyzer/unresolve/resolve.rs @@ -25,8 +25,8 @@ use crate::{ snapshot_callback_table_type, }, common::{ - TypeCacheWriteMode, add_member, bind_resolved_type, holds_unbound_iter_template, - write_type_cache, + TypeCacheWriteMode, add_member, bind_resolved_type, bind_type, + holds_unbound_iter_template, write_type_cache, }, lua::{ analyze_return_correlations, analyze_return_point, compute_module_semantic_id, @@ -163,7 +163,20 @@ pub fn try_resolve_decl( return Err(InferFailReason::UnResolveIterTemplate); } - bind_resolved_type(db, decl_id.into(), LuaTypeCache::InferType(expr_type)); + // Narrowing an uninformative decl cache is reserved for a right-hand side + // that reads through a call or index: that is the boundary both routes into + // this pass enforce before they queue an item + // (`should_retry_uninformative_initializer`, + // `should_retry_narrowing_decl_assignment`). A write that landed here only + // because its right-hand side could not be inferred while its file was + // walked arrives without that check, so applying the narrowing policy to it + // let any shape overwrite an authoritative `any` — but only in the builds + // where the inference happened to fail. + if crate::compilation::analyzer::initializer_reads_through_call_or_index(&expr) { + bind_resolved_type(db, decl_id.into(), LuaTypeCache::InferType(expr_type)); + } else { + bind_type(db, decl_id.into(), LuaTypeCache::InferType(expr_type)); + } Ok(()) } @@ -382,7 +395,15 @@ pub fn try_resolve_member( } let member_id = unresolve_member.member_id; - bind_resolved_type(db, member_id.into(), LuaTypeCache::InferType(expr_type)); + bind_resolved_type( + db, + member_id.into(), + LuaTypeCache::InferType(expr_type.clone()), + ); + crate::compilation::analyzer::lua::record_resolved_member_assignment_contribution( + db, member_id, &expr_type, + ); + crate::compilation::analyzer::lua::mark_resolved_member_assignment(db, member_id); } Ok(()) @@ -444,17 +465,16 @@ pub fn try_resolve_table_field( let field_key = field.get_field_key().ok_or(InferFailReason::None)?; let field_expr = field_key.get_expr().ok_or(InferFailReason::None)?; let field_type = infer_expr(db, cache, field_expr.clone())?; - let member_key: LuaMemberKey = match field_type { - LuaType::StringConst(s) => LuaMemberKey::Name((*s).clone()), - LuaType::IntegerConst(i) => LuaMemberKey::Integer(i), - _ => { - if field_type.is_table() { - LuaMemberKey::ExprType(field_type) - } else { - return Err(InferFailReason::None); - } - } - }; + // The same mapping the immediate path uses. Re-deriving it here used to + // drop the member for every key type that is neither a literal nor a table, + // so a table field whose key type was known straight away got a member while + // an identical one that had to wait for inference got none — the analysis + // disagreed with itself depending on the order files happened to be + // analysed in. + let member_key = LuaMemberKey::from_expr_type(field_type); + if matches!(member_key, LuaMemberKey::ExprType(ref typ) if typ.is_unknown()) { + return Err(InferFailReason::None); + } let file_id = unresolve_table_field.file_id; let table_expr = unresolve_table_field.table_expr.clone(); let owner_id = LuaMemberOwner::Element(InFiled { @@ -544,6 +564,21 @@ pub fn try_resolve_return_point( cache: &mut LuaInferCache, return_: &mut UnResolveReturn, ) -> ResolveResult { + // Deriving a return means inferring every return expression in the + // function, and `should_apply_resolved_return_docs` then discards the + // result whenever the signature already holds a concrete inferred return — + // it can only ever upgrade `unknown`/`any`. Asking that question first + // costs two field reads instead of a full inference, and this pass + // re-attempts the same signatures across waves. + if let Some(signature) = db.get_signature_index().get(&return_.signature_id) + && signature.resolve_return == SignatureReturnStatus::InferResolve + { + let current_return = signature.get_return_type(); + if !current_return.is_unknown() && !current_return.is_any() { + return Ok(()); + } + } + let return_correlations = analyze_return_correlations(db, cache, &return_.return_points); let return_docs = analyze_return_point(db, cache, &return_.return_points)?; @@ -2128,7 +2163,7 @@ fn semantic_decl_from_var_ref_id(var_ref_id: &VarRefId) -> Option Option { match expr { LuaExpr::NameExpr(name_expr) => Some(name_expr.get_name_text()?.to_string()), - LuaExpr::IndexExpr(index_expr) => index_expr.get_access_path(), + LuaExpr::IndexExpr(index_expr) => index_expr.get_access_path().map(Into::into), _ => None, } } diff --git a/crates/glua_code_analysis/src/compilation/analyzer/unresolve/resolve_closure.rs b/crates/glua_code_analysis/src/compilation/analyzer/unresolve/resolve_closure.rs index 65f1474de..f61f9904f 100644 --- a/crates/glua_code_analysis/src/compilation/analyzer/unresolve/resolve_closure.rs +++ b/crates/glua_code_analysis/src/compilation/analyzer/unresolve/resolve_closure.rs @@ -153,7 +153,11 @@ pub fn try_resolve_closure_return( .get_mut(&closure_return.signature_id) .ok_or(InferFailReason::None)?; - if ret_type.contain_tpl() { + // An `unknown`/`any` contextual return carries no information, but taking it + // would clear the body-derived return below and stamp `DocResolve` over the + // result, which every later repair pass then refuses to touch. Fall back to + // the body exactly as an unbound template return does. + if ret_type.contain_tpl() || ret_type.is_unknown() || ret_type.is_any() { return try_convert_to_func_body_infer(db, cache, closure_return); } @@ -486,6 +490,15 @@ fn resolve_closure_member_type( let signature = db.get_signature_index().get(id); if let Some(signature) = signature { + // An empty `return_docs` renders as `-> nil`, so a base whose + // return has not settled yet hands out a contract claiming it + // returns nothing. `resolve_doc_function` stamps that as + // `DocResolve`, which no later pass reopens, so the override + // would keep whichever answer the base happened to hold when + // this ran. Wait for the base to settle instead. + if !signature.is_resolve_return() { + return Err(InferFailReason::UnResolveSignatureReturn(*id)); + } let fake_doc_function = signature.to_doc_func_type(); resolve_doc_function(db, closure_params, &fake_doc_function, self_type) } else { diff --git a/crates/glua_code_analysis/src/compilation/test/annotation_test.rs b/crates/glua_code_analysis/src/compilation/test/annotation_test.rs index a69deca70..9b10eded2 100644 --- a/crates/glua_code_analysis/src/compilation/test/annotation_test.rs +++ b/crates/glua_code_analysis/src/compilation/test/annotation_test.rs @@ -787,6 +787,38 @@ mod test { assert_eq!(index_expr_ty(&ws, file_id, "ray.Hit"), ws.ty("boolean")); } + #[test] + fn test_outparam_without_field_updates_the_parameter_itself() { + let mut ws = VirtualWorkspace::new(); + ws.def_file( + "layout.lua", + r#" + ---@class HUDStackLayout + ---@field rowHeight integer + + glide = {} + + ---@outparam out HUDStackLayout + ---@param out table + function glide.GetHUDStackLayout(out) end + "#, + ); + let file_id = ws.def_file( + "test.lua", + r#" + local layout = {} + + glide.GetHUDStackLayout(layout) + + local rowHeight = layout.rowHeight + "#, + ); + assert_eq!( + index_expr_ty(&ws, file_id, "layout.rowHeight"), + ws.ty("integer") + ); + } + #[test] fn test_outparam_updates_assigned_output_field() { let mut ws = VirtualWorkspace::new(); diff --git a/crates/glua_code_analysis/src/compilation/test/assign_widening_scaling_test.rs b/crates/glua_code_analysis/src/compilation/test/assign_widening_scaling_test.rs index 250bbaac9..c537154f0 100644 --- a/crates/glua_code_analysis/src/compilation/test/assign_widening_scaling_test.rs +++ b/crates/glua_code_analysis/src/compilation/test/assign_widening_scaling_test.rs @@ -2,12 +2,88 @@ mod test { use std::time::Instant; + use crate::semantic::BASELINE_FLOW_WALKS; use crate::{Emmyrc, EmmyrcGmodScriptedClassScopeEntry, VirtualWorkspace}; fn legacy_scope(pattern: &str) -> EmmyrcGmodScriptedClassScopeEntry { EmmyrcGmodScriptedClassScopeEntry::LegacyGlob(pattern.to_string()) } + /// A closure reading an outer local resolves its baseline type by walking + /// back past every branch merge ahead of it, so the branch count is the + /// exponent if a merge point is derived once per path instead of once. + fn index_branch_merges_before_closure(count: usize) -> std::time::Duration { + let mut body = String::from("local value = 1\n"); + for i in 0..count { + body.push_str(&format!( + "if cond{i} then value = {i} else value = {} end\n", + i + 100 + )); + } + body.push_str("local read = function() return value end\n"); + + let mut ws = VirtualWorkspace::new(); + let start = Instant::now(); + ws.def(&body); + start.elapsed() + } + + /// Closure-baseline walks performed while indexing `count` branch merges. + fn baseline_walks_for_branch_merges(count: usize) -> u64 { + BASELINE_FLOW_WALKS.with(|walks| walks.set(0)); + let _ = index_branch_merges_before_closure(count); + BASELINE_FLOW_WALKS.with(|walks| walks.get()) + } + + /// The closure-baseline flow walk must memoise each merge point. Without + /// that it derives one per path reaching the merge, which is `2^n` in the + /// branch count rather than `n`. + /// + /// Counted rather than timed. The two behaviours differ by orders of + /// magnitude in *work done*, so counting says so directly, runs in + /// milliseconds, and cannot flake when the test suite saturates the CPU — + /// which a wall-clock ceiling here demonstrably does. + #[test] + fn closure_baseline_memoises_branch_merges() { + let small = baseline_walks_for_branch_merges(8); + let large = baseline_walks_for_branch_merges(16); + + assert!( + small > 0, + "no closure-baseline walk ran; the guard is vacuous" + ); + // Twice the branches. Memoised that is about twice the walks; deriving + // once per path would be squaring it, so anything near linear passes and + // the regression cannot. + assert!( + large <= small * 4, + "closure-baseline walks grew from {small} to {large} when the branch \ + count doubled; merge points are being re-derived once per path" + ); + } + + /// The ratio form of [`closure_baseline_memoises_branch_merges`], for + /// bisecting a regression that test has already caught. + #[test] + #[ignore = "wall-clock ratio, for bisecting; the counted guard above runs by default"] + fn closure_baseline_cost_stays_linear_in_branch_merges() { + // Warm up so first-file fixed costs (std/global setup) don't skew the ratio. + let _ = index_branch_merges_before_closure(4); + + let small = index_branch_merges_before_closure(10); + let large = index_branch_merges_before_closure(20); + + // Memoised this is linear; deriving per path doubles per branch. A 20x + // ceiling sits well above linear noise and well below exponential. + let ratio = large.as_secs_f64() / small.as_secs_f64().max(1e-6); + assert!( + ratio < 20.0, + "closure-baseline narrowing scaled exponentially with branch merges \ + (10 -> {small:?}, 20 -> {large:?}, ratio {ratio:.1}x); \ + merge points are being re-derived once per path" + ); + } + /// Regression guard for issue #36: a field assigned a very large number of /// times under distinct (branched / guarded) writes used to drive /// `lua analyze` into O(N²) behaviour — each assignment re-scanned every @@ -74,8 +150,15 @@ local unrelated = {} (start.elapsed(), ws.humanize_type(result_type)) } + /// Ignored because it is wall-clock: the ratio is stable against machine + /// speed but not against the test suite saturating the CPU around it, and + /// the sizes it needs to separate linear from quadratic take too long to + /// belong in a default run. There is no counted stand-in — the widening does + /// not route through an owner-scoped lookup that could be counted — so this + /// regression has no default-run guard. Run it before and after changes to + /// member assignment widening. #[test] - #[ignore = "wall-clock performance smoke; direct cache unit tests cover the hot path in default runs"] + #[ignore = "wall-clock ratio; run manually when touching member assignment widening"] fn repeated_field_assignment_indexing_stays_near_linear() { // Warm up so the first-file fixed costs (std/global setup) don't skew the // ratio, then measure two sizes that differ by 4×. @@ -215,7 +298,7 @@ local unrelated = {} } #[test] - #[ignore = "wall-clock performance smoke; direct cache unit tests cover the hot path in default runs"] + #[ignore = "wall-clock ratio, for bisecting"] fn distinct_self_field_assignments_index_near_linearly() { let _ = index_distinct_self_field_assignments(200); @@ -305,7 +388,7 @@ local result = T["entry"].name } #[test] - #[ignore = "wall-clock performance smoke; direct cache unit tests cover the hot path in default runs"] + #[ignore = "wall-clock ratio, for bisecting"] fn dynamic_key_collection_assignments_do_not_scan_owner_members_quadratically() { let _ = index_dynamic_key_collection_assignments(100); @@ -322,4 +405,64 @@ local result = T["entry"].name (500 -> {small:?}, 2000 -> {large:?}, ratio {ratio:.1}x)" ); } + + /// `count` reads of fields that do not exist, against one table that ends + /// up `count` fields wide. + fn index_misses_on_one_wide_table(count: usize) -> std::time::Duration { + let mut body = String::from("local store = {}\n"); + for i in 0..count { + body.push_str(&format!("store.f{i} = {i}\nlocal miss{i} = store.g{i}\n")); + } + body.push_str("return store\n"); + + let mut ws = VirtualWorkspace::new(); + let start = Instant::now(); + ws.def(&body); + start.elapsed() + } + + /// The same reads and the same field count, spread so that no table is + /// more than one field wide. + fn index_misses_on_narrow_tables(count: usize) -> std::time::Duration { + let mut body = String::new(); + for i in 0..count { + body.push_str(&format!( + "local store{i} = {{}}\nstore{i}.f{i} = {i}\nlocal miss{i} = store{i}.g{i}\n" + )); + } + + let mut ws = VirtualWorkspace::new(); + let start = Instant::now(); + ws.def(&body); + start.elapsed() + } + + /// A read of a field the table does not declare must not cost the width of + /// the owner. Both halves do the same number of reads over the same number + /// of fields and differ only in how wide any single table gets, so the ratio + /// isolates per-access cost that grows with owner width. + /// + /// Ignored because it is wall-clock, and the wide half takes tens of seconds + /// even when it is behaving — the residual is flow narrowing over the + /// writes, which this does not guard and which no cheap absolute ceiling can + /// separate from a regression. So this one has no default-run guard either. + #[test] + #[ignore = "wall-clock ratio; run manually when touching owner member lookup"] + fn named_field_misses_do_not_scan_every_owner_member() { + // Warm up so first-file fixed costs (std/global setup) don't skew the ratio. + let _ = index_misses_on_narrow_tables(100); + + let narrow = index_misses_on_narrow_tables(2000); + let wide = index_misses_on_one_wide_table(2000); + + // The residual gap is flow narrowing over the writes, which this does + // not guard, so the ceiling sits above that and below a full scan. + let ratio = wide.as_secs_f64() / narrow.as_secs_f64().max(1e-6); + assert!( + ratio < 25.0, + "field misses cost more on a wide table than on narrow ones \ + (narrow -> {narrow:?}, wide -> {wide:?}, ratio {ratio:.1}x); \ + a miss is probably walking every member of the owner again" + ); + } } diff --git a/crates/glua_code_analysis/src/compilation/test/closure_return_test.rs b/crates/glua_code_analysis/src/compilation/test/closure_return_test.rs index 5501ce73f..b6ebf375c 100644 --- a/crates/glua_code_analysis/src/compilation/test/closure_return_test.rs +++ b/crates/glua_code_analysis/src/compilation/test/closure_return_test.rs @@ -1,9 +1,9 @@ #[cfg(test)] mod test { - use glua_parser::{LuaAstNode, LuaNameExpr}; + use glua_parser::{LuaAstNode, LuaClosureExpr, LuaNameExpr}; use tokio_util::sync::CancellationToken; - use crate::{DiagnosticCode, VirtualWorkspace}; + use crate::{DiagnosticCode, LuaSignatureId, LuaType, VirtualWorkspace}; fn local_name_type( ws: &VirtualWorkspace, @@ -210,4 +210,41 @@ mod test { assert!(before.is_empty(), "unexpected diagnostics: {before:?}"); assert_eq!(before, after); } + + /// An `any`/`unknown` return on the expected callback type says nothing + /// about what this callback returns. Taking it cleared the body-derived + /// return and stamped `DocResolve` over the result, and every later repair + /// pass refuses to correct a documented return. + #[test] + fn uninformative_callback_return_keeps_body_inference() { + let mut ws = VirtualWorkspace::new(); + let file_id = ws.def( + r#" + ---@param cb fun(): any + local function register(cb) end + + register(function() + _side_effect = 1 + end) + "#, + ); + + let semantic_model = ws + .analysis + .compilation + .get_semantic_model(file_id) + .expect("semantic model"); + let closure = semantic_model + .get_root() + .descendants::() + .last() + .expect("callback closure"); + let signature = semantic_model + .get_db() + .get_signature_index() + .get(&LuaSignatureId::from_closure(file_id, &closure)) + .expect("callback signature"); + + assert_eq!(signature.get_return_type(), LuaType::Nil); + } } diff --git a/crates/glua_code_analysis/src/compilation/test/gmod_network_test.rs b/crates/glua_code_analysis/src/compilation/test/gmod_network_test.rs index 77d82c59f..2b033e2a9 100644 --- a/crates/glua_code_analysis/src/compilation/test/gmod_network_test.rs +++ b/crates/glua_code_analysis/src/compilation/test/gmod_network_test.rs @@ -375,4 +375,42 @@ mod test { assert_that!(wrapped_flow.is_wrapped, eq(true)); assert_that!(wrapped_flow.send_range, eq(wrapped_flow.start_range)); } + + #[gtest] + fn test_send_flow_through_two_local_wrapper_levels() { + let mut ws = VirtualWorkspace::new(); + set_gmod_enabled(&mut ws); + + let file_id = ws.def_file( + "addons/mytest/lua/autorun/server/net_local_chain.lua", + r#" + local function fwd(name) + net.Start(name) + net.WriteString("payload") + net.Broadcast() + end + + local function api(name) + fwd(name) + end + + api("ChainedMessage") + "#, + ); + + let data = ws + .get_db_mut() + .get_gmod_network_index() + .get_file_data(file_id) + .expect("expected network data"); + + let chained: Vec<_> = data + .send_flows + .iter() + .filter(|flow| flow.message_name == "ChainedMessage") + .collect(); + + assert_that!(chained.len(), ge(1usize)); + assert_that!(send_op_kinds(chained[0]), eq(&vec!["string".to_string()])); + } } diff --git a/crates/glua_code_analysis/src/compilation/test/member_infer_test.rs b/crates/glua_code_analysis/src/compilation/test/member_infer_test.rs index e89227227..c929ee01f 100644 --- a/crates/glua_code_analysis/src/compilation/test/member_infer_test.rs +++ b/crates/glua_code_analysis/src/compilation/test/member_infer_test.rs @@ -3658,3 +3658,84 @@ end) ); } } + +#[cfg(test)] +mod undefined_global_reads_as_nil { + use crate::{LuaType, VirtualWorkspace}; + + /// Reading a name that is declared nowhere yields `nil` at runtime, so an + /// assignment from one contributes `nil` to the target — not `unknown`. + /// + /// The flow walk used to take `unknown` from the failed inference and let it + /// swallow everything the other branches had established, so one undefined + /// name erased the type of a local for the rest of its life and then rode + /// into every member that local was assigned to. + #[test] + fn assignment_from_an_undefined_global_narrows_to_nil() { + let mut ws = VirtualWorkspace::new_with_init_std_lib(); + ws.def( + r#" + local tax = 1 + if tax > 0 then + tax = never_declared_anywhere + end + after_branch = tax + bare_read = never_declared_anywhere + "#, + ); + + assert_eq!( + ws.expr_ty("after_branch"), + ws.ty("integer?"), + "an undefined global contributes nil, so the local stays integer?" + ); + assert_eq!(ws.expr_ty("bare_read"), LuaType::Nil); + } +} + +#[cfg(test)] +mod default_value_idiom_is_walk_order_independent { + use crate::VirtualWorkspace; + + /// `p = p or DEFAULT` must mean the same thing whichever file declared + /// `DEFAULT` first. + /// + /// `special_or_rule` used to answer `unknown | right` whenever the left arm + /// was `unknown`, which contradicted the general rule beneath it ("an + /// unresolved left operand has no enumerable truthy half, so it contributes + /// nothing"). Whether the left arm had resolved yet is a property of how far + /// the walk had run, so the same source read `integer` when the constant's + /// file came first and `integer|unknown` when it came second — and a + /// re-index of a subset then disagreed with the build that produced it. + fn default_type_for_order(files: Vec<(&str, &str)>, probe: &str) -> crate::LuaType { + let mut ws = VirtualWorkspace::new_with_init_std_lib(); + ws.def_files(files); + ws.expr_ty(probe) + } + + const READER: &str = r#" + function SetData(dataType) + dataType = dataType or DATA_PLAYER + probe_result = dataType + end + "#; + const CONST_DEF: &str = "DATA_PLAYER = 2"; + + #[test] + fn constant_declared_after_the_reader() { + let ty = default_type_for_order( + vec![("a.lua", READER), ("b.lua", CONST_DEF)], + "probe_result", + ); + assert_eq!(ty, crate::LuaType::Integer, "got: {ty:?}"); + } + + #[test] + fn constant_declared_before_the_reader() { + let ty = default_type_for_order( + vec![("a.lua", CONST_DEF), ("b.lua", READER)], + "probe_result", + ); + assert_eq!(ty, crate::LuaType::Integer, "got: {ty:?}"); + } +} diff --git a/crates/glua_code_analysis/src/compilation/test/unguarded_child_inference_test.rs b/crates/glua_code_analysis/src/compilation/test/unguarded_child_inference_test.rs index 04b72ed68..5a59b7971 100644 --- a/crates/glua_code_analysis/src/compilation/test/unguarded_child_inference_test.rs +++ b/crates/glua_code_analysis/src/compilation/test/unguarded_child_inference_test.rs @@ -91,6 +91,31 @@ mod test { .collect() } + fn member_path_receiver_types( + ws: &VirtualWorkspace, + file_id: crate::FileId, + path_text: &str, + ) -> Vec { + let semantic_model = ws + .analysis + .compilation + .get_semantic_model(file_id) + .expect("semantic model"); + semantic_model + .get_root() + .descendants::() + .filter_map(|index_expr| match index_expr.get_prefix_expr() { + Some(LuaExpr::IndexExpr(receiver)) + if receiver.syntax().text().to_string().trim() == path_text => + { + semantic_model.infer_expr(LuaExpr::IndexExpr(receiver)).ok() + } + _ => None, + }) + .map(|typ| ws.humanize_type(typ)) + .collect() + } + #[derive(Debug, PartialEq, Eq)] struct NestedCallbackState { receiver_types: Vec, @@ -1871,6 +1896,283 @@ mod test { ); } + #[test] + fn declared_field_narrows_to_the_only_child_defining_the_member() { + let mut ws = VirtualWorkspace::new(); + enable_gmod(&mut ws); + let file_id = ws.def( + r#" + ---@class Entity + ---@class Player: Entity + ---@field ConCommand fun(self: Player, cmd: string) + ---@class Holder + ---@field Owner Entity + ---@type Holder + local h + h.Owner:ConCommand("kill") + h.Owner:ConCommand("say") + "#, + ); + + assert_eq!( + diagnostic_count(&mut ws, file_id, DiagnosticCode::UndefinedMethod), + 0 + ); + let diagnostics = diagnostics_for(&mut ws, file_id, DiagnosticCode::InferUnguardedChild); + assert_eq!(diagnostics.len(), 1, "{diagnostics:?}"); + assert_eq!( + diagnostics[0].message, + "expected `Player` but found `Entity`. Add a guard to narrow the parent to `Player`." + ); + assert_eq!( + member_path_receiver_types(&ws, file_id, "h.Owner"), + vec!["Player", "Player"] + ); + } + + #[test] + fn declared_field_narrows_from_a_single_use() { + let mut ws = VirtualWorkspace::new(); + enable_gmod(&mut ws); + let file_id = ws.def( + r#" + ---@class Entity + ---@class Player: Entity + ---@field ConCommand fun(self: Player, cmd: string) + ---@class Holder + ---@field Owner Entity + ---@type Holder + local h + h.Owner:ConCommand("kill") + "#, + ); + + assert_eq!( + diagnostic_count(&mut ws, file_id, DiagnosticCode::UndefinedMethod), + 0 + ); + assert_eq!( + diagnostic_count(&mut ws, file_id, DiagnosticCode::InferUnguardedChild), + 1 + ); + assert_eq!( + member_path_receiver_types(&ws, file_id, "h.Owner"), + vec!["Player"] + ); + } + + #[test] + fn declared_field_narrowing_follows_a_deeper_member_path() { + let mut ws = VirtualWorkspace::new(); + enable_gmod(&mut ws); + let file_id = ws.def( + r#" + ---@class Entity + ---@class Player: Entity + ---@field ConCommand fun(self: Player, cmd: string) + ---@class Inner + ---@field Owner Entity + ---@class Outer + ---@field data Inner + ---@type Outer + local o + o.data.Owner:ConCommand("kill") + "#, + ); + + assert_eq!( + diagnostic_count(&mut ws, file_id, DiagnosticCode::UndefinedMethod), + 0 + ); + assert_eq!( + diagnostic_count(&mut ws, file_id, DiagnosticCode::InferUnguardedChild), + 1 + ); + assert_eq!( + member_path_receiver_types(&ws, file_id, "o.data.Owner"), + vec!["Player"] + ); + } + + #[test] + fn declared_field_member_owned_by_the_base_is_not_narrowed() { + let mut ws = VirtualWorkspace::new(); + enable_gmod(&mut ws); + let file_id = ws.def( + r#" + ---@class Entity + ---@field GetClass fun(self: Entity): string + ---@class Player: Entity + ---@field GetClass fun(self: Player): string + ---@class Holder + ---@field Owner Entity + ---@type Holder + local h + local class = h.Owner:GetClass() + print(class) + "#, + ); + + assert_eq!( + diagnostic_count(&mut ws, file_id, DiagnosticCode::InferUnguardedChild), + 0 + ); + assert_eq!( + member_path_receiver_types(&ws, file_id, "h.Owner"), + vec!["Entity"] + ); + } + + #[test] + fn guarded_declared_field_is_not_unguarded_child_evidence() { + let mut ws = VirtualWorkspace::new(); + enable_gmod(&mut ws); + let file_id = ws.def( + r#" + ---@class Entity + ---@class Player: Entity + ---@field ConCommand fun(self: Player, cmd: string) + + ---@return boolean + ---@return_cast self Player + function Entity:IsPlayer() end + + ---@class Holder + ---@field Owner Entity + ---@type Holder + local h + if h.Owner:IsPlayer() then + h.Owner:ConCommand("kill") + end + "#, + ); + + assert_eq!( + diagnostic_count(&mut ws, file_id, DiagnosticCode::UndefinedMethod), + 0 + ); + assert_eq!( + diagnostic_count(&mut ws, file_id, DiagnosticCode::InferUnguardedChild), + 0 + ); + } + + #[test] + fn declared_field_tie_names_the_member_instead_of_an_unwritable_union() { + let mut ws = VirtualWorkspace::new(); + enable_gmod(&mut ws); + let file_id = ws.def( + r#" + ---@class Entity + ---@class Bravo: Entity + ---@field Shared fun(self: Bravo) + ---@class Alpha: Entity + ---@field Shared fun(self: Alpha) + ---@class Holder + ---@field Owner Entity + ---@type Holder + local h + h.Owner:Shared() + "#, + ); + + let diagnostics = diagnostics_for(&mut ws, file_id, DiagnosticCode::InferUnguardedChild); + assert_eq!(diagnostics.len(), 1, "{diagnostics:?}"); + assert_eq!( + diagnostics[0].message, + "`Shared` is not defined on `Entity`. Add a guard that narrows the parent to one of \ + `Alpha`, `Bravo`." + ); + assert_eq!( + member_path_receiver_types(&ws, file_id, "h.Owner"), + vec!["(Alpha|Bravo)"] + ); + } + + #[test] + fn unguarded_child_tie_caps_the_listed_candidate_types() { + let mut ws = VirtualWorkspace::new(); + enable_gmod(&mut ws); + let file_id = ws.def( + r#" + ---@class Panel + ---@class DAlpha: Panel + ---@field Shared fun(self: DAlpha) + ---@class DBravo: Panel + ---@field Shared fun(self: DBravo) + ---@class DCharlie: Panel + ---@field Shared fun(self: DCharlie) + ---@class DDelta: Panel + ---@field Shared fun(self: DDelta) + ---@class DEcho: Panel + ---@field Shared fun(self: DEcho) + ---@type Panel + local value + value:Shared() + "#, + ); + + let diagnostics = diagnostics_for(&mut ws, file_id, DiagnosticCode::InferUnguardedChild); + assert_eq!(diagnostics.len(), 1, "{diagnostics:?}"); + assert_eq!( + diagnostics[0].message, + "`Shared` is not defined on `Panel`. Add a guard that narrows the parent to one of \ + `DAlpha`, `DBravo`, `DCharlie` and 2 more." + ); + } + + /// `Alpha` and `Bravo` sit inside a `return`, so the early evidence pass + /// narrows their receiver to `Middle` and publishes it. The late pass then + /// sees that narrowed receiver again at the `Shared` use, and `Shared` is + /// defined on `Leaf`, a child of `Middle`. Scoring against the published + /// type would descend one more level of the class tree per pass, so every + /// use has to keep scoring against `Entity`. + #[test] + fn unguarded_child_path_does_not_rescore_against_its_own_narrowing() { + let mut ws = VirtualWorkspace::new(); + enable_gmod(&mut ws); + let file_id = ws.def( + r#" + ---@class Entity + ---@class Middle: Entity + ---@field Alpha fun(self: Middle) + ---@field Bravo fun(self: Middle) + ---@class Other: Entity + ---@field Shared fun(self: Other) + ---@class Leaf: Middle + ---@field Shared fun(self: Leaf) + + ---@param value any + ---@return TypeGuard + function isentity(value) end + + ---@class Holder + ---@field proc unknown + + ---@type Holder + local h + + local function run(a, b) + if not isentity(h.proc) then return end + if a then return h.proc:Alpha() end + if b then return h.proc:Bravo() end + return h.proc:Shared() + end + "#, + ); + + let diagnostics = diagnostics_for(&mut ws, file_id, DiagnosticCode::InferUnguardedChild); + assert_eq!(diagnostics.len(), 1, "{diagnostics:?}"); + assert_eq!( + diagnostics[0].message, + "expected `Middle` but found `Entity`. Add a guard to narrow the parent to `Middle`." + ); + assert_eq!( + member_path_receiver_types(&ws, file_id, "h.proc"), + vec!["Middle", "Middle", "Middle"] + ); + } + #[test] fn table_literal_through_declared_field_does_not_report_unguarded_child() { let mut ws = VirtualWorkspace::new(); diff --git a/crates/glua_code_analysis/src/db_index/declaration/decl_tree.rs b/crates/glua_code_analysis/src/db_index/declaration/decl_tree.rs index daa3a1413..9d5d64e0b 100644 --- a/crates/glua_code_analysis/src/db_index/declaration/decl_tree.rs +++ b/crates/glua_code_analysis/src/db_index/declaration/decl_tree.rs @@ -1,4 +1,5 @@ -use std::collections::{BTreeMap, HashMap}; +use rustc_hash::FxHashMap; +use std::collections::BTreeMap; use super::{LuaDeclId, decl, scope}; use crate::{FileId, db_index::LuaMemberId}; @@ -9,8 +10,8 @@ use scope::{LuaScope, LuaScopeId, LuaScopeKind, ScopeOrDeclId}; #[derive(Debug)] pub struct LuaDeclarationTree { file_id: FileId, - decls: HashMap, - module_decls_by_name: HashMap>, + decls: FxHashMap, + module_decls_by_name: FxHashMap>, scopes: Vec, } @@ -18,8 +19,8 @@ impl LuaDeclarationTree { pub fn new(file_id: FileId) -> Self { Self { file_id, - decls: HashMap::new(), - module_decls_by_name: HashMap::new(), + decls: FxHashMap::default(), + module_decls_by_name: FxHashMap::default(), scopes: Vec::new(), } } @@ -288,7 +289,7 @@ impl LuaDeclarationTree { self.scopes.get(scope_id.id as usize) } - pub fn get_decls(&self) -> &HashMap { + pub fn get_decls(&self) -> &FxHashMap { &self.decls } } diff --git a/crates/glua_code_analysis/src/db_index/declaration/mod.rs b/crates/glua_code_analysis/src/db_index/declaration/mod.rs index 6dc5f88a8..7c53ba6f2 100644 --- a/crates/glua_code_analysis/src/db_index/declaration/mod.rs +++ b/crates/glua_code_analysis/src/db_index/declaration/mod.rs @@ -8,8 +8,8 @@ pub use decl::{LocalAttribute, LuaDecl, LuaDeclInitializer}; pub use decl_id::LuaDeclId; pub use decl_tree::{LuaDeclOrMemberId, LuaDeclarationTree}; use rowan::TextRange; +use rustc_hash::FxHashMap; pub use scope::{LuaScope, LuaScopeId, LuaScopeKind, ScopeOrDeclId}; -use std::collections::HashMap; use crate::{FileId, LuaMemberId}; @@ -17,13 +17,13 @@ use super::traits::LuaIndex; #[derive(Debug)] pub struct LuaDeclIndex { - decl_trees: HashMap, + decl_trees: FxHashMap, /// The table literal a global declaration is written with — the `{}` of /// `X = {}` or of the GLua-idiomatic `X = X or {}`. - global_initializer_tables: HashMap, + global_initializer_tables: FxHashMap, /// The same fact for a *nested* global path: the `{}` of `X.k = {}` or /// of `X.k = X.k or {}`, keyed by the member that declares it. - global_member_initializer_tables: HashMap, + global_member_initializer_tables: FxHashMap, } impl Default for LuaDeclIndex { @@ -35,9 +35,9 @@ impl Default for LuaDeclIndex { impl LuaDeclIndex { pub fn new() -> Self { Self { - decl_trees: HashMap::new(), - global_initializer_tables: HashMap::new(), - global_member_initializer_tables: HashMap::new(), + decl_trees: FxHashMap::default(), + global_initializer_tables: FxHashMap::default(), + global_member_initializer_tables: FxHashMap::default(), } } diff --git a/crates/glua_code_analysis/src/db_index/dependency/file_dependency_relation.rs b/crates/glua_code_analysis/src/db_index/dependency/file_dependency_relation.rs index 3dba53073..a5a998fca 100644 --- a/crates/glua_code_analysis/src/db_index/dependency/file_dependency_relation.rs +++ b/crates/glua_code_analysis/src/db_index/dependency/file_dependency_relation.rs @@ -16,9 +16,23 @@ impl<'a> FileDependencyRelation<'a> { file_ids: &[FileId], metas: &HashSet, ) -> Vec { + self.get_analysis_levels(file_ids, metas) + .into_iter() + .flatten() + .collect() + } + + /// [`Self::get_best_analysis_order`] grouped into dependency levels: no + /// file depends on a same-level sibling; flattening reproduces the flat + /// order exactly. Cycle leftovers become single-file levels. + pub fn get_analysis_levels( + &self, + file_ids: &[FileId], + metas: &HashSet, + ) -> Vec> { let n = file_ids.len(); if n < 2 { - return file_ids.to_vec(); + return file_ids.iter().map(|&f| vec![f]).collect(); } let file_to_idx: HashMap = @@ -37,8 +51,10 @@ impl<'a> FileDependencyRelation<'a> { } } } - let mut result = Vec::with_capacity(n); + let mut levels: Vec> = Vec::new(); + let mut node_level = vec![0usize; n]; let mut queue = VecDeque::with_capacity(n); + let mut popped = 0usize; // 入度为0的节点，按优先级排序：meta文件优先，然后按FileId排序 let mut zero_in_degree: Vec = (0..n).filter(|&i| in_degree[i] == 0).collect(); @@ -58,13 +74,20 @@ impl<'a> FileDependencyRelation<'a> { } while let Some(idx) = queue.pop_front() { - result.push(file_ids[idx]); + let level = node_level[idx]; + if levels.len() == level { + levels.push(Vec::new()); + } + levels[level].push(file_ids[idx]); + popped += 1; // 收集新的入度为0的节点 let mut new_zero: Vec = Vec::new(); for &neighbor in &adjacency[idx] { in_degree[neighbor] -= 1; if in_degree[neighbor] == 0 { + // FIFO pops breadth-first, so `idx` is the deepest dependency. + node_level[neighbor] = level + 1; new_zero.push(neighbor); } } @@ -87,15 +110,16 @@ impl<'a> FileDependencyRelation<'a> { } // 处理循环依赖 - if result.len() < n { + if popped < n { for (idx, °) in in_degree.iter().enumerate() { if deg > 0 { - result.push(file_ids[idx]); + // One file per level: a cycle has no safe concurrent order. + levels.push(vec![file_ids[idx]]); } } } - result + levels } /// Get all direct and indirect dependencies for the file list @@ -210,6 +234,76 @@ mod tests { assert!(result.contains(&FileId::new(2))); } + #[test] + fn the_analysis_order_places_every_dependency_before_its_dependents() { + let mut map = HashMap::new(); + map.insert(1.into(), [2.into(), 3.into()].into_iter().collect()); + map.insert(2.into(), [3.into()].into_iter().collect()); + map.insert(3.into(), HashSet::new()); + map.insert(4.into(), [1.into()].into_iter().collect()); + map.insert(5.into(), HashSet::new()); + let rel = FileDependencyRelation::new(&map); + let files: Vec = (1..=5).map(FileId::new).collect(); + let metas = HashSet::from_iter([FileId::new(5)]); + + let order = rel.get_best_analysis_order(&files, &metas); + assert_eq!(order.len(), files.len()); + + let position = |file: FileId| order.iter().position(|&f| f == file).expect("file ordered"); + for (&file, deps) in &map { + for &dep in deps { + assert!( + position(dep) < position(file), + "{dep:?} is a dependency of {file:?} but was ordered after it: {order:?}" + ); + } + } + + // A meta file depends on nothing, so it must lead rather than merely + // land somewhere legal. + assert_eq!(order.first(), Some(&FileId::new(5))); + } + + #[test] + fn a_level_never_contains_a_file_depending_on_a_sibling() { + let mut map = HashMap::new(); + map.insert(1.into(), [2.into(), 3.into()].into_iter().collect()); + map.insert(2.into(), [3.into()].into_iter().collect()); + map.insert(3.into(), HashSet::new()); + map.insert(4.into(), [1.into()].into_iter().collect()); + map.insert(5.into(), HashSet::new()); + let rel = FileDependencyRelation::new(&map); + let files: Vec = (1..=5).map(FileId::new).collect(); + + let levels = rel.get_analysis_levels(&files, &HashSet::default()); + + for level in &levels { + for file in level { + let deps = &map[file]; + assert!( + !level.iter().any(|sibling| deps.contains(sibling)), + "{file:?} depends on a file in its own level {level:?}" + ); + } + } + } + + #[test] + fn cyclic_files_each_get_their_own_level() { + let mut map = HashMap::new(); + map.insert(1.into(), [2.into()].into_iter().collect()); + map.insert(2.into(), [1.into()].into_iter().collect()); + map.insert(3.into(), HashSet::new()); + let rel = FileDependencyRelation::new(&map); + let files: Vec = (1..=3).map(FileId::new).collect(); + + let levels = rel.get_analysis_levels(&files, &HashSet::default()); + + assert_eq!(levels[0], vec![FileId::new(3)]); + assert_eq!(levels[1], vec![FileId::new(1)]); + assert_eq!(levels[2], vec![FileId::new(2)]); + } + #[test] fn test_collect_file_dependents() { let mut deps = HashMap::new(); diff --git a/crates/glua_code_analysis/src/db_index/flow/mod.rs b/crates/glua_code_analysis/src/db_index/flow/mod.rs index 5c2486e36..46fe128a5 100644 --- a/crates/glua_code_analysis/src/db_index/flow/mod.rs +++ b/crates/glua_code_analysis/src/db_index/flow/mod.rs @@ -2,9 +2,8 @@ mod flow_node; mod flow_tree; mod signature_cast; -use std::collections::HashMap; - use rowan::TextSize; +use rustc_hash::FxHashMap as HashMap; use crate::{FileId, LuaSignatureId, LuaType, VarRefId}; pub use flow_node::*; @@ -32,9 +31,9 @@ impl Default for LuaFlowIndex { impl LuaFlowIndex { pub fn new() -> Self { Self { - file_flow_tree: HashMap::new(), - signature_cast_cache: HashMap::new(), - special_call_effects: HashMap::new(), + file_flow_tree: HashMap::default(), + signature_cast_cache: HashMap::default(), + special_call_effects: HashMap::default(), } } diff --git a/crates/glua_code_analysis/src/db_index/global/mod.rs b/crates/glua_code_analysis/src/db_index/global/mod.rs index 232735459..9376e1395 100644 --- a/crates/glua_code_analysis/src/db_index/global/mod.rs +++ b/crates/glua_code_analysis/src/db_index/global/mod.rs @@ -64,6 +64,19 @@ impl LuaGlobalIndex { self.global_decl.get(&id) } + /// Every global name that more than one declaration writes, sorted by + /// name so parents settle before the nested paths derived from them. + pub fn sorted_multi_declaration_globals(&self) -> Vec { + let mut global_ids = self + .global_decl + .iter() + .filter(|(_, decl_ids)| decl_ids.len() > 1) + .map(|(global_id, _)| global_id.clone()) + .collect::>(); + global_ids.sort_unstable_by(|left, right| left.get_name().cmp(right.get_name())); + global_ids + } + pub fn get_global_decl_ids_in_workspace( &self, name: &str, diff --git a/crates/glua_code_analysis/src/db_index/gmod_class/mod.rs b/crates/glua_code_analysis/src/db_index/gmod_class/mod.rs index f4d2a113d..4ca5efd91 100644 --- a/crates/glua_code_analysis/src/db_index/gmod_class/mod.rs +++ b/crates/glua_code_analysis/src/db_index/gmod_class/mod.rs @@ -254,6 +254,37 @@ pub struct GmodVguiParentCallMetadata { pub parent: GmodVguiParentSource, pub relations: Vec, pub origin: GmodVguiParentCallOrigin, + /// This call resolved against its file's syntax tree, before the + /// inheritance chain is walked. + /// + /// The chain walk is global and has to see every call, but resolving a call + /// means walking the declaring file's syntax tree — and re-analysing one + /// file cannot change what another file's call resolves to on its own. + /// Caching it here means a re-analysis only walks the files it rebuilt: + /// analysis creates calls with this empty, so a rebuilt file recomputes and + /// an untouched file reuses, with no dirty set to keep in step. + pub resolved_source: Option, +} + +/// How a vgui parent call names its parent, resolved to type ids. +#[derive(Debug, Clone, PartialEq, Eq)] +pub enum GmodVguiParentSourceResolution { + Direct(Vec), + AssignedField { + field_type_ids: Vec, + assignment_parent_type_ids: Vec, + }, + ReceiverField { + field_type_ids: Vec, + receiver_type_ids: Vec, + receiver_field_parent_type_ids: Option>, + }, +} + +#[derive(Debug, Clone, PartialEq, Eq)] +pub struct GmodVguiResolvedParentSource { + pub child_type_ids: Vec, + pub parent: GmodVguiParentSourceResolution, } impl GmodScriptedClassFileMetadata { @@ -630,6 +661,25 @@ impl GmodClassMetadataIndex { .unwrap_or_default() } + /// Cache each call's pre-chain resolution, so the next re-analysis only + /// walks the syntax trees of files it rebuilt. + pub fn set_vgui_resolved_parent_sources( + &mut self, + resolved_by_file: &[(FileId, Vec<(LuaSyntaxId, GmodVguiResolvedParentSource)>)], + ) { + for (file_id, resolved) in resolved_by_file { + let Some(metadata) = self.file_metadata.get_mut(file_id) else { + continue; + }; + for call in &mut metadata.vgui_parent_calls { + call.resolved_source = resolved + .iter() + .find(|(syntax_id, _)| *syntax_id == call.syntax_id) + .map(|(_, source)| source.clone()); + } + } + } + pub fn set_vgui_parent_relations( &mut self, resolved_by_file: Vec<(FileId, Vec<(LuaSyntaxId, Vec)>)>, diff --git a/crates/glua_code_analysis/src/db_index/gmod_infer/mod.rs b/crates/glua_code_analysis/src/db_index/gmod_infer/mod.rs index ed6a76a09..5a9c999f8 100644 --- a/crates/glua_code_analysis/src/db_index/gmod_infer/mod.rs +++ b/crates/glua_code_analysis/src/db_index/gmod_infer/mod.rs @@ -1,7 +1,5 @@ -use std::{ - collections::{HashMap, HashSet}, - sync::OnceLock, -}; +use rustc_hash::{FxHashMap as HashMap, FxHashSet as HashSet}; +use std::sync::OnceLock; use glua_parser::LuaSyntaxId; use rowan::TextRange; @@ -191,16 +189,22 @@ impl GmodSystemAggregate { file_id: FileId, name_range: TextRange, ) { - self.duplicate_registrations + let registrations = self + .duplicate_registrations .entry((kind, name.to_string())) - .or_default() - .push(GmodSystemRegistration { - kind, - convar_kind, - name: name.to_string(), - file_id, - name_range, - }); + .or_default(); + registrations.push(GmodSystemRegistration { + kind, + convar_kind, + name: name.to_string(), + file_id, + name_range, + }); + // Which registration a duplicate report calls the original is read off + // this list, so it has to come from source position rather than from + // the order the files happened to be analysed in. + registrations + .sort_by_key(|registration| (registration.file_id, registration.name_range.start())); } pub fn registrations( @@ -294,14 +298,14 @@ pub struct GmodInferIndex { impl GmodInferIndex { pub fn new() -> Self { Self { - hook_file_metadata: HashMap::new(), - system_file_metadata: HashMap::new(), + hook_file_metadata: HashMap::default(), + system_file_metadata: HashMap::default(), system_aggregate_cache: OnceLock::new(), - realm_file_metadata: HashMap::new(), - gm_method_realm_annotations: HashMap::new(), - member_realm_ranges: HashMap::new(), - fileparam_index: HashMap::new(), - scoped_class_info: HashMap::new(), + realm_file_metadata: HashMap::default(), + gm_method_realm_annotations: HashMap::default(), + member_realm_ranges: HashMap::default(), + fileparam_index: HashMap::default(), + scoped_class_info: HashMap::default(), } } diff --git a/crates/glua_code_analysis/src/db_index/member/assignment_contribution.rs b/crates/glua_code_analysis/src/db_index/member/assignment_contribution.rs index 18e53c5fd..58e9db156 100644 --- a/crates/glua_code_analysis/src/db_index/member/assignment_contribution.rs +++ b/crates/glua_code_analysis/src/db_index/member/assignment_contribution.rs @@ -76,6 +76,27 @@ impl MemberAssignmentContributionStore { self.by_owner_key.get(store_key) } + /// The contribution this member recorded, wherever its writer group + /// currently sits. + pub fn contribution_of( + &self, + member_id: &LuaMemberId, + ) -> Option<&MemberAssignmentContribution> { + let store_key = self.by_file.get(&member_id.file_id)?.get(member_id)?; + self.by_owner_key.get(store_key)?.get(member_id) + } + + /// The `(owner, key)` group this member's write currently contributes to. + pub fn contribution_group_of( + &self, + member_id: &LuaMemberId, + ) -> Option<(LuaMemberOwner, LuaMemberKey)> { + self.by_file + .get(&member_id.file_id)? + .get(member_id) + .cloned() + } + /// The distinct groups the given files wrote to. pub fn keys_for_files( &self, diff --git a/crates/glua_code_analysis/src/db_index/member/lua_member.rs b/crates/glua_code_analysis/src/db_index/member/lua_member.rs index 42d37ecd3..7fcdd0cca 100644 --- a/crates/glua_code_analysis/src/db_index/member/lua_member.rs +++ b/crates/glua_code_analysis/src/db_index/member/lua_member.rs @@ -139,7 +139,7 @@ impl LuaMemberKey { } } - fn from_expr_type(expr_type: LuaType) -> Self { + pub(crate) fn from_expr_type(expr_type: LuaType) -> Self { match expr_type { LuaType::StringConst(s) => LuaMemberKey::Name(s.deref().clone()), LuaType::DocStringConst(s) => LuaMemberKey::Name(s.deref().clone()), diff --git a/crates/glua_code_analysis/src/db_index/member/lua_member_item.rs b/crates/glua_code_analysis/src/db_index/member/lua_member_item.rs index ff2d2dbfb..98fd038b7 100644 --- a/crates/glua_code_analysis/src/db_index/member/lua_member_item.rs +++ b/crates/glua_code_analysis/src/db_index/member/lua_member_item.rs @@ -368,12 +368,14 @@ fn member_hidden_by_enclosing_assignment( return false; }; let member_range = member.get_range(); - let Some(token) = root.token_at_offset(member_range.start()).right_biased() else { + // By syntax id, not by offset: `token_at_offset` rescans the siblings at + // every level it descends, and this lookup is memoised. + let Some(member_node) = member_id.get_syntax_id().to_node_from_root(&root) else { return false; }; - token - .parent_ancestors() + member_node + .ancestors() .find_map(LuaAssignStat::cast) .is_some_and(|assign_stat| { assign_stat.get_range().contains(caller_position) @@ -423,7 +425,7 @@ fn assignment_rhs_self_coalesces_member( false } -fn expr_access_path(expr: &LuaExpr) -> Option { +fn expr_access_path(expr: &LuaExpr) -> Option { match expr { LuaExpr::NameExpr(name_expr) => name_expr.get_access_path(), LuaExpr::IndexExpr(index_expr) => index_expr.get_access_path(), diff --git a/crates/glua_code_analysis/src/db_index/member/lua_owner_members.rs b/crates/glua_code_analysis/src/db_index/member/lua_owner_members.rs index de4235460..25d3ea114 100644 --- a/crates/glua_code_analysis/src/db_index/member/lua_owner_members.rs +++ b/crates/glua_code_analysis/src/db_index/member/lua_owner_members.rs @@ -11,6 +11,11 @@ pub struct LuaOwnerMembers { // invalidation surface: `add_member`, `get_member_mut`, `iter_mut`, and // `remove_member`. sorted_ids_cache: OnceLock>, + /// The subset of `members` keyed by an expression rather than a name. + /// + /// Only `add_member` and `remove_member` change the key set; the mutators + /// that hand out an item leave keys alone. + expr_keys: Vec, resolve_state: OwnerMemberStatus, } @@ -20,15 +25,23 @@ impl LuaOwnerMembers { Self { members: HashMap::new(), sorted_ids_cache: OnceLock::new(), + expr_keys: Vec::new(), resolve_state: OwnerMemberStatus::UnResolved, } } pub fn add_member(&mut self, key: LuaMemberKey, item: LuaMemberIndexItem) { self.invalidate_sorted_member_ids(); + if key.is_expr() && !self.members.contains_key(&key) { + self.expr_keys.push(key.clone()); + } self.members.insert(key, item); } + pub fn expr_keys(&self) -> impl Iterator { + self.expr_keys.iter() + } + pub fn get_member(&self, key: &LuaMemberKey) -> Option<&LuaMemberIndexItem> { self.members.get(key) } @@ -75,6 +88,9 @@ impl LuaOwnerMembers { pub fn remove_member(&mut self, key: &LuaMemberKey) -> Option { self.invalidate_sorted_member_ids(); + if key.is_expr() { + self.expr_keys.retain(|expr_key| expr_key != key); + } self.members.remove(key) } diff --git a/crates/glua_code_analysis/src/db_index/member/mod.rs b/crates/glua_code_analysis/src/db_index/member/mod.rs index 994c2015f..d9089d7ae 100644 --- a/crates/glua_code_analysis/src/db_index/member/mod.rs +++ b/crates/glua_code_analysis/src/db_index/member/mod.rs @@ -12,6 +12,7 @@ use std::collections::BTreeMap; use super::traits::LuaIndex; use crate::{FileId, GlobalId, db_index::member::lua_owner_members::LuaOwnerMembers}; + pub use assignment_contribution::{ MemberAssignmentContribution, MemberAssignmentContributionKey, MemberAssignmentContributionStore, @@ -44,6 +45,9 @@ pub struct LuaMemberIndex { /// type read mid-fixpoint. deferred_index_expr_members: HashSet, function_scope_ranges: HashMap>, + /// Per file, each `if` branch's range paired with the range of the `if` it + /// belongs to, sorted by branch start. Recorded on the decl walk. + conditional_branch_ranges: HashMap>, member_function_scope_ranges: HashMap, /// Per-writer evidence for the member assignment widening merge. See /// [`MemberAssignmentContribution`]. @@ -89,6 +93,7 @@ impl LuaMemberIndex { synthesized_owner_members: HashSet::default(), deferred_index_expr_members: HashSet::default(), function_scope_ranges: HashMap::default(), + conditional_branch_ranges: HashMap::default(), member_function_scope_ranges: HashMap::default(), assignment_contributions: MemberAssignmentContributionStore::default(), } @@ -111,6 +116,12 @@ impl LuaMemberIndex { &self.assignment_contributions } + pub fn member_assignment_contributions_mut( + &mut self, + ) -> &mut MemberAssignmentContributionStore { + &mut self.assignment_contributions + } + pub fn add_member(&mut self, owner: LuaMemberOwner, member: LuaMember) -> LuaMemberId { let id = member.get_id(); let file_id = member.get_file_id(); @@ -281,13 +292,29 @@ impl LuaMemberIndex { match item { LuaMemberIndexItem::One(old_id) if *old_id == id => MemberInsertAction::Noop, _ => { - let winner = latest_defined_member(&old_member_ids, id); - if matches!(item, LuaMemberIndexItem::One(current) if *current == winner) { + // Ids this owner only *aliases* belong to another owner, and + // `add_member_alias_to_owner` never displaces what it finds. Letting + // one win this slot -- or evicting one -- would make the outcome + // depend on whether the owner's own write or the alias arrived + // first, which is a property of the batch, not of the source. + let (aliased, owned): (Vec<_>, Vec<_>) = old_member_ids + .iter() + .copied() + .partition(|old_id| self.member_current_owner.get(old_id) != Some(owner)); + let winner = latest_defined_member(&owned, id); + let mut visible = aliased; + visible.push(winner); + visible.sort_by_key(|visible_id| member_id_sort_key(*visible_id)); + let new_item = match visible.as_slice() { + [only] => LuaMemberIndexItem::One(*only), + _ => LuaMemberIndexItem::Many(visible), + }; + if item == &new_item { return MemberInsertAction::Noop; } MemberInsertAction::StoreRemovingVisibleOldIds { - item: LuaMemberIndexItem::One(winner), - old_ids: old_member_ids + item: new_item, + old_ids: owned .into_iter() .chain(std::iter::once(id)) .filter(|candidate| { @@ -326,21 +353,41 @@ impl LuaMemberIndex { }) } - /// The visible item for a slot that `candidates` write to, when at least one - /// of them is a conditional-branch write: every conditional writer, plus the - /// latest plain one because plain writers do dominate each other. Ordered by - /// [`member_id_sort_key`], so it is a pure function of the candidate set. + /// The visible item for a slot at least one conditional write reaches. + /// + /// Only writes that run in the same flow can overwrite each other, so the + /// candidates are bucketed by function scope: two writes in different scopes + /// are parallel -- each runs on its own object, which is how a class collects + /// a callback field from every instance that sets one -- and both stay + /// visible. Within one scope the writes are successive, so only the last one + /// survives, unless they sit in different branches of the same `if`, where + /// exactly one of them runs and all of them survive. + /// + /// Ordered by [`member_id_sort_key`], so it is a pure function of the + /// candidate set. fn conditional_branch_item(&self, candidates: &[LuaMemberId]) -> Option { - let (mut kept, plain): (Vec<_>, Vec<_>) = - candidates.iter().copied().partition(|candidate| { - self.conditional_branch_assignment_members - .contains(candidate) - }); - if kept.is_empty() { + if !candidates.iter().any(|candidate| { + self.conditional_branch_assignment_members + .contains(candidate) + }) { return None; } - if let Some(latest_plain) = plain.into_iter().max_by_key(|id| member_id_sort_key(*id)) { - kept.push(latest_plain); + + let mut scopes: Vec<(Option, Vec)> = Vec::new(); + for candidate in candidates.iter().copied() { + let scope = self.member_function_scope_range(candidate); + match scopes.iter_mut().find(|(seen, _)| *seen == scope) { + Some((_, members)) => members.push(candidate), + None => scopes.push((scope, vec![candidate])), + } + } + + let mut kept = Vec::new(); + for (_, members) in &scopes { + kept.extend(self.live_writes_in_one_scope(members)); + } + if kept.is_empty() { + return None; } kept.sort_by_key(|id| member_id_sort_key(*id)); @@ -350,6 +397,60 @@ impl LuaMemberIndex { }) } + /// The writes among `members` that can still be live at the end of the one + /// function scope they share: the branches of an `if` they write to from more + /// than one side, because exactly one of those runs, and otherwise just the + /// latest write, because successive writes in one flow overwrite each other. + fn live_writes_in_one_scope(&self, members: &[LuaMemberId]) -> Vec { + let chains = members + .iter() + .map(|member_id| self.enclosing_conditional_branches(*member_id)) + .collect::>(); + + // A write survives its scope only if some other write sits in a different + // branch of an `if` that encloses them both. Two survivors in the same + // branch still overwrite each other, so a branch contributes its latest. + let mut latest_per_branch: Vec<(Option<(TextRange, TextRange)>, LuaMemberId)> = Vec::new(); + for (index, member_id) in members.iter().copied().enumerate() { + let has_alternative = chains.iter().enumerate().any(|(other, other_chain)| { + other != index + && chains[index].iter().any(|(branch, if_range)| { + other_chain + .iter() + .any(|(seen, seen_if)| seen_if == if_range && seen != branch) + }) + }); + if !has_alternative { + continue; + } + let branch = chains[index].first().copied(); + match latest_per_branch + .iter_mut() + .find(|(seen, _)| *seen == branch) + { + Some(entry) => { + if member_id_sort_key(member_id) > member_id_sort_key(entry.1) { + entry.1 = member_id; + } + } + None => latest_per_branch.push((branch, member_id)), + } + } + if !latest_per_branch.is_empty() { + return latest_per_branch + .into_iter() + .map(|(_, member_id)| member_id) + .collect(); + } + + members + .iter() + .copied() + .max_by_key(|member_id| member_id_sort_key(*member_id)) + .into_iter() + .collect() + } + /// Re-resolves the slot `member_id` writes to, now that it is known to /// be a conditional-branch write. fn resolve_conditional_branch_owner_key_item(&mut self, member_id: LuaMemberId) -> Option<()> { @@ -487,6 +588,50 @@ impl LuaMemberIndex { Some(()) } + /// Whether every write + /// [`add_member_alias_to_owner`](Self::add_member_alias_to_owner) would + /// perform for `(owner, id)` is already in the index, so calling it would + /// leave the index unchanged. + pub(crate) fn alias_to_owner_is_recorded( + &self, + owner: &LuaMemberOwner, + id: LuaMemberId, + ) -> bool { + let Some(member) = self.get_member(&id) else { + return false; + }; + let key = member.get_key(); + + let is_indexed = |index: &HashMap>>| { + index + .get(owner) + .and_then(|members_by_key| members_by_key.get(key)) + .is_some_and(|member_ids| member_ids.contains(&id)) + }; + if self.member_current_owner.get(&id) != Some(owner) + && !(is_indexed(&self.member_owner_key_index) + && is_indexed(&self.member_owner_key_history_index)) + { + return false; + } + + let item_holds_member = self + .owner_members + .get(owner) + .and_then(|owner_members| owner_members.get_member(key)) + .is_some_and(|item| match item { + LuaMemberIndexItem::One(existing_id) => *existing_id == id, + LuaMemberIndexItem::Many(ids) => ids.contains(&id), + }); + if !item_holds_member { + return false; + } + + self.in_filed + .get(&member.get_file_id()) + .is_some_and(|objects| objects.contains(&MemberOrOwner::Owner(owner.clone()))) + } + fn should_preserve_assignment_file_define_member( &self, owner: &LuaMemberOwner, @@ -819,7 +964,6 @@ impl LuaMemberIndex { pub fn get_members(&self, owner: &LuaMemberOwner) -> Option> { let owner_members = self.owner_members.get(owner)?; - if owner_members.get_member_len() == 0 { return Some(Vec::new()); } @@ -842,6 +986,52 @@ impl LuaMemberIndex { ) } + /// The owner's members whose key is an expression rather than a name, in + /// the same order [`Self::get_members`] would yield them. + pub fn get_expr_key_members(&self, owner: &LuaMemberOwner) -> Option> { + let owner_members = self.owner_members.get(owner)?; + let mut member_ids = Vec::new(); + for key in owner_members.expr_keys() { + match owner_members.get_member(key) { + Some(LuaMemberIndexItem::One(id)) => member_ids.push(*id), + Some(LuaMemberIndexItem::Many(ids)) => member_ids.extend(ids.iter().copied()), + None => {} + } + } + member_ids.sort_by_key(|member_id| member_id_sort_key(*member_id)); + Some( + member_ids + .iter() + .filter_map(|member_id| self.get_member(member_id)) + .collect(), + ) + } + + /// The owner's members under one key, in the same order + /// [`Self::get_members`] would yield them. + /// + /// Returns `None` only when the owner has no member map at all, so a + /// caller can still tell "no such owner" from "owner without that key". + pub fn get_members_with_key( + &self, + owner: &LuaMemberOwner, + key: &LuaMemberKey, + ) -> Option> { + let owner_members = self.owner_members.get(owner)?; + let mut member_ids = match owner_members.get_member(key) { + Some(LuaMemberIndexItem::One(id)) => vec![*id], + Some(LuaMemberIndexItem::Many(ids)) => ids.clone(), + None => return Some(Vec::new()), + }; + member_ids.sort_by_key(|member_id| member_id_sort_key(*member_id)); + Some( + member_ids + .iter() + .filter_map(|member_id| self.get_member(member_id)) + .collect(), + ) + } + pub fn get_member_keys<'a>( &'a self, owner: &LuaMemberOwner, @@ -883,6 +1073,22 @@ impl LuaMemberIndex { .map_or(0, |map| map.get_member_len()) } + /// Whether the owner holds at least one member that still resolves. + /// + /// Not the same as a non-zero [`Self::get_member_len`]: removing a member + /// whose entry is already gone leaves its id behind under the owner, so a + /// key can outlive the member it points at. Stops at the first live id + /// rather than materialising the owner's whole member list. + pub fn has_live_member(&self, owner: &LuaMemberOwner) -> bool { + let Some(owner_members) = self.owner_members.get(owner) else { + return false; + }; + owner_members.get_member_items().any(|item| match item { + LuaMemberIndexItem::One(id) => self.get_member(id).is_some(), + LuaMemberIndexItem::Many(ids) => ids.iter().any(|id| self.get_member(id).is_some()), + }) + } + pub fn get_current_owner(&self, id: &LuaMemberId) -> Option<&LuaMemberOwner> { self.member_current_owner.get(id) } @@ -971,11 +1177,45 @@ impl LuaMemberIndex { owner: &LuaMemberOwner, global_id: &GlobalId, ) -> Vec { - self.get_member_history(owner) - .into_iter() - .filter(|member| member.get_global_id() == Some(global_id)) - .map(|member| member.get_id()) - .collect() + // The member key is the part of the path below the owner, so the members + // declaring it are one bucket of the owner's history rather than all of + // it. Reading the whole history to filter it built and sorted every + // member the owner has ever held, once per resolution event, against + // paths that gain a member per assignment. + let Some(owner_items) = self.member_owner_key_history_index.get(owner) else { + return Vec::new(); + }; + let name = global_id.get_name(); + // Not the last dotted segment: a bracketed string key keeps its dots, so + // `T["a.b"] = v` stores one member keyed `a.b` under `T`. + let key_text = match owner { + LuaMemberOwner::GlobalPath(owner_id) => name + .strip_prefix(owner_id.get_name()) + .and_then(|rest| rest.strip_prefix('.')) + .unwrap_or(name), + _ => name.rsplit_once('.').map_or(name, |(_, last)| last), + }; + + let mut matched = Vec::new(); + let collect = |key: &LuaMemberKey, matched: &mut Vec| { + let Some(member_ids) = owner_items.get(key) else { + return; + }; + matched.extend(member_ids.iter().copied().filter(|member_id| { + self.get_member(member_id) + .and_then(|member| member.get_global_id()) + == Some(global_id) + })); + }; + collect(&LuaMemberKey::Name(key_text.into()), &mut matched); + // A numeric field is keyed by its integer, not by its spelling. + if let Ok(index) = key_text.parse::() { + collect(&LuaMemberKey::Integer(index), &mut matched); + } + + matched.sort_unstable_by_key(|member_id| member_id_sort_key(*member_id)); + matched.dedup(); + matched } pub(crate) fn iter_current_owner_keys( @@ -993,6 +1233,41 @@ impl LuaMemberIndex { } } + pub fn add_conditional_branch_range( + &mut self, + file_id: FileId, + branch: TextRange, + if_range: TextRange, + ) { + let ranges = self.conditional_branch_ranges.entry(file_id).or_default(); + match ranges.binary_search_by_key(&branch.start(), |(branch, _)| branch.start()) { + Ok(index) | Err(index) => ranges.insert(index, (branch, if_range)), + } + } + + /// Every `if` branch containing `member_id`, each paired with the `if` it + /// belongs to, innermost first. Empty when the write is not inside any + /// branch. + fn enclosing_conditional_branches( + &self, + member_id: LuaMemberId, + ) -> Vec<(TextRange, TextRange)> { + let Some(ranges) = self.conditional_branch_ranges.get(&member_id.file_id) else { + return Vec::new(); + }; + let position = member_id.get_position(); + let mut enclosing = Vec::new(); + let mut index = ranges.partition_point(|(branch, _)| branch.start() <= position); + while index > 0 { + index -= 1; + let entry = ranges[index]; + if entry.0.contains(position) { + enclosing.push(entry); + } + } + enclosing + } + pub fn enclosing_function_scope_range( &self, file_id: FileId, @@ -1298,6 +1573,7 @@ impl LuaMemberIndex { } } self.function_scope_ranges.remove(&file_id); + self.conditional_branch_ranges.remove(&file_id); } } @@ -1331,6 +1607,7 @@ impl LuaIndex for LuaMemberIndex { self.synthesized_owner_members.clear(); self.deferred_index_expr_members.clear(); self.function_scope_ranges.clear(); + self.conditional_branch_ranges.clear(); self.member_function_scope_ranges.clear(); self.assignment_contributions.clear(); } @@ -2102,6 +2379,64 @@ mod tests { assert_eq!(owner_member_ids(&index, &owner).len(), 2); } + #[test] + fn alias_to_owner_is_recorded_exactly_when_the_alias_is_a_no_op() { + let owner = LuaMemberOwner::Type(LuaTypeDeclId::global("SharedTable")); + let other_owner = LuaMemberOwner::Type(LuaTypeDeclId::global("OtherTable")); + let own_member_id = make_index_member_id(FileId::new(1), 10); + let aliased_member_id = make_index_member_id(FileId::new(2), 20); + + let mut index = LuaMemberIndex::new(); + index.add_member( + owner.clone(), + make_member_with_feature(own_member_id, "field", LuaMemberFeature::FileDefine), + ); + index.add_member( + other_owner, + make_member_with_feature(aliased_member_id, "field", LuaMemberFeature::FileDefine), + ); + + // Everything `add_member_alias_to_owner` writes, read back in a + // deterministic order. Comparing the index's `Debug` instead would + // compare `HashSet` iteration order, which varies per process and makes + // the assertion pass or fail at random. + let key = LuaMemberKey::Name("field".into()); + let written_state = |index: &LuaMemberIndex| { + let mut in_filed: Vec = index + .in_filed + .get(&aliased_member_id.file_id) + .map(|objects| objects.iter().map(|object| format!("{object:?}")).collect()) + .unwrap_or_default(); + in_filed.sort(); + ( + index.get_member_item(&owner, &key).cloned(), + index + .member_owner_key_index + .get(&owner) + .and_then(|keys| keys.get(&key)) + .cloned(), + index + .member_owner_key_history_index + .get(&owner) + .and_then(|keys| keys.get(&key)) + .cloned(), + in_filed, + ) + }; + + assert!(!index.alias_to_owner_is_recorded(&owner, aliased_member_id)); + index.add_member_alias_to_owner(owner.clone(), aliased_member_id); + assert!(index.alias_to_owner_is_recorded(&owner, aliased_member_id)); + + let recorded = written_state(&index); + index.add_member_alias_to_owner(owner.clone(), aliased_member_id); + assert_eq!( + recorded, + written_state(&index), + "a recorded alias must write nothing when applied again" + ); + } + #[test] fn alias_adds_to_an_existing_file_define_without_displacing_it() { let owner = LuaMemberOwner::Type(LuaTypeDeclId::global("SharedTable")); @@ -2132,6 +2467,37 @@ mod tests { ); } + #[test] + fn a_table_field_write_arriving_after_an_alias_still_takes_the_slot() { + let owner = LuaMemberOwner::Type(LuaTypeDeclId::global("SharedTable")); + let other_owner = LuaMemberOwner::Type(LuaTypeDeclId::global("OtherTable")); + let key = LuaMemberKey::Name("field".into()); + // A table-literal field is a `FileDefine` that is not an index-expr + // assignment, so it falls through to the latest-defined rule. + let own_member_id = make_member_id(FileId::new(1), 10); + let aliased_member_id = make_index_member_id(FileId::new(2), 20); + + let mut index = LuaMemberIndex::new(); + index.add_member( + other_owner, + make_member_with_feature(aliased_member_id, "field", LuaMemberFeature::FileDefine), + ); + index.add_member_alias_to_owner(owner.clone(), aliased_member_id); + index.add_member( + owner.clone(), + make_member_with_feature(own_member_id, "field", LuaMemberFeature::FileDefine), + ); + + assert_eq!( + index.get_member_item(&owner, &key), + Some(&LuaMemberIndexItem::Many(vec![ + own_member_id, + aliased_member_id, + ])), + "which of the two arrived first must not decide the slot" + ); + } + #[test] fn global_path_key_keeps_every_writer_in_history_while_one_wins_the_visible_slot() { let owner = LuaMemberOwner::GlobalPath(crate::GlobalId::new("cityrp")); @@ -2549,4 +2915,67 @@ mod tests { ); assert_eq!(index.member_function_scope_range(member_id), None); } + + /// `MYTBL["net.handler"] = f` keys one member `net.handler` under `MYTBL`. + /// Its global path is `MYTBL.net.handler`, whose last dotted segment is + /// `handler` — a key nothing is stored under. + #[test] + fn history_for_a_global_path_finds_a_member_whose_key_contains_dots() { + let owner = LuaMemberOwner::GlobalPath(GlobalId::new("MYTBL")); + let global_id = GlobalId::new("MYTBL.net.handler"); + let member_id = make_index_member_id(FileId::new(1), 10); + + let mut index = LuaMemberIndex::new(); + index.add_member( + owner.clone(), + LuaMember::new( + member_id, + LuaMemberKey::Name("net.handler".into()), + LuaMemberFeature::FileFieldDecl, + Some(global_id.clone()), + ), + ); + + assert_eq!( + index.get_member_history_for_global_path(&owner, &global_id), + vec![member_id] + ); + } + + #[test] + fn history_for_a_global_path_still_finds_a_plain_nested_member() { + let owner = LuaMemberOwner::GlobalPath(GlobalId::new("MYTBL.net")); + let global_id = GlobalId::new("MYTBL.net.handler"); + let member_id = make_index_member_id(FileId::new(1), 10); + + let mut index = LuaMemberIndex::new(); + index.add_member( + owner.clone(), + LuaMember::new( + member_id, + LuaMemberKey::Name("handler".into()), + LuaMemberFeature::FileFieldDecl, + Some(global_id.clone()), + ), + ); + + assert_eq!( + index.get_member_history_for_global_path(&owner, &global_id), + vec![member_id] + ); + } + + #[test] + fn an_owner_whose_only_member_is_gone_has_no_live_member() { + let owner = LuaMemberOwner::Type(LuaTypeDeclId::global("OwnedType")); + let file_id = FileId::new(1); + let member_id = make_member_id(file_id, 10); + + let mut index = LuaMemberIndex::new(); + index.add_member(owner.clone(), make_member(member_id, "field")); + assert!(index.has_live_member(&owner)); + + index.remove(file_id); + assert!(!index.has_live_member(&owner)); + } } diff --git a/crates/glua_code_analysis/src/db_index/mod.rs b/crates/glua_code_analysis/src/db_index/mod.rs index e6ac408f0..13b4bd751 100644 --- a/crates/glua_code_analysis/src/db_index/mod.rs +++ b/crates/glua_code_analysis/src/db_index/mod.rs @@ -89,6 +89,22 @@ pub struct DbIndex { /// Invalidated automatically by comparing `Vfs::content_revision`. helper_registry_cache: RevisionedCache, file_helper_scan_cache: HashMap>, + /// Bumped on every *mutable* handle to the type, member, signature or module + /// index; memos over facts derived from those key on it. It covers those four + /// and no others — the decl, global, dynamic-field and gmod-infer indexes all + /// mutate without bumping it, so a memo reading those is not protected here + /// and has to say how it stays correct. Widening a memo's read set means + /// widening this too. May over-invalidate, never misses. + /// Values come from a process-global counter so they are unique across + /// instances (the memos are thread-local and outlive any one `DbIndex`). + type_structure_revision: u64, +} + +/// See [`DbIndex::type_structure_revision`] — process-global so revision values +/// are unique across instances. +fn next_type_structure_revision() -> u64 { + static NEXT: std::sync::atomic::AtomicU64 = std::sync::atomic::AtomicU64::new(1); + NEXT.fetch_add(1, std::sync::atomic::Ordering::Relaxed) } /// Type-erased, revision-keyed cache slot (see `DbIndex::helper_registry_cache`). @@ -114,6 +130,7 @@ impl Default for DbIndex { impl DbIndex { pub fn new() -> Self { Self { + type_structure_revision: next_type_structure_revision(), decl_index: LuaDeclIndex::new(), references_index: LuaReferenceIndex::new(), types_index: LuaTypeIndex::new(), @@ -233,9 +250,16 @@ impl DbIndex { } pub fn get_type_index_mut(&mut self) -> &mut LuaTypeIndex { + self.type_structure_revision = next_type_structure_revision(); &mut self.types_index } + /// See [`Self::type_structure_revision`]. Memos over type/member-derived + /// facts must be discarded when this changes. + pub fn type_structure_revision(&self) -> u64 { + self.type_structure_revision + } + pub fn get_inference_fact(&self, node: &LuaInferenceNodeId) -> Option { match node { LuaInferenceNodeId::TypeOwner(owner) => self.types_index.get_type_fact(owner), @@ -259,6 +283,9 @@ impl DbIndex { &mut self, mut updates: Vec<(LuaInferenceNodeId, LuaTypeFact)>, ) -> HashSet { + // Writes into `types_index` below go direct rather than through + // `get_type_index_mut`, so bump the revision here too. + self.type_structure_revision = next_type_structure_revision(); updates.sort_by(|(left_node, _), (right_node, _)| left_node.stable_cmp(right_node)); let mut conflicting_nodes = HashSet::new(); @@ -315,10 +342,12 @@ impl DbIndex { } pub fn get_module_index_mut(&mut self) -> &mut LuaModuleIndex { + self.type_structure_revision = next_type_structure_revision(); &mut self.modules_index } pub fn get_member_index_mut(&mut self) -> &mut LuaMemberIndex { + self.type_structure_revision = next_type_structure_revision(); &mut self.members_index } @@ -327,6 +356,7 @@ impl DbIndex { } pub fn get_signature_index_mut(&mut self) -> &mut LuaSignatureIndex { + self.type_structure_revision = next_type_structure_revision(); &mut self.signature_index } @@ -511,6 +541,7 @@ impl DbIndex { impl LuaIndex for DbIndex { fn remove(&mut self, file_id: FileId) { + self.type_structure_revision = next_type_structure_revision(); self.decl_index.remove(file_id); self.references_index.remove(file_id); self.types_index.remove(file_id); @@ -537,6 +568,7 @@ impl LuaIndex for DbIndex { } fn remove_files(&mut self, file_ids: &[FileId]) { + self.type_structure_revision = next_type_structure_revision(); if let [file_id] = file_ids { self.remove(*file_id); return; @@ -609,6 +641,7 @@ impl LuaIndex for DbIndex { } fn clear(&mut self) { + self.type_structure_revision = next_type_structure_revision(); self.decl_index.clear(); self.references_index.clear(); self.types_index.clear(); diff --git a/crates/glua_code_analysis/src/db_index/module/mod.rs b/crates/glua_code_analysis/src/db_index/module/mod.rs index 9ce671897..440a9fdc7 100644 --- a/crates/glua_code_analysis/src/db_index/module/mod.rs +++ b/crates/glua_code_analysis/src/db_index/module/mod.rs @@ -11,6 +11,7 @@ pub use module_info::ModuleInfo; pub use module_node::{ModuleNode, ModuleNodeId}; use regex::Regex; use rowan::TextSize; +use rustc_hash::FxHashMap; pub(crate) use workspace::WorkspaceResolutionKey; pub use workspace::{Workspace, WorkspaceId, WorkspaceKind}; @@ -26,13 +27,13 @@ use std::{ pub struct LuaModuleIndex { module_patterns: Vec, module_root_id: ModuleNodeId, - module_nodes: HashMap, - file_module_map: HashMap, - file_module_paths: HashMap, - module_name_to_file_ids: HashMap>, - legacy_module_envs: HashMap>, + module_nodes: FxHashMap, + file_module_map: FxHashMap, + file_module_paths: FxHashMap, + module_name_to_file_ids: FxHashMap>, + legacy_module_envs: FxHashMap>, workspaces: Vec, - workspace_kind_map: HashMap, + workspace_kind_map: FxHashMap, id_counter: u32, fuzzy_search: bool, module_replace_vec: Vec<(Regex, String)>, @@ -50,13 +51,13 @@ impl LuaModuleIndex { let mut index = Self { module_patterns: Vec::new(), module_root_id: ModuleNodeId { id: 0 }, - module_nodes: HashMap::new(), - file_module_map: HashMap::new(), - file_module_paths: HashMap::new(), - module_name_to_file_ids: HashMap::new(), - legacy_module_envs: HashMap::new(), + module_nodes: FxHashMap::default(), + file_module_map: FxHashMap::default(), + file_module_paths: FxHashMap::default(), + module_name_to_file_ids: FxHashMap::default(), + legacy_module_envs: FxHashMap::default(), workspaces: Vec::new(), - workspace_kind_map: HashMap::new(), + workspace_kind_map: FxHashMap::default(), id_counter: 1, fuzzy_search: false, module_replace_vec: Vec::new(), @@ -639,8 +640,12 @@ impl LuaModuleIndex { self.module_nodes.get(module_id) } + /// Sorted by file id: the result reaches completion output, so hash order + /// would let two runs of the same workspace disagree. pub fn get_module_infos(&self) -> Vec<&ModuleInfo> { - self.file_module_map.values().collect() + let mut module_infos: Vec<&ModuleInfo> = self.file_module_map.values().collect(); + module_infos.sort_unstable_by_key(|module_info| module_info.file_id); + module_infos } pub fn get_workspace_kind(&self, workspace_id: WorkspaceId) -> WorkspaceKind { @@ -944,6 +949,7 @@ impl LuaModuleIndex { } } + file_ids.sort_unstable(); file_ids } diff --git a/crates/glua_code_analysis/src/db_index/operators/mod.rs b/crates/glua_code_analysis/src/db_index/operators/mod.rs index 7256c14ba..e4ae317ba 100644 --- a/crates/glua_code_analysis/src/db_index/operators/mod.rs +++ b/crates/glua_code_analysis/src/db_index/operators/mod.rs @@ -1,7 +1,7 @@ mod lua_operator; mod lua_operator_meta_method; -use std::collections::HashMap; +use rustc_hash::FxHashMap; use crate::FileId; @@ -11,10 +11,10 @@ pub use lua_operator_meta_method::LuaOperatorMetaMethod; #[derive(Debug)] pub struct LuaOperatorIndex { - operators: HashMap, + operators: FxHashMap, type_operators_map: - HashMap>>, - in_filed_operator_map: HashMap>, + FxHashMap>>, + in_filed_operator_map: FxHashMap>, } impl Default for LuaOperatorIndex { @@ -26,9 +26,9 @@ impl Default for LuaOperatorIndex { impl LuaOperatorIndex { pub fn new() -> Self { Self { - operators: HashMap::new(), - type_operators_map: HashMap::new(), - in_filed_operator_map: HashMap::new(), + operators: FxHashMap::default(), + type_operators_map: FxHashMap::default(), + in_filed_operator_map: FxHashMap::default(), } } diff --git a/crates/glua_code_analysis/src/db_index/property/mod.rs b/crates/glua_code_analysis/src/db_index/property/mod.rs index bd2d396b3..7170c08f2 100644 --- a/crates/glua_code_analysis/src/db_index/property/mod.rs +++ b/crates/glua_code_analysis/src/db_index/property/mod.rs @@ -2,7 +2,7 @@ mod decl_feature; #[allow(clippy::module_inception)] mod property; -use std::collections::{HashMap, HashSet}; +use rustc_hash::{FxHashMap as HashMap, FxHashSet as HashSet}; pub use decl_feature::{DeclFeatureFlag, PropertyDeclFeature}; use glua_parser::{LuaAstNode, LuaDocTagField, LuaDocType, LuaVersionCondition, VisibilityKind}; @@ -53,12 +53,12 @@ impl LuaPropertyIndex { pub fn new() -> Self { Self { id_count: 0, - in_filed_owner: HashMap::new(), - properties: HashMap::new(), - property_owners_map: HashMap::new(), - signature_owner_by_property: HashMap::new(), - inferred_string_defaults: HashMap::new(), - inferred_string_defaults_file_owners: HashMap::new(), + in_filed_owner: HashMap::default(), + properties: HashMap::default(), + property_owners_map: HashMap::default(), + signature_owner_by_property: HashMap::default(), + inferred_string_defaults: HashMap::default(), + inferred_string_defaults_file_owners: HashMap::default(), } } diff --git a/crates/glua_code_analysis/src/db_index/reference/file_reference.rs b/crates/glua_code_analysis/src/db_index/reference/file_reference.rs index e3739138d..4fdb26eee 100644 --- a/crates/glua_code_analysis/src/db_index/reference/file_reference.rs +++ b/crates/glua_code_analysis/src/db_index/reference/file_reference.rs @@ -1,5 +1,5 @@ use rowan::TextRange; -use std::collections::HashMap; +use rustc_hash::FxHashMap as HashMap; use crate::db_index::LuaDeclId; @@ -18,8 +18,8 @@ impl Default for FileReference { impl FileReference { pub fn new() -> Self { Self { - decl_references: HashMap::new(), - references_to_decl: HashMap::new(), + decl_references: HashMap::default(), + references_to_decl: HashMap::default(), } } diff --git a/crates/glua_code_analysis/src/db_index/reference/mod.rs b/crates/glua_code_analysis/src/db_index/reference/mod.rs index a4ff652f9..204642f9d 100644 --- a/crates/glua_code_analysis/src/db_index/reference/mod.rs +++ b/crates/glua_code_analysis/src/db_index/reference/mod.rs @@ -1,7 +1,7 @@ mod file_reference; mod string_reference; -use std::collections::{HashMap, HashSet}; +use rustc_hash::{FxHashMap as HashMap, FxHashSet as HashSet}; pub use file_reference::{DeclReference, DeclReferenceCell, FileReference}; use glua_parser::LuaSyntaxId; @@ -30,11 +30,11 @@ impl Default for LuaReferenceIndex { impl LuaReferenceIndex { pub fn new() -> Self { Self { - file_references: HashMap::new(), - index_reference: HashMap::new(), - global_references: HashMap::new(), - string_references: HashMap::new(), - type_references: HashMap::new(), + file_references: HashMap::default(), + index_reference: HashMap::default(), + global_references: HashMap::default(), + string_references: HashMap::default(), + type_references: HashMap::default(), } } diff --git a/crates/glua_code_analysis/src/db_index/type/mod.rs b/crates/glua_code_analysis/src/db_index/type/mod.rs index f32c5b2c9..095bd0bdd 100644 --- a/crates/glua_code_analysis/src/db_index/type/mod.rs +++ b/crates/glua_code_analysis/src/db_index/type/mod.rs @@ -477,6 +477,111 @@ fn is_guarded_table_bootstrap_branch(db: &DbIndex, typ: &LuaType) -> bool { } } +/// What a cached type points at, as tracked by [`TypeCacheRefIndex`]. +/// +/// Class references are keyed by declaration id rather than by file: a class's +/// definition sites move as files are indexed, so the file set behind a +/// `Decl` key is resolved from the live declaration at query time. +#[derive(Debug, Clone, Hash, PartialEq, Eq)] +enum TypeCacheRef { + File(FileId), + Decl(LuaTypeDeclId), +} + +/// Reverse map of `referenced thing -> files whose cached types reference it`, +/// so incremental expansion is a lookup instead of a scan over every cache. +#[derive(Debug, Default, PartialEq, Eq)] +struct TypeCacheRefIndex { + owner_refs: HashMap>, + ref_owners: HashMap>, +} + +impl TypeCacheRefIndex { + fn add(&mut self, owner_file_id: FileId, typ: &LuaType) { + let owner_entry = self.owner_refs.entry(owner_file_id).or_default(); + for type_ref in collect_type_cache_refs(typ) { + let count = owner_entry.entry(type_ref.clone()).or_insert(0); + *count += 1; + if *count == 1 { + self.ref_owners + .entry(type_ref) + .or_default() + .insert(owner_file_id); + } + } + } + + fn remove(&mut self, owner_file_id: FileId, typ: &LuaType) { + let Some(owner_entry) = self.owner_refs.get_mut(&owner_file_id) else { + return; + }; + for type_ref in collect_type_cache_refs(typ) { + let Some(count) = owner_entry.get_mut(&type_ref) else { + continue; + }; + *count -= 1; + if *count > 0 { + continue; + } + owner_entry.remove(&type_ref); + if let Some(owners) = self.ref_owners.get_mut(&type_ref) { + owners.remove(&owner_file_id); + if owners.is_empty() { + self.ref_owners.remove(&type_ref); + } + } + } + + if owner_entry.is_empty() { + self.owner_refs.remove(&owner_file_id); + } + } + + fn remove_file(&mut self, owner_file_id: FileId) { + let Some(owner_entry) = self.owner_refs.remove(&owner_file_id) else { + return; + }; + for type_ref in owner_entry.into_keys() { + if let Some(owners) = self.ref_owners.get_mut(&type_ref) { + owners.remove(&owner_file_id); + if owners.is_empty() { + self.ref_owners.remove(&type_ref); + } + } + } + } + + fn owners(&self, type_ref: &TypeCacheRef) -> Option<&HashSet> { + self.ref_owners.get(type_ref) + } +} + +fn collect_type_cache_refs(typ: &LuaType) -> HashSet { + let mut refs = HashSet::default(); + typ.visit_type(&mut |inner| { + match inner { + LuaType::TableConst(range) => { + refs.insert(TypeCacheRef::File(range.file_id)); + } + LuaType::Instance(instance) => { + refs.insert(TypeCacheRef::File(instance.get_range().file_id)); + } + LuaType::Signature(signature_id) => { + refs.insert(TypeCacheRef::File(signature_id.get_file_id())); + } + LuaType::ModuleRef(file_id) => { + refs.insert(TypeCacheRef::File(*file_id)); + } + LuaType::Ref(type_id) | LuaType::Def(type_id) => { + refs.insert(TypeCacheRef::Decl(type_id.clone())); + } + _ => {} + }; + }); + + refs +} + #[derive(Debug)] pub struct LuaTypeIndex { file_namespace: HashMap, @@ -486,6 +591,7 @@ pub struct LuaTypeIndex { generic_params: HashMap>, supers: HashMap>>, types: HashMap, + cache_refs: TypeCacheRefIndex, in_filed_type_owner: HashMap>, fact_metadata: HashMap, definition_facts: HashMap, @@ -509,6 +615,7 @@ impl LuaTypeIndex { generic_params: HashMap::default(), supers: HashMap::default(), types: HashMap::default(), + cache_refs: TypeCacheRefIndex::default(), in_filed_type_owner: HashMap::default(), fact_metadata: HashMap::default(), definition_facts: HashMap::default(), @@ -825,7 +932,7 @@ impl LuaTypeIndex { return; } let file_id = owner.get_file_id(); - let replaced = self.types.insert(owner.clone(), cache).is_some(); + let replaced = self.insert_type_cache(owner.clone(), cache); self.in_filed_type_owner .entry(file_id) .or_default() @@ -837,7 +944,7 @@ impl LuaTypeIndex { pub fn force_bind_type(&mut self, owner: LuaTypeOwner, cache: LuaTypeCache) { let file_id = owner.get_file_id(); - self.types.insert(owner.clone(), cache); + self.insert_type_cache(owner.clone(), cache); self.in_filed_type_owner .entry(file_id) .or_default() @@ -863,7 +970,7 @@ impl LuaTypeIndex { let file_id = owner.get_file_id(); let metadata = metadata.normalized(); - self.types.insert(owner.clone(), cache); + self.insert_type_cache(owner.clone(), cache); self.in_filed_type_owner .entry(file_id) .or_default() @@ -937,7 +1044,7 @@ impl LuaTypeIndex { ) -> FileId { let file_id = owner.get_file_id(); let metadata = metadata.normalized(); - self.types.insert(owner.clone(), cache); + self.insert_type_cache(owner.clone(), cache); self.in_filed_type_owner .entry(file_id) .or_default() @@ -946,6 +1053,18 @@ impl LuaTypeIndex { file_id } + /// Stores `cache`, keeping [`Self::cache_refs`] in step, and reports + /// whether a cache was replaced. + fn insert_type_cache(&mut self, owner: LuaTypeOwner, cache: LuaTypeCache) -> bool { + let file_id = owner.get_file_id(); + self.cache_refs.add(file_id, cache.as_type()); + let Some(previous) = self.types.insert(owner, cache) else { + return false; + }; + self.cache_refs.remove(file_id, previous.as_type()); + true + } + pub(crate) fn bind_definition_fact_unchecked( &mut self, definition: LuaDefinitionId, @@ -1034,7 +1153,7 @@ impl LuaTypeIndex { let mut changed_files = HashSet::default(); for (owner, new_cache) in updates { changed_files.insert(owner.get_file_id()); - self.types.insert(owner, new_cache); + self.insert_type_cache(owner, new_cache); } self.rebuild_inference_derived_state(&changed_files); } @@ -1064,7 +1183,7 @@ impl LuaTypeIndex { let mut changed_files = HashSet::default(); for (owner, new_cache) in updates { changed_files.insert(owner.get_file_id()); - self.types.insert(owner, new_cache); + self.insert_type_cache(owner, new_cache); } self.rebuild_inference_derived_state(&changed_files); } @@ -1074,17 +1193,55 @@ impl LuaTypeIndex { file_ids: &std::collections::HashSet, ) -> HashSet { let mut dependent_files = HashSet::default(); - for (owner, cache) in &self.types { - let owner_file_id = owner.get_file_id(); - if file_ids.contains(&owner_file_id) { - continue; + let mut visited_decls = HashSet::default(); + for file_id in file_ids { + if let Some(owners) = self.cache_refs.owners(&TypeCacheRef::File(*file_id)) { + dependent_files.extend(owners.iter().copied().filter(|o| !file_ids.contains(o))); } - if self.type_references_any_file(cache.as_type(), file_ids, owner_file_id) { - dependent_files.insert(owner_file_id); + // A file that only *names* a class still has to be re-analysed when + // a changed file is one of that class's definition sites: its + // inference reads the class's full member set, which that file + // contributes to. + let Some(decl_ids) = self.file_types.get(file_id) else { + continue; + }; + for decl_id in decl_ids { + if !visited_decls.insert(decl_id) { + continue; + } + + let Some(owners) = self.cache_refs.owners(&TypeCacheRef::Decl(decl_id.clone())) + else { + continue; + }; + let Some(decl) = self.full_name_type_map.get(decl_id) else { + continue; + }; + let locations = decl.get_locations(); + if !locations + .iter() + .any(|location| file_ids.contains(&location.file_id)) + { + continue; + } + + dependent_files.extend(owners.iter().copied().filter(|owner_file_id| { + !file_ids.contains(owner_file_id) + && !locations + .iter() + .any(|location| location.file_id == *owner_file_id) + })); } } + #[cfg(feature = "verify_type_cache_refs")] + assert_eq!( + dependent_files, + self.files_with_type_caches_referencing_files_by_scan(file_ids), + "type cache reverse index disagrees with a full scan" + ); + dependent_files } @@ -1102,6 +1259,31 @@ impl LuaTypeIndex { dependent_files } + /// Reference implementation of + /// [`files_with_type_caches_referencing_files`](Self::files_with_type_caches_referencing_files): + /// the same answer by scanning every cached type. Kept so tests can pin the + /// indexed lookup to it. + #[cfg(any(test, feature = "verify_type_cache_refs"))] + pub fn files_with_type_caches_referencing_files_by_scan( + &self, + file_ids: &std::collections::HashSet, + ) -> HashSet { + let mut dependent_files = HashSet::default(); + for (owner, cache) in &self.types { + let owner_file_id = owner.get_file_id(); + if file_ids.contains(&owner_file_id) { + continue; + } + + if self.type_references_any_file(cache.as_type(), file_ids, owner_file_id) { + dependent_files.insert(owner_file_id); + } + } + + dependent_files + } + + #[cfg(any(test, feature = "verify_type_cache_refs"))] fn type_references_any_file( &self, typ: &LuaType, @@ -1215,6 +1397,7 @@ impl LuaIndex for LuaTypeIndex { self.generic_params.clear(); self.supers.clear(); self.types.clear(); + self.cache_refs = TypeCacheRefIndex::default(); self.in_filed_type_owner.clear(); self.fact_metadata.clear(); self.definition_facts.clear(); @@ -1256,6 +1439,8 @@ impl LuaTypeIndex { self.fact_metadata.remove(&type_owner); } } + + self.cache_refs.remove_file(file_id); } } @@ -1586,6 +1771,7 @@ mod batch_removal_tests { right.inference_events_by_file ); assert_eq!(left.support_file_dependents, right.support_file_dependents); + assert_eq!(left.cache_refs, right.cache_refs); } #[test] diff --git a/crates/glua_code_analysis/src/db_index/type/test.rs b/crates/glua_code_analysis/src/db_index/type/test.rs index a3d902c81..ac7bbf925 100644 --- a/crates/glua_code_analysis/src/db_index/type/test.rs +++ b/crates/glua_code_analysis/src/db_index/type/test.rs @@ -12,7 +12,8 @@ mod test { use crate::{ DbIndex, FileId, InFiled, LuaDeclId, LuaDeclLocation, LuaDefinitionId, LuaInferenceConfidence, LuaInferenceEventId, LuaInferenceNodeId, - LuaInferenceProvenanceKind, LuaInferenceStep, LuaType, LuaTypeCache, LuaTypeDecl, + LuaInferenceProvenanceKind, LuaInferenceStep, LuaSignatureId, LuaType, LuaTypeCache, + LuaTypeDecl, LuaTypeDeclId, LuaTypeFact, LuaTypeFactMetadata, LuaTypeOwner, resolve_alias_type, }; @@ -317,6 +318,100 @@ mod test { ); } + fn class_decl(file_id: FileId, name: &str, type_id: LuaTypeDeclId) -> LuaTypeDecl { + LuaTypeDecl::new( + file_id, + TextRange::new(0.into(), 1.into()), + name.to_string(), + LuaDeclTypeKind::Class, + LuaTypeFlag::None.into(), + type_id, + ) + } + + fn decl_location(file_id: FileId) -> LuaDeclLocation { + LuaDeclLocation { + file_id, + range: TextRange::new(0.into(), 1.into()), + flag: LuaTypeFlag::None.into(), + } + } + + fn assert_reference_lookup_matches_scan(index: &LuaTypeIndex, files: &[FileId]) { + for size in 1..=files.len() { + for window in files.windows(size) { + let query = window.iter().copied().collect::>(); + assert_eq!( + index.files_with_type_caches_referencing_files(&query), + index.files_with_type_caches_referencing_files_by_scan(&query), + "reverse index disagrees with the scan for {window:?}" + ); + } + } + } + + #[test] + fn type_cache_reference_lookup_matches_a_full_scan() { + let files = (1..=6).map(FileId::new).collect::>(); + let (provider, contributor, consumer, table_owner, signature_owner, module_owner) = + (files[0], files[1], files[2], files[3], files[4], files[5]); + let shared_id = LuaTypeDeclId::global("SharedType"); + let provider_id = LuaTypeDeclId::global("ProviderType"); + + let mut index = LuaTypeIndex::new(); + index.add_type_decl(provider, class_decl(provider, "SharedType", shared_id.clone())); + index.add_type_decl_location(contributor, &shared_id, decl_location(contributor)); + index.add_type_decl( + provider, + class_decl(provider, "ProviderType", provider_id.clone()), + ); + + index.bind_type( + owner_in(consumer, 10), + LuaTypeCache::DocType(LuaType::from_vec(vec![ + LuaType::Ref(shared_id.clone()), + LuaType::Def(provider_id.clone()), + ])), + ); + index.bind_type( + owner_in(contributor, 10), + LuaTypeCache::DocType(LuaType::Ref(shared_id.clone())), + ); + index.bind_type( + owner_in(table_owner, 10), + LuaTypeCache::DocType(LuaType::TableConst(InFiled::new( + provider, + TextRange::new(0.into(), 1.into()), + ))), + ); + index.bind_type( + owner_in(signature_owner, 10), + LuaTypeCache::DocType(LuaType::from_vec(vec![ + LuaType::Signature(LuaSignatureId::new(consumer, 5.into())), + LuaType::Ref(shared_id.clone()), + ])), + ); + index.bind_type( + owner_in(module_owner, 10), + LuaTypeCache::DocType(LuaType::ModuleRef(table_owner)), + ); + + assert_reference_lookup_matches_scan(&index, &files); + + // Rebinding must retire the superseded type's references. + index.force_bind_type( + owner_in(signature_owner, 10), + LuaTypeCache::DocType(LuaType::ModuleRef(provider)), + ); + assert_reference_lookup_matches_scan(&index, &files); + + index.remove_files(&[contributor]); + assert_reference_lookup_matches_scan(&index, &files); + + index.remove_files(&[provider, table_owner]); + assert_reference_lookup_matches_scan(&index, &files); + } + #[test] fn ref_type_dependency_excludes_files_that_contribute_to_the_same_type() { let provider = FileId::new(1); diff --git a/crates/glua_code_analysis/src/db_index/type/type_ops/union_type.rs b/crates/glua_code_analysis/src/db_index/type/type_ops/union_type.rs index e858f7243..3df247ddf 100644 --- a/crates/glua_code_analysis/src/db_index/type/type_ops/union_type.rs +++ b/crates/glua_code_analysis/src/db_index/type/type_ops/union_type.rs @@ -1,5 +1,6 @@ use std::ops::Deref; +use crate::db_index::r#type::types::lua_type_sort_key; use crate::{DbIndex, LuaMultiLineUnion, LuaType, LuaUnionType, get_real_type}; // Union member *order* is preserved here, but the member *set* is @@ -44,11 +45,92 @@ pub(crate) fn union_type_all(types: Vec) -> LuaType { return LuaType::from_vec_structural(types); } - let mut result = LuaType::Never; + if visiting_order_is_observable(&types) { + return types.into_iter().fold(LuaType::Never, |result, typ| { + union_type_shallow(&result, &typ) + }); + } + union_all_absorbed(types) +} + +/// `union_type_all`'s fold without the per-step canonicalisation. +/// +/// The pairwise fold rebuilds, de-duplicates and re-sorts the whole accumulated +/// union on every step, so joining n members costs O(n² log n) — and GMod +/// workspaces routinely produce unions with thousands of members (a `pairs()` +/// key type over a large config table, for one). Absorbing into a single member +/// list and canonicalising once gives the same answer for the same reason +/// `from_vec_structural` is safe to call last: the intermediate sorting cannot +/// change which members survive, only the order they are visited in, and the +/// final order comes from that last call either way. +/// +/// Only valid where that visiting order is not observable, which +/// [`visiting_order_is_observable`] decides for the caller. +fn union_all_absorbed(types: Vec) -> LuaType { + let mut members: Vec = Vec::with_capacity(types.len()); for typ in types { - result = union_type_shallow(&result, &typ); + match typ { + // `never` is absorbed by any sibling, so it only survives when it is + // all there is — and then the answer is `never`, not the empty union + // `from_vec_structural` would turn into `nil`. + LuaType::Never => {} + LuaType::Union(union) => { + for member in union.into_vec() { + if !matches!(member, LuaType::Never) { + absorb(&mut members, member); + } + } + } + other => absorb(&mut members, other), + } + } + + if members.is_empty() { + return LuaType::Never; + } + LuaType::from_vec_structural(members) +} + +/// Whether the order `union_type_all` visits members in can change its answer. +/// +/// A `MultiLineUnion` always matters: it matches an incoming literal against its +/// own arms rather than going through the absorption rules, so which side of the +/// join it lands on decides the result. +/// +/// Otherwise the two paths can only disagree about member *order*, and only +/// when both of these hold. Sorting settles it if every member is +/// order-insensitive, since the final `from_vec_structural` orders them anyway. +/// And splicing only happens for a nested union: the pairwise rule joining a +/// plain accumulator to a union puts that union's members *first* and the +/// accumulator last, where absorbing in sequence keeps the accumulator first. +/// With no nested union to splice, the two visit members identically. +fn visiting_order_is_observable(types: &[LuaType]) -> bool { + fn is_multi_line_union(typ: &LuaType) -> bool { + matches!(typ, LuaType::MultiLineUnion(_)) + } + + let union_members = |typ: &LuaType, predicate: &dyn Fn(&LuaType) -> bool| match typ { + LuaType::Union(union) => match union.as_ref() { + LuaUnionType::Nullable(inner) => predicate(inner), + LuaUnionType::Multi(members) => members.iter().any(predicate), + }, + other => predicate(other), + }; + + if types + .iter() + .any(|typ| union_members(typ, &is_multi_line_union)) + { + return true; } - result + + let order_sensitive = types.iter().any(|typ| { + union_members(typ, &|member| { + !LuaUnionType::is_order_insensitive_member(member) + }) + }); + + order_sensitive && types.iter().any(LuaType::is_union) } /// Whether `LuaType::from_vec_structural` alone matches the pairwise fold. @@ -132,6 +214,9 @@ fn union_type_impl(match_source: &LuaType, source: &LuaType, target: &LuaType) - } // union (LuaType::Union(left), right) if !right.is_union() => { + if let Some(merged) = union_sorted_insert(left, source, right) { + return merged; + } let mut members = left.deref().clone().into_vec(); absorb(&mut members, right.clone()); LuaType::from_vec_structural(members) @@ -248,3 +333,241 @@ fn absorb(members: &mut Vec, ty: LuaType) { fn nullable_any_type() -> LuaType { LuaType::Union(LuaUnionType::Nullable(LuaType::Any).into()) } + +/// Adding one member to an already-canonical union, without rebuilding it. +/// +/// The general arm clones every member, rescans them all for something to +/// collapse with (its last rule is a full structural equality), then +/// de-duplicates through a hash set and re-sorts — and the sort key hashes a +/// type's *name*. That is O(n log n) with an expensive constant, paid for every +/// `or` in a chain, and GMod workspaces build unions thousands of members wide: +/// measured on a gamemode edit, this arm alone walked 12.5M members across 23k +/// calls for a single keystroke. +/// +/// `LuaUnionType::from_vec` leaves an order-insensitive union sorted by +/// `lua_type_sort_key`, so for those the same answer is a binary search. Returns +/// `None` whenever that shortcut cannot be justified, leaving the general arm to +/// decide. +fn union_sorted_insert(left: &LuaUnionType, source: &LuaType, right: &LuaType) -> Option { + let LuaUnionType::Multi(members) = left else { + // A `Nullable` is not stored in sort order. + return None; + }; + // `any` and `never` have absorbing rules of their own, and a multi-line + // union matches by value rather than by these rules. + if matches!( + right, + LuaType::Never | LuaType::Any | LuaType::MultiLineUnion(_) + ) || !LuaUnionType::is_order_insensitive_member(right) + { + return None; + } + if !members.iter().all(|member| { + LuaUnionType::is_order_insensitive_member(member) + && !matches!(member, LuaType::Never | LuaType::MultiLineUnion(_)) + }) { + return None; + } + + // Anything `right` could collapse with sorts under a known discriminant, so + // its absence is a binary search rather than a scan. Finding one means a + // merge is due, which the general arm performs. + if collapse_partner_ordinals(right) + .iter() + .any(|ordinal| contains_ordinal(members, *ordinal)) + { + return None; + } + + let key = lua_type_sort_key(right); + match members.binary_search_by(|member| lua_type_sort_key(member).cmp(&key)) { + // Equal sort keys: usually the same member already present, leaving the + // union unchanged. Otherwise two types collided on the key and the + // general arm settles it. + Ok(hit) => { + let mut start = hit; + while start > 0 && lua_type_sort_key(&members[start - 1]) == key { + start -= 1; + } + members[start..] + .iter() + .take_while(|member| lua_type_sort_key(member) == key) + .any(|member| member == right) + .then(|| source.clone()) + } + Err(at) => { + let mut inserted = Vec::with_capacity(members.len() + 1); + inserted.extend_from_slice(&members[..at]); + inserted.push(right.clone()); + inserted.extend_from_slice(&members[at..]); + // Already at least three members, so `from_vec`'s nullable collapse + // cannot apply and this is the order it would have produced. + Some(LuaType::Union(LuaUnionType::Multi(inserted).into())) + } + } +} + +/// The `lua_type_sort_key` discriminants of everything [`try_collapse`] would +/// merge `typ` with, other than an equal member. +fn collapse_partner_ordinals(typ: &LuaType) -> &'static [u8] { + match typ { + LuaType::IntegerConst(_) | LuaType::DocIntegerConst(_) => &[4, 7], + LuaType::FloatConst(_) => &[7], + LuaType::StringConst(_) | LuaType::DocStringConst(_) => &[9], + LuaType::BooleanConst(_) => &[1, 2], + LuaType::TableConst(_) => &[12], + LuaType::DocFunction(_) | LuaType::Signature(_) => &[16], + LuaType::Integer => &[5, 6, 7], + LuaType::Number => &[4, 5, 6, 8], + LuaType::String => &[10, 11], + LuaType::Boolean => &[2], + LuaType::Table => &[13], + LuaType::Function => &[17, 38], + _ => &[], + } +} + +/// Whether a sorted member list holds any type with this sort discriminant. +fn contains_ordinal(members: &[LuaType], ordinal: u8) -> bool { + let at = members.partition_point(|member| lua_type_sort_key(member).0 < ordinal); + members + .get(at) + .is_some_and(|member| lua_type_sort_key(member).0 == ordinal) +} + +#[cfg(test)] +mod union_shortcut_tests { + use super::*; + use crate::LuaTypeDeclId; + use internment::ArcIntern; + use smol_str::SmolStr; + + /// The pairwise fold both shortcuts replace. + fn fold(types: Vec) -> LuaType { + types.into_iter().fold(LuaType::Never, |result, typ| { + union_type_shallow(&result, &typ) + }) + } + + fn sample(pick: u64) -> LuaType { + match pick % 16 { + 0 => LuaType::Nil, + 1 => LuaType::Boolean, + 2 => LuaType::BooleanConst(pick % 32 < 16), + 3 => LuaType::Integer, + 4 => LuaType::IntegerConst((pick % 5) as i64), + 5 => LuaType::Number, + 6 => LuaType::FloatConst((pick % 3) as f64), + 7 => LuaType::String, + 8 => LuaType::StringConst(ArcIntern::new(SmolStr::new(match pick % 4 { + 0 => "a", + 1 => "b", + 2 => "c", + _ => "d", + }))), + 9 => LuaType::Table, + 10 => LuaType::Function, + 11 => LuaType::Userdata, + 12 => LuaType::Thread, + 13 => LuaType::Ref(LuaTypeDeclId::global(match pick % 3 { + 0 => "Alpha", + 1 => "Beta", + _ => "Gamma", + })), + 14 => LuaType::Unknown, + _ => LuaType::Never, + } + } + + fn rng(seed: u64) -> impl FnMut() -> u64 { + let mut state = seed; + move || { + state ^= state << 13; + state ^= state >> 7; + state ^= state << 17; + state + } + } + + /// `union_all_absorbed` exists only to be a faster spelling of the fold, so + /// the thing worth testing is that it never disagrees with it — including + /// the collapses that cascade (`1 | 2 | integer`) and the merges that move a + /// member into an earlier slot. + #[test] + fn absorbing_in_one_pass_matches_the_pairwise_fold() { + let mut next = rng(0x2545_f491_4f6c_dd1d); + for _ in 0..2000 { + let count = (next() % 10) as usize + 1; + let types = (0..count).map(|_| sample(next())).collect::>(); + if visiting_order_is_observable(&types) { + continue; + } + assert_eq!( + union_all_absorbed(types.clone()), + fold(types.clone()), + "absorbed and folded unions disagree for {types:?}" + ); + } + } + + /// Likewise the sorted insert: it has to decline the collapses (a literal + /// meeting its primitive), spot the duplicates, and reproduce the ordering. + #[test] + fn sorted_insert_matches_rebuilding_the_union() { + let mut next = rng(0x9e37_79b9_7f4a_7c15); + let mut exercised = 0; + for _ in 0..4000 { + let count = (next() % 8) as usize + 2; + let members = (0..count).map(|_| sample(next())).collect::>(); + let LuaType::Union(union) = LuaType::from_vec_structural(members) else { + continue; + }; + let incoming = sample(next()); + let source = LuaType::Union(union.clone()); + + let general = { + let mut rebuilt = union.deref().clone().into_vec(); + absorb(&mut rebuilt, incoming.clone()); + LuaType::from_vec_structural(rebuilt) + }; + if let Some(fast) = union_sorted_insert(&union, &source, &incoming) { + exercised += 1; + assert_eq!( + fast, general, + "sorted insert disagreed for {union:?} | {incoming:?}" + ); + } + } + assert!( + exercised > 100, + "fixture never exercised the fast path ({exercised} hits)" + ); + } + + #[test] + fn a_primitive_absorbs_every_literal_of_its_family_at_once() { + let types = vec![ + LuaType::IntegerConst(1), + LuaType::IntegerConst(2), + LuaType::IntegerConst(3), + LuaType::Integer, + ]; + assert_eq!(union_all_absorbed(types.clone()), fold(types)); + } + + #[test] + fn two_different_boolean_literals_collapse_to_boolean() { + let types = vec![LuaType::BooleanConst(true), LuaType::BooleanConst(false)]; + assert_eq!(union_all_absorbed(types.clone()), fold(types)); + } + + #[test] + fn distinct_class_references_are_not_confused_by_sharing_a_variant() { + let types = vec![ + LuaType::Ref(LuaTypeDeclId::global("Alpha")), + LuaType::Ref(LuaTypeDeclId::global("Beta")), + LuaType::Ref(LuaTypeDeclId::global("Alpha")), + ]; + assert_eq!(union_all_absorbed(types.clone()), fold(types)); + } +} diff --git a/crates/glua_code_analysis/src/db_index/type/types.rs b/crates/glua_code_analysis/src/db_index/type/types.rs index d5306bbd8..1b76a9b6d 100644 --- a/crates/glua_code_analysis/src/db_index/type/types.rs +++ b/crates/glua_code_analysis/src/db_index/type/types.rs @@ -497,7 +497,9 @@ impl LuaType { 1 => types[0].clone(), _ => { let mut result_types = Vec::new(); - let mut hash_set = HashSet::new(); + // Membership only: the union's order comes from `result_types`, + // so the hasher cannot affect the result. + let mut hash_set = rustc_hash::FxHashSet::default(); for typ in types { match typ { LuaType::Union(u) => { @@ -1125,7 +1127,7 @@ impl LuaUnionType { /// Callables are matched and rendered in declaration order, and template /// refs drive `` `T` ``|T dispatch, so a union containing either keeps the /// order it was built with. - fn is_order_insensitive_member(typ: &LuaType) -> bool { + pub(crate) fn is_order_insensitive_member(typ: &LuaType) -> bool { !matches!( typ, LuaType::Signature(_) diff --git a/crates/glua_code_analysis/src/diagnostic/checker/access_invisible.rs b/crates/glua_code_analysis/src/diagnostic/checker/access_invisible.rs index adfd14ace..754e684c4 100644 --- a/crates/glua_code_analysis/src/diagnostic/checker/access_invisible.rs +++ b/crates/glua_code_analysis/src/diagnostic/checker/access_invisible.rs @@ -1,4 +1,5 @@ use std::collections::HashSet; +use std::sync::Arc; use glua_parser::{LuaAst, LuaAstNode, LuaAstToken, LuaIndexExpr, LuaNameExpr, VisibilityKind}; use rowan::TextRange; @@ -8,7 +9,10 @@ use crate::{ SemanticDeclLevel, SemanticModel, }; -use super::{Checker, DiagnosticContext}; +use super::{ + Checker, DiagnosticContext, PrecomputedPropertyNameCandidates, + precompute_property_name_candidates, +}; pub struct AccessInvisibleChecker; @@ -17,7 +21,7 @@ impl Checker for AccessInvisibleChecker { fn check(context: &mut DiagnosticContext, semantic_model: &SemanticModel) { let root = semantic_model.get_root().clone(); - let candidates = AccessInvisibleCandidates::new(context); + let candidates = AccessInvisibleCandidates::new(context, semantic_model.get_db()); if candidates.is_empty() { return; } @@ -97,52 +101,35 @@ fn check_index_expr( } struct AccessInvisibleCandidates { - explicit_names: HashSet, + candidates: Arc, private_name_patterns: Vec, } impl AccessInvisibleCandidates { - fn new(context: &DiagnosticContext) -> Self { - let db = context.db; - let mut explicit_names = HashSet::new(); - for (owner_id, property) in db.get_property_index().iter_owner_properties() { - if !property_can_report_access_invisible(property) { - continue; - } - - match owner_id { - LuaSemanticDeclId::LuaDecl(decl_id) => { - if let Some(decl) = db.get_decl_index().get_decl(decl_id) { - explicit_names.insert(decl.get_name().to_string()); - } - } - LuaSemanticDeclId::Member(member_id) => { - if let Some(member) = db.get_member_index().get_member(member_id) - && let Some(name) = member.get_key().get_name() - { - explicit_names.insert(name.to_string()); - } - } - LuaSemanticDeclId::Signature(_) | LuaSemanticDeclId::TypeDecl(_) => {} - } - } - + fn new(context: &DiagnosticContext, db: &crate::DbIndex) -> Self { Self { - explicit_names, - private_name_patterns: context.db.get_emmyrc().doc.private_name.clone(), + candidates: context + .get_shared_data_arc() + .map(|shared_data| shared_data.property_name_candidates.clone()) + .unwrap_or_else(|| Arc::new(precompute_property_name_candidates(db))), + private_name_patterns: db.get_emmyrc().doc.private_name.clone(), } } + fn explicit_names(&self) -> &HashSet { + &self.candidates.access_invisible + } + fn is_empty(&self) -> bool { - self.explicit_names.is_empty() && self.private_name_patterns.is_empty() + self.explicit_names().is_empty() && self.private_name_patterns.is_empty() } fn should_check_name(&self, name: &str) -> bool { - self.explicit_names.contains(name) + self.explicit_names().contains(name) } fn should_check_member_name(&self, name: &str) -> bool { - self.explicit_names.contains(name) || self.matches_private_name_pattern(name) + self.explicit_names().contains(name) || self.matches_private_name_pattern(name) } fn matches_private_name_pattern(&self, name: &str) -> bool { @@ -158,7 +145,7 @@ impl AccessInvisibleCandidates { } } -fn property_can_report_access_invisible(property: &LuaCommonProperty) -> bool { +pub(super) fn property_can_report_access_invisible(property: &LuaCommonProperty) -> bool { !matches!(property.visibility, VisibilityKind::Public) || property.version_conds().is_some() } diff --git a/crates/glua_code_analysis/src/diagnostic/checker/assign_type_mismatch.rs b/crates/glua_code_analysis/src/diagnostic/checker/assign_type_mismatch.rs index 5d114372f..53aca7274 100644 --- a/crates/glua_code_analysis/src/diagnostic/checker/assign_type_mismatch.rs +++ b/crates/glua_code_analysis/src/diagnostic/checker/assign_type_mismatch.rs @@ -465,7 +465,7 @@ fn is_collection_append_write(index_expr: &LuaIndexExpr) -> Option { Some(expr_access_path(&prefix_expr) == expr_access_path(&len_expr)) } -fn expr_access_path(expr: &LuaExpr) -> Option { +fn expr_access_path(expr: &LuaExpr) -> Option { match expr { LuaExpr::NameExpr(name_expr) => name_expr.get_access_path(), LuaExpr::IndexExpr(index_expr) => index_expr.get_access_path(), diff --git a/crates/glua_code_analysis/src/diagnostic/checker/check_export.rs b/crates/glua_code_analysis/src/diagnostic/checker/check_export.rs index 5c91f6d41..e0d64fcdf 100644 --- a/crates/glua_code_analysis/src/diagnostic/checker/check_export.rs +++ b/crates/glua_code_analysis/src/diagnostic/checker/check_export.rs @@ -348,7 +348,7 @@ fn module_source_declares_exported_key( }; local_assigned_keys - .entry(prefix_name_text) + .entry(prefix_name_text.to_string()) .or_default() .insert(index_key.get_path_part()); } @@ -374,7 +374,7 @@ fn module_source_declares_exported_key( }; local_assigned_keys - .entry(prefix_name_text) + .entry(prefix_name_text.to_string()) .or_default() .insert(index_key.get_path_part()); } @@ -384,10 +384,10 @@ fn module_source_declares_exported_key( if !exported_local_names.is_empty() { for name in exported_local_names { - if let Some(keys) = local_table_init_keys.get(&name) { + if let Some(keys) = local_table_init_keys.get(name.as_str()) { exported_keys.extend(keys.iter().cloned()); } - if let Some(keys) = local_assigned_keys.get(&name) { + if let Some(keys) = local_assigned_keys.get(name.as_str()) { exported_keys.extend(keys.iter().cloned()); } } diff --git a/crates/glua_code_analysis/src/diagnostic/checker/check_field.rs b/crates/glua_code_analysis/src/diagnostic/checker/check_field.rs index b4cbe570e..d261522ed 100644 --- a/crates/glua_code_analysis/src/diagnostic/checker/check_field.rs +++ b/crates/glua_code_analysis/src/diagnostic/checker/check_field.rs @@ -5,8 +5,9 @@ use std::{ use glua_parser::{ LuaAssignStat, LuaAst, LuaAstNode, LuaBinaryExpr, LuaCallExpr, LuaElseIfClauseStat, LuaExpr, - LuaForRangeStat, LuaForStat, LuaIfStat, LuaIndexExpr, LuaIndexKey, LuaLocalStat, LuaRepeatStat, - LuaSyntaxKind, LuaSyntaxNode, LuaTokenKind, LuaVarExpr, LuaWhileStat, PathTrait, + LuaForRangeStat, LuaForStat, LuaIfStat, LuaIndexExpr, LuaIndexKey, LuaLocalStat, LuaNameExpr, + LuaRepeatStat, LuaStat, LuaSyntaxKind, LuaSyntaxNode, LuaTokenKind, LuaUnaryExpr, LuaVarExpr, + LuaWhileStat, PathTrait, UnaryOperator, }; use smol_str::SmolStr; @@ -76,6 +77,7 @@ impl Checker for CheckFieldChecker { semantic_model, index_expr, DiagnosticCode::InjectField, + false, &mut state, profile.as_mut(), ); @@ -93,6 +95,7 @@ impl Checker for CheckFieldChecker { semantic_model, &index_expr, DiagnosticCode::InjectField, + false, &mut state, profile.as_mut(), ); @@ -127,6 +130,7 @@ impl Checker for CheckFieldChecker { semantic_model, &index_expr, code, + weak_receiver, &mut state, profile.as_mut(), ); @@ -255,6 +259,7 @@ fn check_index_expr( semantic_model: &SemanticModel, index_expr: &LuaIndexExpr, code: DiagnosticCode, + weak_receiver: bool, state: &mut CheckFieldState, mut profile: Option<&mut CheckFieldProfile>, ) -> Option<()> { @@ -352,7 +357,7 @@ fn check_index_expr( code, DiagnosticCode::UndefinedField | DiagnosticCode::UndefinedMethod ) && !is_enum_type(db, &prefix_typ) - && is_nil_guarded_in_scope(index_expr) + && is_nil_guarded_in_scope(index_expr, weak_receiver) { return Some(()); } @@ -1504,7 +1509,11 @@ fn is_valid_global_path_table_member( .is_some() } -fn global_expr_access_path(db: &DbIndex, file_id: FileId, expr: &LuaExpr) -> Option { +fn global_expr_access_path( + db: &DbIndex, + file_id: FileId, + expr: &LuaExpr, +) -> Option { if !expr_root_is_global(db, file_id, expr) { return None; } @@ -1580,10 +1589,8 @@ fn get_keyof_keys(db: &DbIndex, alias_call: &LuaAliasCallType) -> Option bool { - let target_text = index_expr.syntax().text().to_string(); - // Normalize colon-access to dot-access so that `obj:Method` matches `obj.Method` - let normalized_target = target_text.replacen(':', ".", 1); +fn is_nil_guarded_in_scope(index_expr: &LuaIndexExpr, weak_receiver: bool) -> bool { + let normalized_target = normalize_colon_access(&index_expr.syntax().text().to_string()); let node_range = index_expr.syntax().text_range(); let target_root_name = extract_root_identifier(&normalized_target); @@ -1697,6 +1704,11 @@ fn is_nil_guarded_in_scope(index_expr: &LuaIndexExpr) -> bool { if first_range.contains_range(node_range) { return true; } + // `obj.method and obj:method()` — the left operand + // already tested this same member for presence. + if is_positive_member_test(first, &normalized_target) { + return true; + } } } } @@ -1711,7 +1723,7 @@ fn is_nil_guarded_in_scope(index_expr: &LuaIndexExpr) -> bool { // Pattern: local assignment followed by nil-check of the assigned variable. // e.g., `local x = obj.field; if x then ...` - if is_local_assign_with_nil_check(index_expr, &normalized_target) { + if is_local_assign_with_nil_check(index_expr, weak_receiver) { return true; } @@ -1797,12 +1809,54 @@ fn node_reassigns_root_name(node: &LuaSyntaxNode, root_name: &str) -> bool { false } +/// Colon-access normalized to dot-access, so that `obj:Method` matches `obj.Method`. +/// Every colon is normalized: a chained path such as `obj:Get():Method` has more +/// than one. +fn normalize_colon_access(text: &str) -> String { + text.replace(':', ".") +} + +fn is_not_expr(unary: &LuaUnaryExpr) -> bool { + unary + .get_op_token() + .is_some_and(|op| op.get_op() == UnaryOperator::OpNot) +} + +/// Whether `expr` tests `field_text` itself for presence: the member read alone, +/// or as an operand of `and`/`or`. A negation inverts the test, and a mention in +/// an unrelated call's arguments (`tostring(obj.x)`) does not test anything, so +/// neither vouches for the member. +fn is_positive_member_test(expr: &LuaExpr, field_text: &str) -> bool { + match expr { + LuaExpr::IndexExpr(idx) => { + normalize_colon_access(&idx.syntax().text().to_string()) == field_text + } + LuaExpr::ParenExpr(paren) => paren + .get_expr() + .is_some_and(|inner| is_positive_member_test(&inner, field_text)), + LuaExpr::BinaryExpr(binary) => { + let has_and_or = binary.syntax().children_with_tokens().any(|child| { + let kind = child.kind(); + kind == LuaTokenKind::TkAnd.into() || kind == LuaTokenKind::TkOr.into() + }); + has_and_or + && binary + .syntax() + .children() + .filter_map(LuaExpr::cast) + .any(|operand| is_positive_member_test(&operand, field_text)) + } + _ => false, + } +} + /// Check if a field is being used as a direct truthy/nil check in a condition. -/// Handles: `field`, `not field`, `field or other`, `field and other`, `a or field`, etc. +/// Handles: `field`, `field or other`, `field and other`, `a or field`, +/// `field ~= nil`, and predicate calls taking the field. fn is_truthy_check_in_condition(condition: &LuaExpr, field_text: &str) -> bool { match condition { LuaExpr::IndexExpr(idx) => { - idx.syntax().text().to_string().replacen(':', ".", 1) == field_text + normalize_colon_access(&idx.syntax().text().to_string()) == field_text } LuaExpr::BinaryExpr(binary) => { let has_and_or = binary.syntax().children_with_tokens().any(|child| { @@ -1830,8 +1884,8 @@ fn is_truthy_check_in_condition(condition: &LuaExpr, field_text: &str) -> bool { if exprs.len() == 2 { let lhs = exprs[0].syntax().text().to_string(); let rhs = exprs[1].syntax().text().to_string(); - if (lhs.replacen(':', ".", 1) == field_text && rhs.trim() == "nil") - || (rhs.replacen(':', ".", 1) == field_text && lhs.trim() == "nil") + if (normalize_colon_access(&lhs) == field_text && rhs.trim() == "nil") + || (normalize_colon_access(&rhs) == field_text && lhs.trim() == "nil") { return true; } @@ -1847,6 +1901,9 @@ fn is_truthy_check_in_condition(condition: &LuaExpr, field_text: &str) -> bool { false } LuaExpr::UnaryExpr(unary) => { + if is_not_expr(unary) { + return false; + } for child in unary.syntax().children().filter_map(LuaExpr::cast) { if is_truthy_check_in_condition(&child, field_text) { return true; @@ -1947,7 +2004,7 @@ fn condition_nil_guards_field(condition: &LuaExpr, field_text: &str) -> bool { false } LuaExpr::IndexExpr(idx) => { - idx.syntax().text().to_string().replacen(':', ".", 1) == field_text + normalize_colon_access(&idx.syntax().text().to_string()) == field_text } LuaExpr::ParenExpr(paren) => { if let Some(inner) = paren.get_expr() { @@ -1968,7 +2025,10 @@ fn condition_nil_guards_field(condition: &LuaExpr, field_text: &str) -> bool { false } LuaExpr::UnaryExpr(unary) => { - // Handle `not field` patterns + // `not field` inverts the guard: the body runs when the field is absent. + if is_not_expr(unary) { + return false; + } for child in unary.syntax().children().filter_map(LuaExpr::cast) { if condition_nil_guards_field(&child, field_text) { return true; @@ -1983,20 +2043,40 @@ fn condition_nil_guards_field(condition: &LuaExpr, field_text: &str) -> bool { /// Check if the field access is on the RHS of a local assignment, and the assigned /// variable is nil-checked in a following sibling statement. /// e.g., `local x = obj.field; if x then ...` or `local x = obj.field; if not x then return end` -fn is_local_assign_with_nil_check(index_expr: &LuaIndexExpr, _field_text: &str) -> bool { +fn is_local_assign_with_nil_check(index_expr: &LuaIndexExpr, weak_receiver: bool) -> bool { // Walk up to find the parent LocalStat let local_stat = match index_expr.syntax().ancestors().find_map(LuaLocalStat::cast) { Some(s) => s, None => return false, }; - // Get the variable name assigned in the local statement - let local_names: Vec<_> = local_stat.get_local_name_list().collect(); - if local_names.is_empty() { + // The name the checked expression is bound to, which is the only one a + // guard can speak for. `local a, b = 1, obj.field` binds `b`, not `a`. + let Some(value_expr) = index_expr + .syntax() + .ancestors() + .find(|node| node.parent().as_ref() == Some(local_stat.syntax())) + .and_then(LuaExpr::cast) + else { + return false; + }; + + // The guard tests the bound value, so it only vouches for the member when the + // member read *is* that value. `local x = obj.Missing(...)` binds the call's + // result, and testing a result says nothing about whether the callee resolves. + // An uncertain receiver is exempt: there the checker cannot claim the member is + // missing in the first place. + if !weak_receiver && value_expr.syntax() != index_expr.syntax() { return false; } - let var_name = local_names[0].syntax().text().to_string(); - let var_name = var_name.trim(); + + let Some(var_name) = local_stat + .get_local_name_by_value(value_expr) + .and_then(|name| name.get_name_token()) + else { + return false; + }; + let var_name = var_name.get_name_text(); // Look at following sibling statements (up to 5) for a nil-check of the variable let local_stat_node = local_stat.syntax().clone(); @@ -2014,18 +2094,20 @@ fn is_local_assign_with_nil_check(index_expr: &LuaIndexExpr, _field_text: &str) } continue; } + let kind: LuaSyntaxKind = sibling.kind().into(); + if !LuaStat::can_cast(kind) { + continue; + } checked += 1; if checked > 5 { break; } - let kind: LuaSyntaxKind = sibling.kind().into(); if kind == LuaSyntaxKind::IfStat { // Check if the if-condition references the variable if let Some(if_stat) = LuaIfStat::cast(sibling) { if let Some(cond) = if_stat.get_condition_expr() { - let cond_text = cond.syntax().text().to_string(); - if condition_references_var(&cond_text, var_name) { + if condition_references_var(&cond, var_name) { return true; } } @@ -2076,8 +2158,8 @@ fn is_guarded_by_early_return(index_expr: &LuaIndexExpr, field_text: &str) -> bo if let Some(if_stat) = LuaIfStat::cast(sibling) { // Check if the condition references our field if let Some(cond) = if_stat.get_condition_expr() { - let cond_text = cond.syntax().text().to_string(); - let cond_text_normalized = cond_text.replacen(':', ".", 1); + let cond_text_normalized = + normalize_colon_access(&cond.syntax().text().to_string()); if !cond_text_contains_field_exact(&cond_text_normalized, field_text) { continue; } @@ -2121,16 +2203,18 @@ fn cond_text_contains_field_exact(cond_text: &str, field_text: &str) -> bool { false } -/// Check if a condition text references a variable name. -fn condition_references_var(cond_text: &str, var_name: &str) -> bool { - // Simple text search: the variable appears as a word boundary in the condition - // This handles: `if x then`, `if not x then`, `if x ~= nil then`, predicate guards, etc. - for part in cond_text.split(|c: char| !c.is_alphanumeric() && c != '_') { - if part == var_name { - return true; - } - } - false +/// Whether a condition reads the named variable. Only a name reference counts: +/// `cfg.mode` reads `cfg`, not a local called `mode`. +fn condition_references_var(condition: &LuaExpr, var_name: &str) -> bool { + condition + .syntax() + .descendants() + .filter_map(LuaNameExpr::cast) + .any(|name_expr| { + name_expr + .get_name_text() + .is_some_and(|name| name == var_name) + }) } /// Check if the body of an if-statement contains a return statement. diff --git a/crates/glua_code_analysis/src/diagnostic/checker/code_style/preferred_local_alias.rs b/crates/glua_code_analysis/src/diagnostic/checker/code_style/preferred_local_alias.rs index 0687370a2..c8fc67757 100644 --- a/crates/glua_code_analysis/src/diagnostic/checker/code_style/preferred_local_alias.rs +++ b/crates/glua_code_analysis/src/diagnostic/checker/code_style/preferred_local_alias.rs @@ -100,7 +100,7 @@ fn collect_local_alias( }; local_alias_set.insert( - access_path, + access_path.to_string(), preferred_name.to_string(), semantic_id, ref_var, diff --git a/crates/glua_code_analysis/src/diagnostic/checker/deprecated.rs b/crates/glua_code_analysis/src/diagnostic/checker/deprecated.rs index 300a0b253..81b22f66c 100644 --- a/crates/glua_code_analysis/src/diagnostic/checker/deprecated.rs +++ b/crates/glua_code_analysis/src/diagnostic/checker/deprecated.rs @@ -1,4 +1,4 @@ -use std::collections::HashSet; +use std::sync::Arc; use glua_parser::{LuaAst, LuaAstNode, LuaIndexExpr, LuaNameExpr}; @@ -7,7 +7,10 @@ use crate::{ LuaType, SemanticDeclLevel, SemanticModel, }; -use super::{Checker, DiagnosticContext}; +use super::{ + Checker, DiagnosticContext, PrecomputedPropertyNameCandidates, + precompute_property_name_candidates, +}; pub struct DeprecatedChecker; @@ -16,7 +19,7 @@ impl Checker for DeprecatedChecker { fn check(context: &mut DiagnosticContext, semantic_model: &SemanticModel) { let root = semantic_model.get_root().clone(); - let candidates = DeprecatedCandidates::new(context); + let candidates = DeprecatedCandidates::new(context, semantic_model.get_db()); if candidates.is_empty() { return; } @@ -36,52 +39,29 @@ impl Checker for DeprecatedChecker { } struct DeprecatedCandidates { - names: HashSet, + candidates: Arc, } impl DeprecatedCandidates { - fn new(context: &DiagnosticContext) -> Self { - let db = context.db; - let mut names = HashSet::new(); - for (owner_id, property) in db.get_property_index().iter_owner_properties() { - if !property_can_report_deprecated(property) { - continue; - } - - match owner_id { - LuaSemanticDeclId::LuaDecl(decl_id) => { - if let Some(decl) = db.get_decl_index().get_decl(decl_id) { - names.insert(decl.get_name().to_string()); - } - } - LuaSemanticDeclId::Member(member_id) => { - if let Some(member) = db.get_member_index().get_member(member_id) - && let Some(name) = member.get_key().get_name() - { - names.insert(name.to_string()); - } - } - LuaSemanticDeclId::TypeDecl(type_decl_id) => { - names.insert(type_decl_id.get_name().to_string()); - names.insert(type_decl_id.get_simple_name().to_string()); - } - LuaSemanticDeclId::Signature(_) => {} - } + fn new(context: &DiagnosticContext, db: &crate::DbIndex) -> Self { + Self { + candidates: context + .get_shared_data_arc() + .map(|shared_data| shared_data.property_name_candidates.clone()) + .unwrap_or_else(|| Arc::new(precompute_property_name_candidates(db))), } - - Self { names } } fn is_empty(&self) -> bool { - self.names.is_empty() + self.candidates.deprecated.is_empty() } fn should_check(&self, name: &str) -> bool { - self.names.contains(name) + self.candidates.deprecated.contains(name) } } -fn property_can_report_deprecated(property: &LuaCommonProperty) -> bool { +pub(super) fn property_can_report_deprecated(property: &LuaCommonProperty) -> bool { property.deprecated().is_some() || property.attribute_uses().is_some_and(|attribute_uses| { attribute_uses diff --git a/crates/glua_code_analysis/src/diagnostic/checker/gmod_realm_misuse.rs b/crates/glua_code_analysis/src/diagnostic/checker/gmod_realm_misuse.rs index cd4df58d0..c126cf34a 100644 --- a/crates/glua_code_analysis/src/diagnostic/checker/gmod_realm_misuse.rs +++ b/crates/glua_code_analysis/src/diagnostic/checker/gmod_realm_misuse.rs @@ -228,7 +228,7 @@ impl Checker for GmodRealmMisuseChecker { if let Some(callee_realm) = unknown_realm_candidate(call_realm, &callee_realms) { let call_name = call_expr .get_access_path() - .unwrap_or_else(|| "function".to_string()); + .unwrap_or_else(|| "function".into()); context.add_diagnostic( DiagnosticCode::GmodUnknownRealm, call_expr.get_range(), @@ -282,7 +282,7 @@ impl Checker for GmodRealmMisuseChecker { let call_name = call_expr .get_access_path() - .unwrap_or_else(|| "function".to_string()); + .unwrap_or_else(|| "function".into()); context.add_diagnostic( code, call_expr.get_range(), @@ -658,7 +658,7 @@ fn resolve_global_name_candidate_realms( }; let mut realms = Vec::new(); - let member_key = LuaMemberKey::Name(name.into()); + let member_key = LuaMemberKey::Name(name); if let Some(member_infos) = semantic_model.get_member_info_with_key(&LuaType::Global, member_key, true) { diff --git a/crates/glua_code_analysis/src/diagnostic/checker/inference_trust.rs b/crates/glua_code_analysis/src/diagnostic/checker/inference_trust.rs index 418795fa7..760549bc8 100644 --- a/crates/glua_code_analysis/src/diagnostic/checker/inference_trust.rs +++ b/crates/glua_code_analysis/src/diagnostic/checker/inference_trust.rs @@ -1,9 +1,15 @@ +use glua_parser::{LuaAstNode, LuaIndexExpr, LuaIndexKey, LuaSyntaxId}; + use crate::{ - DiagnosticCode, LuaInferenceProvenanceKind, RenderLevel, SemanticModel, humanize_type, + DiagnosticCode, FileId, InFiled, LuaInferenceProvenanceKind, LuaType, RenderLevel, + SemanticModel, humanize_type, }; use super::{Checker, DiagnosticContext}; +/// A tie lists this many candidate children before it falls back to a count. +const MAX_LISTED_CHILDREN: usize = 3; + pub struct InferenceTrustChecker; impl Checker for InferenceTrustChecker { @@ -53,12 +59,15 @@ impl Checker for InferenceTrustChecker { .and_then(|step| step.found_type.as_deref()) .map_or_else( || "unknown".to_string(), - |typ| { - humanize_type(semantic_model.get_db(), typ, RenderLevel::Simple) - }, + |typ| humanize_type(semantic_model.get_db(), typ, RenderLevel::Simple), ); - format!( - "expected `{typ}` but found `{found}`. Add a guard to narrow the parent to `{typ}`." + unguarded_child_message( + semantic_model, + context.get_file_id(), + &inference.event.source, + inferred_type, + &typ, + &found, ) } else { format!("Type `{typ}` was inferred from usage context and may be incorrect.") @@ -68,3 +77,65 @@ impl Checker for InferenceTrustChecker { } } } + +/// A single winning child can be written into a guard, so it is named directly. +/// A tie cannot: its union is not a type the user can narrow to, so the message +/// names the member that drove the inference and the children that define it. +fn unguarded_child_message( + semantic_model: &SemanticModel, + file_id: FileId, + source: &InFiled, + inferred_type: &LuaType, + inferred_text: &str, + found: &str, +) -> String { + let LuaType::Union(union) = inferred_type else { + return format!( + "expected `{inferred_text}` but found `{found}`. Add a guard to narrow the parent to `{inferred_text}`." + ); + }; + // A union orders its arms by a content hash, so the names are sorted before + // the cap decides which of them the message keeps. + let mut children = union + .types() + .map(|child| humanize_type(semantic_model.get_db(), child, RenderLevel::Simple)) + .collect::>(); + children.sort(); + let listed = children + .iter() + .take(MAX_LISTED_CHILDREN) + .map(|child| format!("`{child}`")) + .collect::>() + .join(", "); + let remaining = children.len().saturating_sub(MAX_LISTED_CHILDREN); + let candidates = if remaining == 0 { + listed + } else { + format!("{listed} and {remaining} more") + }; + match used_member_name(semantic_model, file_id, source) { + Some(member) => format!( + "`{member}` is not defined on `{found}`. Add a guard that narrows the parent to one of {candidates}." + ), + None => format!( + "this member is not defined on `{found}`. Add a guard that narrows the parent to one of {candidates}." + ), + } +} + +fn used_member_name( + semantic_model: &SemanticModel, + file_id: FileId, + source: &InFiled, +) -> Option { + if source.file_id != file_id { + return None; + } + let root = semantic_model.get_root().syntax().clone(); + let index_expr = LuaIndexExpr::cast(source.value.to_node_from_root(&root)?)?; + match index_expr.get_index_key()? { + LuaIndexKey::Name(name) => Some(name.get_name_text().to_string()), + LuaIndexKey::String(string) => Some(string.get_value().to_string()), + _ => None, + } +} diff --git a/crates/glua_code_analysis/src/diagnostic/checker/missing_fields.rs b/crates/glua_code_analysis/src/diagnostic/checker/missing_fields.rs index 7cf6bcb48..60b43c689 100644 --- a/crates/glua_code_analysis/src/diagnostic/checker/missing_fields.rs +++ b/crates/glua_code_analysis/src/diagnostic/checker/missing_fields.rs @@ -353,7 +353,7 @@ fn assigned_target_path(expr: &LuaTableExpr, stat: &LuaStat) -> Option { let index = exprs .iter() .position(|value| value.syntax() == expr.syntax())?; - vars.get(index)?.get_access_path() + vars.get(index)?.get_access_path().map(Into::into) } LuaStat::LocalStat(local) => Some( local diff --git a/crates/glua_code_analysis/src/diagnostic/checker/mod.rs b/crates/glua_code_analysis/src/diagnostic/checker/mod.rs index 94c32e886..5db5cfbe9 100644 --- a/crates/glua_code_analysis/src/diagnostic/checker/mod.rs +++ b/crates/glua_code_analysis/src/diagnostic/checker/mod.rs @@ -31,6 +31,7 @@ mod local_const_reassign; mod missing_fields; mod need_check_nil; mod param_type_check; +mod property_name_candidates; mod readonly_check; mod redefined_local; mod require_module_visibility; @@ -80,6 +81,9 @@ pub use gmod_realm_misuse::precompute_callee_realm_data_for_workspace; pub use gmod_realm_misuse::precompute_gm_method_realms; pub use missing_fields::precompute_missing_required_fields; pub use param_type_check::{PrecomputedParamTypeCandidates, precompute_param_type_candidates}; +pub use property_name_candidates::{ + PrecomputedPropertyNameCandidates, precompute_property_name_candidates, +}; pub type PrecomputedMissingRequiredFields = HashMap>>; pub type AssignmentPrefixKey = (TextSize, TextSize, String); @@ -97,6 +101,18 @@ pub trait Checker { fn check(context: &mut DiagnosticContext, semantic_model: &SemanticModel); } +/// The path a checker is running against, for log lines that would otherwise +/// carry only a `FileId`. +fn checker_file_label(context: &DiagnosticContext, semantic_model: &SemanticModel) -> String { + let file_id = context.get_file_id(); + semantic_model + .get_db() + .get_vfs() + .get_file_path(&file_id) + .map(|path| path.to_string_lossy().to_string()) + .unwrap_or_else(|| format!("{file_id:?}")) +} + fn run_check( context: &mut DiagnosticContext, semantic_model: &SemanticModel, @@ -110,6 +126,15 @@ fn run_check( .iter() .any(|code| context.is_checker_enable_by_code(code)) { + // Named on entry: `checker slow` only reports on completion. + if log::log_enabled!(log::Level::Trace) { + log::trace!( + "checker start: {} for {}", + std::any::type_name::(), + checker_file_label(context, semantic_model) + ); + } + if !log::log_enabled!(log::Level::Info) { T::check(context, semantic_model); return; @@ -124,12 +149,13 @@ fn run_check( .map(|c| c.get_name()) .collect::>() .join(","); + let path = checker_file_label(context, semantic_model); log::info!( - "checker slow: {}({}) cost {:?} for {:?}", + "checker slow: {}({}) cost {:?} for {}", std::any::type_name::(), name, elapsed, - context.get_file_id() + path ); } } @@ -268,6 +294,8 @@ pub struct SharedDiagnosticData { pub param_type_candidates: Arc, /// Static callee names whose signatures are marked @nodiscard. pub nodiscard_candidates: Arc, + /// Names the deprecated, readonly and visibility checkers could report on. + pub property_name_candidates: Arc, /// Precomputed declaration annotation realms for all workspace files. /// Avoids re-scanning syntax trees for @realm annotations per file. pub decl_annotation_realms: Arc>>, diff --git a/crates/glua_code_analysis/src/diagnostic/checker/param_type_check.rs b/crates/glua_code_analysis/src/diagnostic/checker/param_type_check.rs index f67c6704c..0b1552b48 100644 --- a/crates/glua_code_analysis/src/diagnostic/checker/param_type_check.rs +++ b/crates/glua_code_analysis/src/diagnostic/checker/param_type_check.rs @@ -895,7 +895,7 @@ fn rewritten_collection_element_matches_param( last_matching_assignment_is_compatible == Some(true) } -fn expr_access_path(expr: &LuaExpr) -> Option { +fn expr_access_path(expr: &LuaExpr) -> Option { match expr { LuaExpr::NameExpr(name_expr) => name_expr.get_access_path(), LuaExpr::IndexExpr(index_expr) => index_expr.get_access_path(), diff --git a/crates/glua_code_analysis/src/diagnostic/checker/property_name_candidates.rs b/crates/glua_code_analysis/src/diagnostic/checker/property_name_candidates.rs new file mode 100644 index 000000000..6bc420b64 --- /dev/null +++ b/crates/glua_code_analysis/src/diagnostic/checker/property_name_candidates.rs @@ -0,0 +1,88 @@ +use std::collections::HashSet; + +use crate::{DbIndex, LuaSemanticDeclId}; + +use super::access_invisible::property_can_report_access_invisible; +use super::deprecated::property_can_report_deprecated; +use super::readonly_check::property_can_report_readonly; + +/// The names a property-driven checker could possibly report on. Derived from +/// the property index, so it is the same for every file in a run. +#[derive(Debug, Default)] +pub struct PrecomputedPropertyNameCandidates { + pub deprecated: HashSet, + pub readonly: HashSet, + pub access_invisible: HashSet, +} + +pub fn precompute_property_name_candidates(db: &DbIndex) -> PrecomputedPropertyNameCandidates { + let mut candidates = PrecomputedPropertyNameCandidates::default(); + + for (owner_id, property) in db.get_property_index().iter_owner_properties() { + let deprecated = property_can_report_deprecated(property); + let readonly = property_can_report_readonly(property); + let access_invisible = property_can_report_access_invisible(property); + if !deprecated && !readonly && !access_invisible { + continue; + } + + match owner_id { + LuaSemanticDeclId::LuaDecl(decl_id) => { + let Some(decl) = db.get_decl_index().get_decl(decl_id) else { + continue; + }; + let name = decl.get_name(); + if deprecated { + candidates.deprecated.insert(name.to_string()); + } + if readonly { + candidates.readonly.insert(name.to_string()); + } + if access_invisible { + candidates.access_invisible.insert(name.to_string()); + } + } + LuaSemanticDeclId::Member(member_id) => { + let Some(name) = db + .get_member_index() + .get_member(member_id) + .and_then(|member| member.get_key().get_name()) + else { + continue; + }; + if deprecated { + candidates.deprecated.insert(name.to_string()); + } + if readonly { + candidates.readonly.insert(name.to_string()); + } + if access_invisible { + candidates.access_invisible.insert(name.to_string()); + } + } + // A type declaration names no runtime access, so the visibility + // checker does not take it. + LuaSemanticDeclId::TypeDecl(type_decl_id) => { + if deprecated { + candidates + .deprecated + .insert(type_decl_id.get_name().to_string()); + candidates + .deprecated + .insert(type_decl_id.get_simple_name().to_string()); + } + if readonly { + candidates + .readonly + .insert(type_decl_id.get_name().to_string()); + candidates + .readonly + .insert(type_decl_id.get_simple_name().to_string()); + } + } + LuaSemanticDeclId::Signature(_) => {} + } + } + + candidates +} diff --git a/crates/glua_code_analysis/src/diagnostic/checker/readonly_check.rs b/crates/glua_code_analysis/src/diagnostic/checker/readonly_check.rs index 0f6eedd68..a26c82550 100644 --- a/crates/glua_code_analysis/src/diagnostic/checker/readonly_check.rs +++ b/crates/glua_code_analysis/src/diagnostic/checker/readonly_check.rs @@ -1,4 +1,4 @@ -use std::collections::HashSet; +use std::sync::Arc; use glua_parser::{ LuaAssignStat, LuaAst, LuaAstNode, LuaExpr, LuaIndexKey, LuaSyntaxId, LuaSyntaxKind, @@ -10,7 +10,10 @@ use crate::{ PropertyDeclFeature, SemanticDeclLevel, SemanticModel, }; -use super::{Checker, DiagnosticContext}; +use super::{ + Checker, DiagnosticContext, PrecomputedPropertyNameCandidates, + precompute_property_name_candidates, +}; pub struct ReadOnlyChecker; @@ -19,7 +22,7 @@ impl Checker for ReadOnlyChecker { fn check(context: &mut DiagnosticContext, semantic_model: &SemanticModel) { let root = semantic_model.get_root().clone(); - let candidates = ReadOnlyCandidates::new(context); + let candidates = ReadOnlyCandidates::new(context, semantic_model.get_db()); if candidates.is_empty() { return; } @@ -40,44 +43,25 @@ impl Checker for ReadOnlyChecker { } struct ReadOnlyCandidates { - names: HashSet, + candidates: Arc, } impl ReadOnlyCandidates { - fn new(context: &DiagnosticContext) -> Self { - let db = context.db; - let mut names = HashSet::new(); - for (owner_id, property) in db.get_property_index().iter_owner_properties() { - if !property_can_report_readonly(property) { - continue; - } - - match owner_id { - LuaSemanticDeclId::LuaDecl(decl_id) => { - if let Some(decl) = db.get_decl_index().get_decl(decl_id) { - names.insert(decl.get_name().to_string()); - } - } - LuaSemanticDeclId::Member(member_id) => { - if let Some(member) = db.get_member_index().get_member(member_id) - && let Some(name) = member.get_key().get_name() - { - names.insert(name.to_string()); - } - } - LuaSemanticDeclId::TypeDecl(type_decl_id) => { - names.insert(type_decl_id.get_name().to_string()); - names.insert(type_decl_id.get_simple_name().to_string()); - } - LuaSemanticDeclId::Signature(_) => {} - } + fn new(context: &DiagnosticContext, db: &crate::DbIndex) -> Self { + Self { + candidates: context + .get_shared_data_arc() + .map(|shared_data| shared_data.property_name_candidates.clone()) + .unwrap_or_else(|| Arc::new(precompute_property_name_candidates(db))), } + } - Self { names } + fn names(&self) -> &std::collections::HashSet { + &self.candidates.readonly } fn is_empty(&self) -> bool { - self.names.is_empty() + self.names().is_empty() } fn should_check_expr(&self, expr: &LuaExpr) -> bool { @@ -87,7 +71,7 @@ impl ReadOnlyCandidates { LuaExpr::NameExpr(name_expr) => { return name_expr .get_name_text() - .is_some_and(|name| self.names.contains(name.as_ref() as &str)); + .is_some_and(|name| self.names().contains(name.as_ref() as &str)); } LuaExpr::IndexExpr(index_expr) => { if let Some(index_key) = index_expr.get_index_key() @@ -109,18 +93,18 @@ impl ReadOnlyCandidates { match index_key { LuaIndexKey::Name(name) => { let name = name.get_name_text(); - self.names.contains(name) + self.names().contains(name) } LuaIndexKey::String(string) => { let value = string.get_value(); - self.names.contains(value.as_str()) + self.names().contains(value.as_str()) } LuaIndexKey::Integer(_) | LuaIndexKey::Idx(_) | LuaIndexKey::Expr(_) => false, } } } -fn property_can_report_readonly(property: &LuaCommonProperty) -> bool { +pub(super) fn property_can_report_readonly(property: &LuaCommonProperty) -> bool { property .decl_features .has_feature(PropertyDeclFeature::ReadOnly) diff --git a/crates/glua_code_analysis/src/diagnostic/lua_diagnostic.rs b/crates/glua_code_analysis/src/diagnostic/lua_diagnostic.rs index 69c37ea12..c04ce5855 100644 --- a/crates/glua_code_analysis/src/diagnostic/lua_diagnostic.rs +++ b/crates/glua_code_analysis/src/diagnostic/lua_diagnostic.rs @@ -13,6 +13,7 @@ use super::checker::precompute_gm_method_realms; use super::checker::precompute_missing_required_fields; use super::checker::precompute_nodiscard_candidates; use super::checker::precompute_param_type_candidates; +use super::checker::precompute_property_name_candidates; use super::checker::precompute_sorted_send_flows; use super::{checker::check_file, lua_diagnostic_config::LuaDiagnosticConfig}; use crate::semantic::LuaAnalysisPhase; @@ -136,6 +137,7 @@ impl LuaDiagnostic { nodiscard_candidates, decl_annotation_realms, sorted_send_flows, + property_name_candidates, ) = std::thread::scope(|s| { let workspace_realms = s.spawn(|| { let mut gm_method_realms = HashMap::new(); @@ -169,6 +171,7 @@ impl LuaDiagnostic { let await_c = s.spawn(|| precompute_await_candidates(db)); let param_type = s.spawn(|| precompute_param_type_candidates(db)); let nodiscard = s.spawn(|| precompute_nodiscard_candidates(db)); + let property_names = s.spawn(|| precompute_property_name_candidates(db)); let decl_realms = s.spawn(|| precompute_decl_annotation_realms(db, workspace_file_ids_ref)); let send_flows = @@ -197,6 +200,9 @@ impl LuaDiagnostic { .join() .expect("precompute_sorted_send_flows panicked"), ), + property_names + .join() + .expect("precompute_property_name_candidates panicked"), ) }); let (gm_method_realms, callee_realms_by_workspace, realm_call_candidates_by_workspace) = @@ -210,6 +216,7 @@ impl LuaDiagnostic { await_candidates: Arc::new(await_candidates), param_type_candidates: Arc::new(param_type_candidates), nodiscard_candidates: Arc::new(nodiscard_candidates), + property_name_candidates: Arc::new(property_name_candidates), decl_annotation_realms: Arc::new(decl_annotation_realms), sorted_send_flows, }) diff --git a/crates/glua_code_analysis/src/diagnostic/test/undefined_field_test.rs b/crates/glua_code_analysis/src/diagnostic/test/undefined_field_test.rs index ed53529ab..5975c8cb8 100644 --- a/crates/glua_code_analysis/src/diagnostic/test/undefined_field_test.rs +++ b/crates/glua_code_analysis/src/diagnostic/test/undefined_field_test.rs @@ -4612,6 +4612,24 @@ owner:CompletelyMadeUpMethod() )); } + /// The guard has to name the local the field is bound to. Here the check is + /// on `a`, which is bound to `1`, so it says nothing about `unknownField`. + #[test] + fn test_nil_guard_local_assign_binds_the_checked_name() { + let mut ws = VirtualWorkspace::new(); + assert!(!ws.check_code_for( + DiagnosticCode::UndefinedField, + r#" + ---@class LocalAssignBindTest + local obj = {} + local a, b = 1, obj.unknownField + if a then + print(a, b) + end + "# + )); + } + #[test] fn test_nil_guard_early_return() { let mut ws = VirtualWorkspace::new(); @@ -5560,4 +5578,201 @@ owner:CompletelyMadeUpMethod() "unexpected UndefinedMethod diagnostics: {diagnostics:#?}" ); } + + fn def_nil_guard_scope_fixture(ws: &mut VirtualWorkspace) { + ws.def( + r#" + ---@meta + + ---@class GuardOwner + function GuardOwner:IsAlive() end + + ---@class GuardEnt + ---@field owner GuardOwner + function GuardEnt:IsAlive() end + + ---@return GuardOwner + function GuardEnt:GetOwner() end + "#, + ); + } + + #[test] + fn test_local_assign_guard_does_not_cover_dot_call() { + let mut ws = VirtualWorkspace::new(); + def_nil_guard_scope_fixture(&mut ws); + assert!(!ws.check_code_for( + DiagnosticCode::UndefinedField, + r#" + ---@param v GuardEnt + local function use(v) + local tr = v.MissingDotCall(v) + if tr then print(tr) end + end + "#, + )); + } + + #[test] + fn test_local_assign_guard_covers_plain_field_read() { + let mut ws = VirtualWorkspace::new(); + def_nil_guard_scope_fixture(&mut ws); + assert!(ws.check_code_for( + DiagnosticCode::UndefinedField, + r#" + ---@param v GuardEnt + local function use(v) + local tr = v.owner.NoSuchDotField + if tr then print(tr) end + end + "#, + )); + } + + #[test] + fn test_local_assign_guard_covers_weak_receiver_call() { + let mut ws = VirtualWorkspace::new(); + def_valid_guard_fixture(&mut ws); + assert!(ws.check_code_for( + DiagnosticCode::UndefinedField, + r#" + local function use(ent) + local phys = ent:GetPhysicsObject() + if phys then phys:SetMass(100) end + end + "#, + )); + } + + #[test] + fn test_local_assign_guard_does_not_escape_closure() { + let mut ws = VirtualWorkspace::new(); + def_nil_guard_scope_fixture(&mut ws); + assert!(!ws.check_code_for( + DiagnosticCode::UndefinedField, + r#" + ---@param v GuardEnt + local function use(v) + local cb = function() print(v.missingField) end + if cb then print(cb) end + end + "#, + )); + } + + #[test] + fn test_local_assign_guard_ignores_dotted_name_collision() { + let mut ws = VirtualWorkspace::new(); + def_nil_guard_scope_fixture(&mut ws); + assert!(!ws.check_code_for( + DiagnosticCode::UndefinedField, + r#" + ---@param v GuardEnt + ---@param cfg table + local function use(v, cfg) + local mode = v.NoSuchField + if cfg.mode then print(mode) end + end + "#, + )); + } + + #[test] + fn test_local_assign_guard_accepts_matching_name_reference() { + let mut ws = VirtualWorkspace::new(); + def_nil_guard_scope_fixture(&mut ws); + assert!(ws.check_code_for( + DiagnosticCode::UndefinedField, + r#" + ---@param v GuardEnt + local function use(v) + local mode = v.NoSuchField + if mode then print(mode) end + end + "#, + )); + } + + #[test] + fn test_negated_condition_does_not_guard_body() { + let mut ws = VirtualWorkspace::new(); + def_nil_guard_scope_fixture(&mut ws); + assert!(!ws.check_code_for( + DiagnosticCode::UndefinedMethod, + r#" + ---@param v GuardEnt + local function use(v) + if not v.Foo then + v:Foo() + end + end + "#, + )); + } + + #[test] + fn test_local_assign_guard_lookback_ignores_comments() { + let mut ws = VirtualWorkspace::new(); + def_nil_guard_scope_fixture(&mut ws); + assert!(ws.check_code_for( + DiagnosticCode::UndefinedField, + r#" + ---@param v GuardEnt + local function use(v) + local tr = v.NoSuchField + + -- one + + -- two + + -- three + + -- four + + -- five + + if tr then print(tr) end + end + "#, + )); + } + + #[test] + fn test_early_return_guard_matches_chained_colon_path() { + let mut ws = VirtualWorkspace::new(); + def_nil_guard_scope_fixture(&mut ws); + let file_id = ws.def( + r#" + ---@param v GuardEnt + local function use(v) + if not v:IsAlive() or not v:GetOwner():NoSuchMethod() then return end + v:GetOwner():NoSuchMethod() + end + "#, + ); + let fields = diagnostics_for_code(&mut ws, file_id, DiagnosticCode::UndefinedField); + let methods = diagnostics_for_code(&mut ws, file_id, DiagnosticCode::UndefinedMethod); + assert_eq!( + fields.len() + methods.len(), + 1, + "fields: {fields:#?}\nmethods: {methods:#?}" + ); + } + + #[test] + fn test_call_argument_mention_is_not_a_guard() { + let mut ws = VirtualWorkspace::new(); + def_nil_guard_scope_fixture(&mut ws); + let file_id = ws.def( + r#" + ---@param v GuardEnt + local function use(v) + local ok = tostring(v.NoSuchField) and print(v.NoSuchField) + return ok + end + "#, + ); + let fields = diagnostics_for_code(&mut ws, file_id, DiagnosticCode::UndefinedField); + assert_eq!(fields.len(), 1, "{fields:#?}"); + } } diff --git a/crates/glua_code_analysis/src/diagnostic/test/undefined_method_test.rs b/crates/glua_code_analysis/src/diagnostic/test/undefined_method_test.rs index 9767a3265..1f302e7a4 100644 --- a/crates/glua_code_analysis/src/diagnostic/test/undefined_method_test.rs +++ b/crates/glua_code_analysis/src/diagnostic/test/undefined_method_test.rs @@ -1506,4 +1506,40 @@ mod tests { .unwrap(); assert_eq!(diagnostic.severity, Some(DiagnosticSeverity::ERROR)); } + + /// A nil check on a call result speaks for the value, not for the method + /// name that produced it, so it cannot excuse an undefined method. + #[test] + fn nil_check_of_the_call_result_does_not_excuse_the_method() { + let diagnostics = gmod_diagnostics( + r#" + ---@class Entity + ---@class Holder + ---@field owner Entity + ---@type Holder + local holder = nil + local trace = holder.owner:MissingEntityMethod() + if trace.Entity then print(1) end + "#, + ); + + assert!(has_code(&diagnostics, DiagnosticCode::UndefinedMethod)); + } + + /// `obj.method and obj:method()` tests the member itself before calling it, + /// which is a presence check the diagnostic must respect. + #[test] + fn short_circuit_presence_check_excuses_the_call_it_guards() { + let diagnostics = gmod_diagnostics( + r#" + ---@class Entity + ---@type Entity + local ent = nil + local owned = ent.CPPIGetOwner and ent:CPPIGetOwner() == nil + print(owned) + "#, + ); + + assert!(!has_code(&diagnostics, DiagnosticCode::UndefinedMethod)); + } } diff --git a/crates/glua_code_analysis/src/lib.rs b/crates/glua_code_analysis/src/lib.rs index 077f43e5f..61408a8cb 100644 --- a/crates/glua_code_analysis/src/lib.rs +++ b/crates/glua_code_analysis/src/lib.rs @@ -15,7 +15,8 @@ mod db_index; mod diagnostic; mod gamemode_base; mod library_collision; -mod profile; +pub mod profile; +pub mod progress; mod resources; mod semantic; mod test_lib; @@ -512,67 +513,22 @@ impl EmmyLuaAnalysis { return Some(file_id); } - let is_removed = text.is_none(); - let removed_file_ids = existing_file_id - .filter(|_| is_removed) - .into_iter() - .collect::>(); - let mut existing_reindex_file_ids = profile::phase("edit/expand", || { + // The expansion has to be derived before the new text lands, because + // re-indexing a file drops the record of what depends on it. + let existing_reindex_file_ids = profile::phase("edit/expand", || { existing_file_id.map(|file_id| self.expand_reindex_file_ids(vec![file_id])) }); - if let Some(reindex_file_ids) = &mut existing_reindex_file_ids { - self.add_vgui_forwarding_removal_seed(&removed_file_ids, reindex_file_ids); - } - let old_guard_fact_file_ids = existing_reindex_file_ids - .iter() - .flatten() - .copied() - .collect::>(); - let old_guard_facts = profile::phase("edit/guard_snapshot", || { - self.inferred_guard_snapshot(&old_guard_fact_file_ids) - }); let file_id = self .compilation .get_db_mut() .get_vfs_mut() .set_file_content(uri, text); - let incremental_source_file_ids = HashSet::from([file_id]); - let reindex_file_ids = existing_reindex_file_ids + let expansion = existing_reindex_file_ids .unwrap_or_else(|| self.expand_reindex_file_ids(vec![file_id])); - profile::phase("edit/remove_index", || { - self.compilation.remove_index(reindex_file_ids.clone()) - }); - - let update_file_ids = reindex_file_ids - .iter() - .copied() - .filter(|id| !is_removed || *id != file_id) - .collect::>(); - if !update_file_ids.is_empty() { - profile::phase("edit/update_index", || { - self.compilation.update_index(update_file_ids.clone()) - }); - profile::phase("edit/stabilize_type_caches", || { - self.stabilize_cross_file_type_caches(&update_file_ids) - }); - } - self.compilation - .get_db_mut() - .get_call_site_param_index_mut() - .refresh_file_source_dependencies(file_id); - let guard_fact_file_ids = reindex_file_ids.iter().copied().collect::>(); - profile::phase("edit/guard_reference_reindex", || { - self.reindex_changed_inferred_guard_references( - &guard_fact_file_ids, - &old_guard_facts, - &reindex_file_ids, - &incremental_source_file_ids, - ) - }); - profile::phase("edit/param_consumer_reindex", || { - self.reindex_changed_inferred_param_consumers(&old_guard_facts, &reindex_file_ids) + profile::phase("edit/reindex", || { + self.reindex_expanded_files(vec![file_id], expansion) }); profile::phase_report("update_file_by_uri"); @@ -776,6 +732,22 @@ impl EmmyLuaAnalysis { /// Reindex specific files: remove old index entries + run full analysis pipeline. /// Call this after `update_file_text_only` once the user has paused typing. pub fn reindex_files(&mut self, file_ids: Vec) { + let expansion = self.expand_reindex_file_ids(file_ids.clone()); + self.reindex_expanded_files(file_ids, expansion); + } + + /// [`reindex_files`](Self::reindex_files) against an expansion that was + /// computed earlier. + /// + /// The expansion has to be derived from the state *before* the edit landed, + /// so a caller that wants to do anything in between — re-index the edited + /// file on its own first, say, and release the write lock so a completion + /// can be answered — has to capture it up front and hand it back here. + /// Recomputing it against a partly-updated index under-expands badly: + /// measured on a gamemode workspace, an expansion of 739 files collapsed to + /// 8 and the workspace ended up with 18 diagnostics that a cold build does + /// not produce. + pub fn reindex_expanded_files(&mut self, file_ids: Vec, expansion: Vec) { let incremental_source_file_ids = file_ids.iter().copied().collect::>(); let removed_file_ids = file_ids .iter() @@ -788,13 +760,21 @@ impl EmmyLuaAnalysis { .is_none() }) .collect::>(); - let mut file_ids = self.expand_reindex_file_ids(file_ids); + + let mut file_ids = expansion; self.add_vgui_forwarding_removal_seed(&removed_file_ids, &mut file_ids); let guard_fact_file_ids = file_ids.iter().copied().collect::>(); let old_guard_facts = self.inferred_guard_snapshot(&guard_fact_file_ids); self.compilation.remove_index(file_ids.clone()); - self.compilation.update_index(file_ids.clone()); - self.stabilize_cross_file_type_caches(&file_ids); + let update_file_ids = file_ids + .iter() + .copied() + .filter(|file_id| !removed_file_ids.contains(file_id)) + .collect::>(); + if !update_file_ids.is_empty() { + self.compilation.update_index(update_file_ids.clone()); + self.stabilize_cross_file_type_caches(&update_file_ids); + } for file_id in &incremental_source_file_ids { self.compilation .get_db_mut() @@ -810,6 +790,21 @@ impl EmmyLuaAnalysis { self.reindex_changed_inferred_param_consumers(&old_guard_facts, &file_ids); } + /// Rebuilds only these files' own index entries. + /// + /// Nothing cross-file is settled: dependents keep whatever they inferred + /// before, and the caller still owes them a + /// [`reindex_expanded_files`](Self::reindex_expanded_files) against an + /// expansion captured beforehand. What this does buy is that the edited + /// file's declarations, members and signatures line up with its text again, + /// which is all a request positioned *inside that file* needs — the index + /// entries are keyed by position, so an edit that shifts offsets is exactly + /// what makes them stop matching the tree. + pub fn self_index_files(&mut self, file_ids: Vec) { + self.compilation.remove_index(file_ids.clone()); + self.compilation.update_index(file_ids); + } + /// Re-analyses exactly `file_ids`, skipping dependency expansion. pub fn reindex_files_without_expansion(&mut self, file_ids: Vec) { self.compilation.remove_index(file_ids.clone()); @@ -2359,6 +2354,7 @@ mod tests { child: GmodVguiParentSource::Unknown, parent: GmodVguiParentSource::Unknown, relations: Vec::new(), + resolved_source: None, origin: GmodVguiParentCallOrigin::Annotated, }, ); @@ -2370,6 +2366,106 @@ mod tests { assert_eq!(reindex_file_ids, vec![main_file_id, helper_file_id]); } + /// The language server re-indexes an edited file on its own before running + /// its dependency ripple, so a completion positioned in that file can be + /// answered without waiting seconds for the ripple. This checks the split + /// path reaches the same diagnostics as doing it in one go. + /// + /// It does **not** pin the ordering constraint that makes the split safe — + /// that the expansion is captured *before* the self-index, because a + /// self-index drops the edited file's declarations and inbound dependency + /// edges and an expansion taken afterwards under-invalidates. That was + /// measured on a gamemode workspace (739 files before the self-index, 6 + /// after) and this fixture is far too small to reproduce it: it passes with + /// the two swapped. The real guard is `tools/determinism` against a real + /// workspace, and the reasoning lives on `reindex_expanded_files`. + #[test] + fn two_phase_reindex_matches_single_phase_diagnostics() { + // A class definition plus a consumer whose inferred type references it: + // the expansion reaches the consumer through the type-cache relation, + // which is the one that collapses once the definition site is dropped. + let producer_source = |member: &str| { + format!( + "---@class Thing +local Thing = {{}} +function Thing:{member}() end +return Thing +" + ) + }; + let consumer_source = "---@type Thing +local thing +thing:name() +consume(thing) +"; + let helper_source = "function consume(value) end +"; + + let build = |dir: &str| { + let workspace = std::env::temp_dir().join(dir); + let uri = |name: &str| { + Uri::parse_from_file_path(&workspace.join(name)).expect("uri should parse") + }; + let uris = [uri("producer.lua"), uri("consumer.lua"), uri("helper.lua")]; + let mut analysis = EmmyLuaAnalysis::new(); + analysis.add_main_workspace(workspace); + analysis.update_files_by_uri(vec![ + (uris[0].clone(), Some(producer_source("name"))), + (uris[1].clone(), Some(consumer_source.to_string())), + (uris[2].clone(), Some(helper_source.to_string())), + ]); + (analysis, uris) + }; + + let snapshot = |analysis: &EmmyLuaAnalysis, uris: &[Uri; 3]| { + let shared = analysis.precompute_diagnostic_shared_data(); + uris.iter() + .map(|uri| { + let file_id = analysis.get_file_id(uri).expect("file should be indexed"); + analysis + .diagnose_file_with_shared( + file_id, + CancellationToken::new(), + shared.clone(), + ) + .unwrap_or_default() + }) + .collect::>() + }; + + // Renaming the produced field is a real change: the consumer reads the + // old name, so the edit has to reach it. + let (mut single, single_uris) = build("gmod_glua_ls_two_phase_single"); + let single_producer = single + .update_file_text_only(&single_uris[0], producer_source("title")) + .expect("producer should exist"); + single.reindex_files(vec![single_producer]); + + let (mut split, split_uris) = build("gmod_glua_ls_two_phase_split"); + let split_producer = split + .get_file_id(&split_uris[0]) + .expect("producer should be indexed"); + let expansion = split.expand_reindex_file_ids(vec![split_producer]); + split + .update_file_text_only(&split_uris[0], producer_source("title")) + .expect("producer should exist"); + split.self_index_files(vec![split_producer]); + split.reindex_expanded_files(vec![split_producer], expansion); + + let single_diagnostics = snapshot(&single, &single_uris); + assert!( + single_diagnostics + .iter() + .any(|diagnostics| !diagnostics.is_empty()), + "fixture should exercise observable diagnostics" + ); + assert_eq!( + snapshot(&split, &split_uris), + single_diagnostics, + "self-indexing the edited file first must not change the outcome" + ); + } + #[test] fn multi_file_batch_reindex_matches_clean_build_diagnostics() { let incremental_workspace = diff --git a/crates/glua_code_analysis/src/profile/mod.rs b/crates/glua_code_analysis/src/profile/mod.rs index 75530eab5..1f30a2747 100644 --- a/crates/glua_code_analysis/src/profile/mod.rs +++ b/crates/glua_code_analysis/src/profile/mod.rs @@ -1,30 +1,84 @@ use log::info; +use std::sync::atomic::{AtomicU64, Ordering}; use std::sync::{Mutex, OnceLock}; use std::time::{Duration, Instant}; +/// Allocation counters, incremented by the process's global allocator when it +/// opts in (see the `determinism` tool). Zero everywhere else, so `Profile` +/// simply omits the allocation column when nobody is counting. +/// +/// Sampling profilers attribute time to the allocator, not to the code that +/// asked for the memory. These counters answer the complementary question — +/// *how many* allocations a phase performs — deterministically, which makes +/// "is this phase allocation-bound?" a measurement rather than a guess. +pub static ALLOC_COUNT: AtomicU64 = AtomicU64::new(0); +pub static ALLOC_BYTES: AtomicU64 = AtomicU64::new(0); + +/// True while a `Profile` whose name matches `GLUALS_PROFILE_SAMPLE` is alive. +/// An allocation sampler can gate on this to profile one phase instead of the +/// whole process — which is what makes sampling affordable, since the phase +/// worth sampling (`lua analyze`) is single-threaded and the parallel phases +/// would otherwise swamp the sample set and contend on the sampler's lock. +/// A count rather than a flag: the same phase name can be entered more than +/// once at a time, and a plain flag would let the first exit turn sampling off +/// underneath the others. +static SAMPLE_PHASE_DEPTH: std::sync::atomic::AtomicUsize = std::sync::atomic::AtomicUsize::new(0); + +/// Whether the phase named by `GLUALS_PROFILE_SAMPLE` is currently running. +#[inline] +pub fn sample_phase_active() -> bool { + SAMPLE_PHASE_DEPTH.load(Ordering::Relaxed) > 0 +} + +fn sampled_phase() -> Option<&'static str> { + static NAME: OnceLock> = OnceLock::new(); + NAME.get_or_init(|| std::env::var("GLUALS_PROFILE_SAMPLE").ok()) + .as_deref() +} + +/// Bump the allocation counters. Call from a `GlobalAlloc` implementation. +#[inline] +pub fn record_alloc(size: usize) { + ALLOC_COUNT.fetch_add(1, Ordering::Relaxed); + ALLOC_BYTES.fetch_add(size as u64, Ordering::Relaxed); +} + +fn alloc_snapshot() -> (u64, u64) { + ( + ALLOC_COUNT.load(Ordering::Relaxed), + ALLOC_BYTES.load(Ordering::Relaxed), + ) +} + /// Named sub-phase accumulator, gated on `GLUALS_PROFILE_PHASE`. fn phase_enabled() -> bool { static ENABLED: OnceLock = OnceLock::new(); *ENABLED.get_or_init(|| std::env::var_os("GLUALS_PROFILE_PHASE").is_some()) } -static PHASES: Mutex> = Mutex::new(Vec::new()); +static PHASES: Mutex> = Mutex::new(Vec::new()); -/// Run `f`, accumulating its elapsed time under `name` when phase profiling is on. +/// Run `f`, accumulating its elapsed time and allocation count under `name` when +/// phase profiling is on. pub fn phase(name: &'static str, f: impl FnOnce() -> T) -> T { if !phase_enabled() { return f(); } let start = Instant::now(); + let allocs_before = ALLOC_COUNT.load(Ordering::Relaxed); let out = f(); let elapsed = start.elapsed(); + let allocs = ALLOC_COUNT + .load(Ordering::Relaxed) + .saturating_sub(allocs_before); let mut phases = PHASES.lock().unwrap_or_else(|poison| poison.into_inner()); - match phases.iter_mut().find(|(phase, _, _)| *phase == name) { - Some((_, total, count)) => { + match phases.iter_mut().find(|(phase, _, _, _)| *phase == name) { + Some((_, total, count, total_allocs)) => { *total += elapsed; *count += 1; + *total_allocs += allocs; } - None => phases.push((name, elapsed, 1)), + None => phases.push((name, elapsed, 1, allocs)), } out } @@ -33,31 +87,39 @@ pub fn phase(name: &'static str, f: impl FnOnce() -> T) -> T { /// closure. Accumulates from construction until drop. pub struct PhaseGuard { name: &'static str, - start: Option, + /// Timer and allocation baseline, or `None` when phase profiling is off. + start: Option<(Instant, u64)>, } impl PhaseGuard { pub fn new(name: &'static str) -> Self { Self { name, - start: phase_enabled().then(Instant::now), + start: phase_enabled().then(|| (Instant::now(), ALLOC_COUNT.load(Ordering::Relaxed))), } } } impl Drop for PhaseGuard { fn drop(&mut self) { - let Some(start) = self.start else { + let Some((start, allocs_before)) = self.start else { return; }; let elapsed = start.elapsed(); + let allocs = ALLOC_COUNT + .load(Ordering::Relaxed) + .saturating_sub(allocs_before); let mut phases = PHASES.lock().unwrap_or_else(|poison| poison.into_inner()); - match phases.iter_mut().find(|(phase, _, _)| *phase == self.name) { - Some((_, total, count)) => { + match phases + .iter_mut() + .find(|(phase, _, _, _)| *phase == self.name) + { + Some((_, total, count, total_allocs)) => { *total += elapsed; *count += 1; + *total_allocs += allocs; } - None => phases.push((self.name, elapsed, 1)), + None => phases.push((self.name, elapsed, 1, allocs)), } } } @@ -69,10 +131,10 @@ pub fn phase_report(label: &str) { } let mut phases = std::mem::take(&mut *PHASES.lock().unwrap_or_else(|poison| poison.into_inner())); - phases.sort_unstable_by_key(|(_, total, _)| std::cmp::Reverse(*total)); - for (name, total, count) in phases { + phases.sort_unstable_by_key(|(_, total, _, _)| std::cmp::Reverse(*total)); + for (name, total, count, allocs) in phases { eprintln!( - " [phase] {label:<22} {name:<44} {:>8.3}s ({count} calls)", + " [phase] {label:<22} {name:<44} {:>8.3}s ({count} calls, {allocs} allocs)", total.as_secs_f64() ); } @@ -81,6 +143,7 @@ pub fn phase_report(label: &str) { pub struct Profile<'a> { name: &'a str, start: Instant, + allocs: (u64, u64), } /// When `GLUALS_PROFILE` is set, phase-level `Profile` timers print to stderr @@ -94,9 +157,13 @@ fn phase_profile_enabled() -> bool { #[allow(unused)] impl<'a> Profile<'a> { pub fn new(name: &'a str) -> Self { + if sampled_phase() == Some(name) { + SAMPLE_PHASE_DEPTH.fetch_add(1, Ordering::Relaxed); + } Self { name, start: Instant::now(), + allocs: alloc_snapshot(), } } @@ -111,12 +178,27 @@ impl<'a> Profile<'a> { impl<'a> Drop for Profile<'a> { fn drop(&mut self) { + if sampled_phase() == Some(self.name) { + SAMPLE_PHASE_DEPTH.fetch_sub(1, Ordering::Relaxed); + } let duration = self.start.elapsed(); if log::log_enabled!(log::Level::Info) { info!("{}: cost {:?}", self.name, duration); } if phase_profile_enabled() { - eprintln!("[profile] {}: cost {:?}", self.name, duration); + let (count, bytes) = alloc_snapshot(); + let allocs = count.saturating_sub(self.allocs.0); + if allocs == 0 { + eprintln!("[profile] {}: cost {:?}", self.name, duration); + } else { + eprintln!( + "[profile] {}: cost {:?} ({} allocs, {:.1} MiB)", + self.name, + duration, + allocs, + (bytes.saturating_sub(self.allocs.1)) as f64 / (1024.0 * 1024.0), + ); + } } } } diff --git a/crates/glua_code_analysis/src/progress.rs b/crates/glua_code_analysis/src/progress.rs new file mode 100644 index 000000000..7fa642b03 --- /dev/null +++ b/crates/glua_code_analysis/src/progress.rs @@ -0,0 +1,170 @@ +//! Reporting analysis progress back to whoever asked for the analysis. +//! +//! The sink is process-global for the same reason [`crate::profile`]'s is: the +//! passes that report are threaded through `&mut DbIndex`, not through any +//! object the caller owns. + +use std::sync::{Arc, RwLock}; + +/// One progress report from an analysis pass. +pub struct PhaseProgress<'a> { + /// What the pass is doing, already worded for a user. + pub phase: &'a str, + /// How much of `total` is done. Meaningless when `total` is 0. + pub done: usize, + /// How much there is to do, or 0 when the phase has nothing to count. + pub total: usize, + /// What `done` and `total` count, for the message: "files", "deferred + /// types", and so on. + pub unit: &'a str, +} + +pub type ProgressSink = Arc) + Send + Sync>; + +static SINK: RwLock> = RwLock::new(None); + +/// The phase last entered. Only one phase is in flight at a time, so the +/// per-file loops inside it can report counts against this without naming it — +/// but those loops run on several worker threads, so reads and writes here are +/// concurrent and a count may be reported against a phase that has just ended. +static CURRENT_PHASE: RwLock> = RwLock::new(None); + +/// Install `sink` for the duration of an analysis run. Replaces any previous +/// sink; [`clear_sink`] removes it. +pub fn set_sink(sink: ProgressSink) { + if let Ok(mut slot) = SINK.write() { + *slot = Some(sink); + } +} + +pub fn clear_sink() { + if let Ok(mut slot) = SINK.write() { + *slot = None; + } + if let Ok(mut current) = CURRENT_PHASE.write() { + *current = None; + } +} + +/// Whether anything is listening. +pub fn is_active() -> bool { + SINK.read().is_ok_and(|slot| slot.is_some()) +} + +/// Enter `phase`, and report it. +pub fn enter_phase(phase: &str, total: usize, unit: &str) { + if !is_active() { + return; + } + if let Ok(mut current) = CURRENT_PHASE.write() { + *current = Some(phase.to_string()); + } + emit(PhaseProgress { + phase, + done: 0, + total, + unit, + }); +} + +/// Report `done`/`total` under whichever phase is currently running. +pub fn advance_current_phase(done: usize, total: usize, unit: &str) { + if !is_active() { + return; + } + let phase = match CURRENT_PHASE.read() { + Ok(current) => match current.as_ref() { + Some(phase) => phase.clone(), + None => return, + }, + Err(_) => return, + }; + emit(PhaseProgress { + phase: &phase, + done, + total, + unit, + }); +} + +fn emit(progress: PhaseProgress<'_>) { + let sink = match SINK.read() { + Ok(slot) => slot.clone(), + Err(_) => return, + }; + if let Some(sink) = sink { + sink(progress); + } +} + +/// A phase name to show a user, given the pipeline's Rust type name. An +/// unmapped pipeline falls back to its own name. +pub fn phase_label(pipeline_type_name: &str) -> &str { + match pipeline_type_name { + "DeclAnalysisPipeline" => "Collecting declarations", + "DocAnalysisPipeline" => "Reading annotations", + "FlowAnalysisPipeline" => "Building control flow", + "GmodPreAnalysisPipeline" => "Resolving GMod metadata", + "LuaAnalysisPipeline" => "Inferring types", + "EarlyDynamicFieldAnalysisPipeline" | "DynamicFieldAnalysisPipeline" => { + "Resolving dynamic fields" + } + "PreDynamicUnResolveAnalysisPipeline" | "UnResolveAnalysisPipeline" => { + "Resolving deferred types" + } + "CallSiteParamAnalysisPipeline" => "Inferring parameters from call sites", + "GmodNetworkAnalysisPipeline" => "Analysing net messages", + "GmodPostAnalysisPipeline" => "Finishing GMod analysis", + other => other, + } +} + +#[cfg(test)] +mod tests { + use super::*; + use std::sync::Mutex; + use std::sync::atomic::{AtomicUsize, Ordering}; + + /// The sink is process-global, so these must not run concurrently. + static TEST_LOCK: Mutex<()> = Mutex::new(()); + + #[test] + fn phase_label_maps_known_pipelines_and_passes_through_others() { + assert_eq!(phase_label("LuaAnalysisPipeline"), "Inferring types"); + assert_eq!(phase_label("SomeNewPipeline"), "SomeNewPipeline"); + } + + #[test] + fn report_is_a_noop_without_a_sink() { + let _guard = TEST_LOCK.lock().unwrap_or_else(|e| e.into_inner()); + clear_sink(); + assert!(!is_active()); + enter_phase("anything", 2, "files"); + advance_current_phase(1, 2, "files"); + } + + #[test] + fn advance_reports_under_the_phase_last_entered() { + let _guard = TEST_LOCK.lock().unwrap_or_else(|e| e.into_inner()); + let calls = Arc::new(AtomicUsize::new(0)); + let seen_phase = Arc::new(Mutex::new(String::new())); + + let counter = calls.clone(); + let phase_slot = seen_phase.clone(); + set_sink(Arc::new(move |progress: PhaseProgress<'_>| { + counter.fetch_add(1, Ordering::Relaxed); + if let Ok(mut slot) = phase_slot.lock() { + *slot = progress.phase.to_string(); + } + })); + + enter_phase("Inferring types", 10, "files"); + advance_current_phase(5, 10, "files"); + assert_eq!(calls.load(Ordering::Relaxed), 2); + assert_eq!(seen_phase.lock().unwrap().as_str(), "Inferring types"); + + clear_sink(); + advance_current_phase(6, 10, "files"); + assert_eq!(calls.load(Ordering::Relaxed), 2); + } +} diff --git a/crates/glua_code_analysis/src/semantic/cache/mod.rs b/crates/glua_code_analysis/src/semantic/cache/mod.rs index 29845890c..6c3a34e69 100644 --- a/crates/glua_code_analysis/src/semantic/cache/mod.rs +++ b/crates/glua_code_analysis/src/semantic/cache/mod.rs @@ -15,7 +15,7 @@ use crate::{ semantic::infer::{InferFailReason, ParamInferenceSource}, }; -type FlowCacheInnerKey = (FlowId, GmodRealm, FlowOrigin); +pub type FlowCacheInnerKey = (FlowId, GmodRealm, FlowOrigin); #[derive(Debug, Copy, Clone, PartialEq, Eq, Hash, Default)] pub enum FlowOrigin { @@ -89,6 +89,14 @@ pub struct LuaInferCache { pub flow_node_cache: FxHashMap>>, pub flow_query_realm: Option, + /// Scratch memo for one top-level closure-baseline query. Without it that + /// walk re-derives each merge point once per path into it. + /// + /// Cleared when the outermost baseline query returns: a baseline answer + /// depends on how far the pass has got, which is not in the key, so one + /// must never answer a later query. + pub baseline_flow_memo: FxHashMap<(VarRefCacheKey, FlowCacheInnerKey), LuaType>, + pub baseline_flow_depth: u32, pub flow_node_realm_cache: FxHashMap, pub index_ref_origin_type_cache: FxHashMap>, pub param_type_cache: FxHashMap>, @@ -128,6 +136,14 @@ pub struct LuaInferCache { pub dynamic_field_type_cache: FxHashMap>, pub dynamic_field_resolving: HashSet, pub vgui_parent_fallback_calls: FxHashSet, + /// Call sites of a local function, keyed by its declaration. Syntax ids, + /// not nodes: red nodes are `!Send`. + pub local_function_call_sites_cache: FxHashMap>>, + /// Whether a call diverges, keyed by the call expression. The flow walk asks + /// this of every call node it reaches, and answering means resolving what + /// the call targets through the reference, property, member and signature + /// indexes. + pub call_returns_never_cache: FxHashMap, inferred_guard_dependencies: HashSet, } @@ -141,6 +157,8 @@ impl LuaInferCache { call_arg_types_cache: FxHashMap::default(), flow_node_cache: FxHashMap::default(), flow_query_realm: None, + baseline_flow_memo: FxHashMap::default(), + baseline_flow_depth: 0, flow_node_realm_cache: FxHashMap::default(), index_ref_origin_type_cache: FxHashMap::default(), param_type_cache: FxHashMap::default(), @@ -161,6 +179,8 @@ impl LuaInferCache { dynamic_field_type_cache: FxHashMap::default(), dynamic_field_resolving: HashSet::new(), vgui_parent_fallback_calls: FxHashSet::default(), + local_function_call_sites_cache: FxHashMap::default(), + call_returns_never_cache: FxHashMap::default(), inferred_guard_dependencies: HashSet::new(), } } @@ -241,6 +261,7 @@ impl LuaInferCache { self.dynamic_field_type_cache.clear(); self.dynamic_field_resolving.clear(); self.vgui_parent_fallback_calls.clear(); + self.call_returns_never_cache.clear(); } /// Discards the inference a wave of deferred resolution can have @@ -249,6 +270,9 @@ impl LuaInferCache { self.expr_cache.clear(); self.call_cache.clear(); self.call_arg_types_cache.clear(); + // A resolved signature return is exactly what turns this answer from + // `false` to `true`, so it cannot survive a wave. + self.call_returns_never_cache.clear(); self.flow_node_cache.retain(|_, inner| { inner.retain(|_, entry| !matches!(entry, CacheEntry::Error(_))); !inner.is_empty() @@ -274,6 +298,7 @@ impl LuaInferCache { self.index_ref_origin_type_cache.clear(); self.param_type_cache.clear(); self.param_type_source_cache.clear(); + self.call_returns_never_cache.clear(); // Local reference identities come directly from immutable reference // indexes and are safe to retain. Global/member/self roots can be // selected through types and overloads that unresolve is about to diff --git a/crates/glua_code_analysis/src/semantic/infer/infer_binary/infer_binary_or.rs b/crates/glua_code_analysis/src/semantic/infer/infer_binary/infer_binary_or.rs index 89039df79..7bdd20a73 100644 --- a/crates/glua_code_analysis/src/semantic/infer/infer_binary/infer_binary_or.rs +++ b/crates/glua_code_analysis/src/semantic/infer/infer_binary/infer_binary_or.rs @@ -186,7 +186,16 @@ pub fn special_or_rule( _ => return None, } - if right_type.is_nil() || left_type.is_const() { + // The answer below is the left arm's truthy half on its own, which + // stands as the whole answer only while there is one: the + // compatibility check has established the right arm adds nothing to + // it. A left arm that is falsy throughout always evaluates to the + // right arm, and its truthy half is empty, so answering with that + // half drops the only operand the expression can return and hands + // back a `nil` the source cannot produce. `and` already declines on + // the same condition; leave this to the general rule, which returns + // the right arm. + if right_type.is_nil() || left_type.is_const() || left_type.is_always_falsy() { return None; } @@ -198,13 +207,6 @@ pub fn special_or_rule( _ => {} } - // `X = X or {}` with an unresolved `X`: the fallback arm is real - // evidence, the left arm is not, so keep both rather than widening the - // pair to `any`. - if left_type.is_unknown() { - return Some(TypeOps::Union.apply(db, &LuaType::Unknown, right_type)); - } - None } diff --git a/crates/glua_code_analysis/src/semantic/infer/infer_index/mod.rs b/crates/glua_code_analysis/src/semantic/infer/infer_index/mod.rs index a9c9c0f90..ed1169d52 100644 --- a/crates/glua_code_analysis/src/semantic/infer/infer_index/mod.rs +++ b/crates/glua_code_analysis/src/semantic/infer/infer_index/mod.rs @@ -880,12 +880,12 @@ fn infer_table_dynamic_key_member_type( } let access_key_type = member_key_as_type(key)?; - let members = db.get_member_index().get_members(owner)?; + let members = db.get_member_index().get_expr_key_members(owner)?; let mut result_type = LuaType::Never; for member in members { let dynamic_key = member.get_key(); - if dynamic_key == key || !dynamic_key.is_expr() { + if dynamic_key == key { continue; } if is_literal_member_key(key) @@ -917,15 +917,11 @@ fn owner_has_precise_dynamic_value( caller_file_id: FileId, caller_position: Option, ) -> bool { - let Some(members) = db.get_member_index().get_members(owner) else { + let Some(members) = db.get_member_index().get_expr_key_members(owner) else { return false; }; members.iter().any(|member| { - if !member.get_key().is_expr() { - return false; - } - let member_item = LuaMemberIndexItem::One(member.get_id()); resolve_member_item_with_realm(db, &member_item, caller_file_id, caller_position) .is_ok_and(|typ| is_precise_unknown_wildcard_value_type(&typ)) @@ -1187,15 +1183,14 @@ fn infer_cross_file_matching_expr_key_member_type( let allow_wildcard_expr_literal_match = is_literal_member_key(key) && owner_wildcard_covers_literal_key(db, owner); - let members = db.get_member_index().get_members(owner)?; + let members = db.get_member_index().get_expr_key_members(owner)?; let mut result = LuaType::Never; // See `infer_gmod_same_file_expr_key_member_type`: matching is tracked // separately so an Unknown member type does not read as "no match". let mut saw_match = false; for member in members { - if !member.get_key().is_expr() - || member.get_file_id() == access_file_id + if member.get_file_id() == access_file_id || !is_dynamic_field_fallback_realm_compatible( db, access_realm, @@ -1260,11 +1255,10 @@ fn table_has_cross_file_matching_expr_key_member( .unwrap_or(crate::GmodRealm::Unknown); db.get_member_index() - .get_members(owner) + .get_expr_key_members(owner) .is_some_and(|members| { members.iter().any(|member| { - member.get_key().is_expr() - && member.get_file_id() != access_file_id + member.get_file_id() != access_file_id && (!is_literal_member_key(key) || !member_is_finite_named_dynamic_assignment(db, owner, member)) && is_dynamic_field_fallback_realm_compatible( @@ -1432,10 +1426,7 @@ fn table_const_has_no_specific_data( owner: &LuaMemberOwner, inst: &InFiled, ) -> bool { - db.get_member_index() - .get_members(owner) - .is_none_or(|members| members.is_empty()) - && db.get_metatable_index().get(inst).is_none() + !db.get_member_index().has_live_member(owner) && db.get_metatable_index().get(inst).is_none() } fn infer_plain_table_member( @@ -1796,7 +1787,17 @@ fn infer_custom_type_member( return Err(InferFailReason::UnSealedDynamicFields); } - if let Some(dynamic_field) = dynamic_field_result.unwrap_or_default() { + // An entry that carries `unknown`/`nil` names a field without saying what it + // holds, and answering with it ends the lookup: this type's own super walk + // stops, and so does the walk of whichever type asked, before either reaches + // a super that has a real type. Whether the index holds that entry at the + // moment of the read is how far the batch has run — it is unsealed on a cold + // walk and populated on a warm one — so the uninformative entry decides the + // answer on one build and not the other. Treat it as no entry at all. + if let Some(dynamic_field) = dynamic_field_result + .unwrap_or_default() + .filter(|dynamic_field| !dynamic_field.typ.is_unknown() && !dynamic_field.typ.is_nil()) + { if type_decl.is_class() && let Some(super_types) = visible_super_types_for_index(db, cache, &prefix_type_id, &index_expr) @@ -1820,10 +1821,6 @@ fn infer_custom_type_member( ])); } - if dynamic_field.typ.is_nil() || dynamic_field.typ.is_unknown() { - return Ok(super_member_type); - } - return Ok(dynamic_field.typ); } Err(InferFailReason::FieldNotFound) @@ -2598,8 +2595,8 @@ fn global_expr_access_path(db: &DbIndex, file_id: FileId, expr: &LuaExpr) -> Opt } match expr { - LuaExpr::NameExpr(name_expr) => name_expr.get_access_path(), - LuaExpr::IndexExpr(index_expr) => index_expr.get_access_path(), + LuaExpr::NameExpr(name_expr) => name_expr.get_access_path().map(Into::into), + LuaExpr::IndexExpr(index_expr) => index_expr.get_access_path().map(Into::into), _ => None, } } @@ -2667,7 +2664,24 @@ fn infer_member_by_index_table( let index_key = index_expr.get_index_key().ok_or(InferFailReason::None)?; let key_type = index_key_access_type(db, cache, &index_key)?; let owner = LuaMemberOwner::Element(table_range.clone()); - let members = db.get_member_index().get_members(&owner); + let access_key = LuaMemberKey::from_index_key_or_unknown(db, cache, &index_key).ok(); + let member_index = db.get_member_index(); + // A literal key matches a literal member key only when the two are + // equal, so the candidates are that one key plus the + // expression-keyed members. + let members = match &access_key { + Some(key @ (LuaMemberKey::Name(_) | LuaMemberKey::Integer(_))) => member_index + .get_members_with_key(&owner, key) + .map(|mut candidates| { + candidates.extend( + member_index + .get_expr_key_members(&owner) + .unwrap_or_default(), + ); + candidates + }), + _ => member_index.get_members(&owner), + }; if let Some(mut members) = members { members.sort_by(|a, b| a.get_key().cmp(b.get_key())); let mut result_type = LuaType::Never; diff --git a/crates/glua_code_analysis/src/semantic/infer/infer_name.rs b/crates/glua_code_analysis/src/semantic/infer/infer_name.rs index 309f586fb..8a24e4c39 100644 --- a/crates/glua_code_analysis/src/semantic/infer/infer_name.rs +++ b/crates/glua_code_analysis/src/semantic/infer/infer_name.rs @@ -1,9 +1,10 @@ use glua_parser::{ LuaAssignStat, LuaAstNode, LuaAstToken, LuaCallExpr, LuaChunk, LuaClosureExpr, LuaExpr, LuaForRangeStat, LuaFuncStat, LuaIndexExpr, LuaLocalFuncStat, LuaLocalStat, LuaNameExpr, - LuaReturnStat, LuaSyntaxNode, LuaTableExpr, LuaTableField, LuaVarExpr, PathTrait, + LuaReturnStat, LuaSyntaxId, LuaSyntaxNode, LuaTableExpr, LuaTableField, LuaVarExpr, PathTrait, }; use rowan::TextSize; +use std::sync::Arc; use super::{ InferFailReason, InferResult, infer_expr, infer_table_field_value_should_be, @@ -948,7 +949,7 @@ fn infer_param_type_from_call_sites( .get_signature_index() .local_func_decl_for(&signature_id)?; let call_sites = - local_function_call_sites(db, signature_id.get_file_id(), &root, target_decl_id); + local_function_call_sites(db, cache, signature_id.get_file_id(), &root, target_decl_id); infer_param_type_from_local_call_sites_inner(db, cache, call_sites, param_idx, true) } @@ -1061,7 +1062,7 @@ fn infer_unread_local_call_site_args( }; let unread_args = - local_function_call_sites(db, signature_id.get_file_id(), &root, target_decl_id) + local_function_call_sites(db, cache, signature_id.get_file_id(), &root, target_decl_id) .into_iter() .filter_map(|(_, call_expr)| { call_expr @@ -1130,8 +1131,12 @@ fn infer_forwarded_param_arg_type( .get_vfs() .get_syntax_tree(&signature_id.get_file_id())? .get_red_root(); + // The signature's position is where its closure starts, so descend to that + // offset instead of scanning every node in the file. let closure = root - .descendants() + .token_at_offset(signature_id.get_position()) + .right_biased()? + .parent_ancestors() .filter_map(LuaClosureExpr::cast) .find(|closure| closure.get_position() == signature_id.get_position())?; let local_func_name = closure @@ -1139,21 +1144,52 @@ fn infer_forwarded_param_arg_type( .and_then(|local_func| local_func.get_local_name())?; let target_decl_id = LuaDeclId::new(signature_id.get_file_id(), local_func_name.get_position()); - infer_param_type_from_local_call_sites_inner( - db, - cache, - local_function_call_sites(db, signature_id.get_file_id(), &root, target_decl_id), - idx, - false, - ) + let call_sites = + local_function_call_sites(db, cache, signature_id.get_file_id(), &root, target_decl_id); + infer_param_type_from_local_call_sites_inner(db, cache, call_sites, idx, false) } fn local_function_call_sites( db: &DbIndex, + cache: &mut LuaInferCache, file_id: FileId, root: &LuaSyntaxNode, target_decl_id: LuaDeclId, ) -> Vec<(FileId, LuaCallExpr)> { + // Parameter inference asks for the same function's call sites once per + // parameter index, and each miss walks the tree from the root for every + // reference. Derive the set once and re-resolve the ids on later calls. + let syntax_ids = match cache.local_function_call_sites_cache.get(&target_decl_id) { + Some(cached) => cached.clone(), + None => { + let ids = Arc::new(find_local_function_call_sites( + db, + file_id, + root, + target_decl_id, + )); + cache + .local_function_call_sites_cache + .insert(target_decl_id, ids.clone()); + ids + } + }; + + syntax_ids + .iter() + .filter_map(|syntax_id| { + let node = syntax_id.to_node_from_root(root)?; + Some((file_id, LuaCallExpr::cast(node)?)) + }) + .collect() +} + +fn find_local_function_call_sites( + db: &DbIndex, + file_id: FileId, + root: &LuaSyntaxNode, + target_decl_id: LuaDeclId, +) -> Vec { let Some(decl_refs) = db .get_reference_index() .get_local_reference(&file_id) @@ -1175,7 +1211,7 @@ fn local_function_call_sites( }) .filter_map(|name_expr| name_expr.get_parent::()) .filter(|call_expr| matches!(call_expr.get_prefix_expr(), Some(LuaExpr::NameExpr(_)))) - .map(|call_expr| (file_id, call_expr)) + .map(|call_expr| call_expr.get_syntax_id()) .collect() } @@ -1337,6 +1373,35 @@ fn find_param_type_from_sibling_members( final_type } +type InheritedParamKey = (LuaMemberId, usize, bool, bool, FileId, TextSize); + +thread_local! { + /// Memo for [`find_param_type_from_inherited_members`], paired with the + /// `type_structure_revision` it was built against. + /// + /// Thread-local rather than a field on `DbIndex` because `&DbIndex` is + /// shared across worker threads, so the memo cannot live behind a `RefCell` + /// on the struct without giving up `Sync`. + static INHERITED_PARAM_MEMO: std::cell::RefCell<(u64, rustc_hash::FxHashMap>)> = + std::cell::RefCell::new((u64::MAX, rustc_hash::FxHashMap::default())); +} + +/// The parameter's type as declared by an inherited member, if any. +/// +/// This is the most expensive step of parameter inference: the unresolve +/// pipeline's reachability probe calls it once per deferred parameter, and the +/// cost is almost entirely the visibility-aware member lookup it performs per +/// super type. +/// +/// The same key is asked repeatedly across the retry loop's iterations, so the +/// answer is memoized against `type_structure_revision`. Mutating the member, +/// type, signature or module index bumps that revision, which covers the inputs +/// the loop itself moves as it resolves. +/// +/// It is not a complete read set. The visibility-aware lookup below also reads +/// the dynamic-field and gmod-infer indexes, and neither bumps the revision, so +/// a member that becomes visible between two unresolve runs can be missed by a +/// memo entry computed before it existed. fn find_param_type_from_inherited_members( db: &DbIndex, current_member_id: LuaMemberId, @@ -1345,6 +1410,58 @@ fn find_param_type_from_inherited_members( is_dots: bool, caller_file_id: FileId, caller_position: TextSize, +) -> Option { + let revision = db.type_structure_revision(); + let key = ( + current_member_id, + param_idx, + colon_define, + is_dots, + caller_file_id, + caller_position, + ); + + let cached = INHERITED_PARAM_MEMO.with(|memo| { + let mut memo = memo.borrow_mut(); + if memo.0 != revision { + memo.0 = revision; + memo.1.clear(); + return None; + } + memo.1.get(&key).cloned() + }); + if let Some(cached) = cached { + return cached; + } + + let found = find_param_type_from_inherited_members_uncached( + db, + current_member_id, + param_idx, + colon_define, + is_dots, + caller_file_id, + caller_position, + ); + + INHERITED_PARAM_MEMO.with(|memo| { + let mut memo = memo.borrow_mut(); + // Only store if nothing bumped the revision while we were computing. + if memo.0 == revision { + memo.1.insert(key, found.clone()); + } + }); + found +} + +fn find_param_type_from_inherited_members_uncached( + db: &DbIndex, + current_member_id: LuaMemberId, + param_idx: usize, + colon_define: bool, + is_dots: bool, + caller_file_id: FileId, + caller_position: TextSize, ) -> Option { let member_index = db.get_member_index(); let owner = member_index.get_current_owner(¤t_member_id)?; @@ -1541,7 +1658,7 @@ fn member_receiver_name(func: &LuaFuncStat) -> Option { let LuaExpr::NameExpr(name_expr) = index_expr.get_prefix_expr()? else { return None; }; - name_expr.get_name_text().map(Into::into) + name_expr.get_name_text() } fn find_overload_param_type_from_type( @@ -2372,6 +2489,21 @@ fn infer_global_type_from_decl_ids(db: &DbIndex, decl_ids: Vec) -> In } } + // A global's type is the merge of every declaration of it, so a declaration + // whose type cache is missing is a writer this merge cannot see. Answering + // anyway makes the result depend on how far the batch has run rather than on + // the source: `remove_index` clears the batch's caches up front, so which + // declarations are visible is decided by batch composition. Every branch + // below is affected — the callable union loses an arm, `def_or_ref_type` and + // the table merge pick a different winner, and `saw_nil` cannot know whether + // the absent declaration was nil. + // + // Defer instead. The unresolve pass retries once that declaration carries a + // type and floors it to `Unknown` if it never does, so this cannot stall. + if !matches!(last_resolve_reason, InferFailReason::None) { + return Err(last_resolve_reason); + } + if let Some(callable_type) = callable_type { return Ok(callable_type); } diff --git a/crates/glua_code_analysis/src/semantic/infer/mod.rs b/crates/glua_code_analysis/src/semantic/infer/mod.rs index 11f7da57b..436dd66f7 100644 --- a/crates/glua_code_analysis/src/semantic/infer/mod.rs +++ b/crates/glua_code_analysis/src/semantic/infer/mod.rs @@ -39,10 +39,7 @@ use infer_table::infer_table_expr; pub use infer_table::{infer_table_field_value_should_be, infer_table_should_be}; use infer_unary::infer_unary_expr; pub use narrow::{SelfRefId, VarRefId, VarRefRootId}; -pub(crate) use narrow::{ - contains_gmod_null_type, expr_may_have_condition_narrowing, get_var_expr_var_ref_id, - remove_false_or_nil, -}; +pub(crate) use narrow::{contains_gmod_null_type, get_var_expr_var_ref_id, remove_false_or_nil}; use rowan::TextRange; use smol_str::SmolStr; diff --git a/crates/glua_code_analysis/src/semantic/infer/narrow/get_type_at_flow.rs b/crates/glua_code_analysis/src/semantic/infer/narrow/get_type_at_flow.rs index 990be60d4..470b7c117 100644 --- a/crates/glua_code_analysis/src/semantic/infer/narrow/get_type_at_flow.rs +++ b/crates/glua_code_analysis/src/semantic/infer/narrow/get_type_at_flow.rs @@ -1,4 +1,8 @@ -use std::{collections::HashSet, ops::Deref}; +use std::ops::Deref; + +// Every set below is a cycle guard for a graph walk: membership only, never +// iterated, so the hasher cannot reach a result. +use rustc_hash::FxHashSet as HashSet; use glua_parser::{ BinaryOperator, LuaAssignStat, LuaAstNode, LuaBlock, LuaCallExpr, LuaChunk, LuaClosureExpr, @@ -12,7 +16,7 @@ use crate::{ FlowTree, GlobalId, GmodRealm, InferFailReason, LuaArrayType, LuaDeclId, LuaInferCache, LuaMemberId, LuaMemberKey, LuaMemberOwner, LuaSemanticDeclId, LuaSignatureId, LuaType, LuaTypeDeclId, LuaTypeOwner, LuaUnionType, TypeOps, infer_expr, - semantic::cache::FlowOrigin, + semantic::cache::{FlowOrigin, VarRefCacheKey}, semantic::gmod_call_effect::{GmodCallWriteEffect, gmod_call_write_effect}, semantic::infer::{ InferResult, VarRefId, infer_expr_list_value_type_at, @@ -175,6 +179,18 @@ pub fn get_type_at_flow_with_origin( result } +#[cfg(test)] +thread_local! { + /// Closure-baseline walks that missed the memo, on this thread. + /// + /// The memo turns a per-path derivation into a per-merge-point one, and the + /// difference is a count, not a duration — asserting on the count instead of + /// on wall-clock keeps the guard immune to how loaded the machine is. A + /// single-file `VirtualWorkspace` analyses inline, so the walks land on the + /// thread that asked for them and one test cannot see another's. + pub(crate) static BASELINE_FLOW_WALKS: std::cell::Cell = const { std::cell::Cell::new(0) }; +} + pub(super) fn get_type_at_flow_in_mode( db: &DbIndex, tree: &FlowTree, @@ -193,8 +209,20 @@ pub(super) fn get_type_at_flow_in_mode( db.get_gmod_infer_index() .get_realm_at_offset(&cache.get_file_id(), var_ref_id.get_position()) }); + let memo_key = ( + VarRefCacheKey::from(var_ref_id), + (flow_id, query_realm, policy.origin), + ); + if let Some(narrow_type) = cache.baseline_flow_memo.get(&memo_key) { + return Ok(narrow_type.clone()); + } + + #[cfg(test)] + BASELINE_FLOW_WALKS.with(|walks| walks.set(walks.get() + 1)); + + cache.baseline_flow_depth += 1; let mut visited_flow_ids = Vec::new(); - get_type_at_flow_walk( + let result = get_type_at_flow_walk( db, tree, cache, @@ -204,7 +232,17 @@ pub(super) fn get_type_at_flow_in_mode( flow_id, &mut visited_flow_ids, policy, - ) + ); + if let Ok(narrow_type) = &result { + cache + .baseline_flow_memo + .insert(memo_key, narrow_type.clone()); + } + cache.baseline_flow_depth -= 1; + if cache.baseline_flow_depth == 0 { + cache.baseline_flow_memo.clear(); + } + result } } } @@ -1276,10 +1314,30 @@ fn call_flow_node_returns_never( call_expr_returns_never(db, cache, call_expr) } +/// The flow walk asks this of every call node it reaches, and the same call is +/// reached again by every later query that walks through it, so the answer is +/// memoised for as long as the types it reads hold still. fn call_expr_returns_never( db: &DbIndex, cache: &mut LuaInferCache, call_expr: glua_parser::LuaCallExpr, +) -> bool { + let syntax_id = call_expr.get_syntax_id(); + if let Some(returns_never) = cache.call_returns_never_cache.get(&syntax_id) { + return *returns_never; + } + + let returns_never = call_expr_returns_never_uncached(db, cache, call_expr); + cache + .call_returns_never_cache + .insert(syntax_id, returns_never); + returns_never +} + +fn call_expr_returns_never_uncached( + db: &DbIndex, + cache: &mut LuaInferCache, + call_expr: glua_parser::LuaCallExpr, ) -> bool { if call_expr.is_error() { return true; @@ -1598,7 +1656,7 @@ fn branch_has_relevant_special_call_effects( return false; }; - let mut visited = HashSet::new(); + let mut visited = HashSet::default(); antecedents.iter().copied().any(|flow_id| { antecedent_has_relevant_special_call_effect( db, @@ -1706,6 +1764,56 @@ fn special_call_effect_matches_var_ref(effect_target: &VarRefId, var_ref_id: &Va ) } +/// The type an assignment writes, with a read of an undefined global counted as +/// the `nil` it is at runtime. +/// +/// `infer_expr` reports an undefined global as `InferFailReason::None` rather +/// than a type, so propagating that failure abandons the whole flow walk and +/// the variable falls back to `unknown` — one unresolvable name erases what +/// every other branch established about it. `analyze_assign_stat` already +/// applies this rule when it binds the assignment's own cache ("undefined-global +/// RHS is `nil` at runtime, not unknown"); the flow walk has to agree with it, +/// or the same assignment means two different things depending on which path +/// asked. +fn infer_assigned_value_type_at( + db: &DbIndex, + cache: &mut LuaInferCache, + exprs: &[LuaExpr], + value_idx: usize, +) -> Result, InferFailReason> { + let is_undefined_global = exprs + .get(value_idx) + .is_some_and(|expr| expr_reads_undefined_global(db, cache, expr)); + match infer_expr_list_value_type_at(db, cache, exprs, value_idx) { + Err(InferFailReason::None) if is_undefined_global => Ok(Some(LuaType::Nil)), + Ok(Some(typ)) if typ.is_unknown() && is_undefined_global => Ok(Some(LuaType::Nil)), + other => other, + } +} + +/// Whether `expr` is a bare name that resolves to no declaration at all. +fn expr_reads_undefined_global(db: &DbIndex, cache: &LuaInferCache, expr: &LuaExpr) -> bool { + let LuaExpr::NameExpr(name_expr) = expr else { + return false; + }; + let Some(name) = name_expr.get_name_text() else { + return false; + }; + if name == "self" || name == "_G" || name == "_ENV" { + return false; + } + let file_id = cache.get_file_id(); + let has_local = db + .get_decl_index() + .get_decl_tree(&file_id) + .and_then(|tree| tree.find_local_decl(&name, name_expr.get_position())) + .is_some(); + if has_local { + return false; + } + db.get_global_index().get_global_decl_ids(&name).is_none() +} + fn get_type_at_assign_stat( db: &DbIndex, tree: &FlowTree, @@ -1729,7 +1837,7 @@ fn get_type_at_assign_stat( }; if numeric_table_index_query_key_name(var_ref_id) - .map(str::to_string) + .map(smol_str::SmolStr::new) .or_else(|| numeric_table_index_query_key_name_from_initializer(db, root, var_ref_id)) .is_some_and(|key_name| var_ref_is_name(db, &maybe_ref_id, &key_name)) { @@ -1758,7 +1866,7 @@ fn get_type_at_assign_stat( )?)); } if numeric_table_index_query_key_name(var_ref_id) - .map(str::to_string) + .map(smol_str::SmolStr::new) .or_else(|| { numeric_table_index_query_key_name_from_initializer(db, root, var_ref_id) }) @@ -1768,7 +1876,7 @@ fn get_type_at_assign_stat( { return Ok(ResultTypeOrContinue::Result(LuaType::Nil)); } - let Some(expr_type) = infer_expr_list_value_type_at(db, cache, &exprs, i)? else { + let Some(expr_type) = infer_assigned_value_type_at(db, cache, &exprs, i)? else { return Ok(ResultTypeOrContinue::Continue); }; return Ok(ResultTypeOrContinue::Result(expr_type)); @@ -1787,7 +1895,7 @@ fn get_type_at_assign_stat( } if numeric_table_index_query_key_name(var_ref_id) - .map(str::to_string) + .map(smol_str::SmolStr::new) .or_else(|| numeric_table_index_query_key_name_from_initializer(db, root, var_ref_id)) .is_some_and(|key_name| { assignment_vars_write_dynamic_key_name(db, cache, &vars, &key_name) @@ -1818,7 +1926,7 @@ fn get_type_at_assign_stat( let expr_type = match guarded_global_type { Some(typ) => Some(typ), - None => infer_expr_list_value_type_at(db, cache, &exprs, i)?, + None => infer_assigned_value_type_at(db, cache, &exprs, i)?, }; let Some(expr_type) = expr_type else { return Ok(ResultTypeOrContinue::Continue); @@ -1929,7 +2037,7 @@ fn try_get_numeric_range_table_arg_population_type( return Ok(None); }; let key_name = numeric_table_index_query_key_name(var_ref_id) - .map(str::to_string) + .map(smol_str::SmolStr::new) .or_else(|| numeric_table_index_query_key_name_from_initializer(db, root, var_ref_id)); let args = call_expr @@ -1997,7 +2105,7 @@ fn try_get_numeric_range_table_arg_population_type( rhs_expr, &query_root, key_name.as_deref(), - &mut HashSet::new(), + &mut HashSet::default(), ) { return Ok(None); } @@ -2202,7 +2310,7 @@ fn numeric_global_table_index_query( db: &DbIndex, cache: &mut LuaInferCache, index_expr: &LuaIndexExpr, -) -> Option<(String, i64, Option)> { +) -> Option<(smol_str::SmolStr, i64, Option)> { let LuaExpr::NameExpr(table_name) = index_expr.get_prefix_expr()? else { return None; }; @@ -2427,7 +2535,7 @@ fn call_effect_overlaps_mutation_roots( cache, call_expr, mutation_roots, - &mut HashSet::new(), + &mut HashSet::default(), ), }; call_overlaps @@ -2566,7 +2674,7 @@ fn var_expr_may_mutate_global_table(var: &LuaVarExpr, mutation_roots: &[&str]) - } } -fn index_expr_global_root_name(index_expr: &LuaIndexExpr) -> Option { +fn index_expr_global_root_name(index_expr: &LuaIndexExpr) -> Option { let mut prefix = index_expr.get_prefix_expr()?; while let LuaExpr::IndexExpr(parent_index) = prefix { prefix = parent_index.get_prefix_expr()?; @@ -2785,7 +2893,7 @@ fn numeric_table_index_query_key_name_from_initializer( db: &DbIndex, root: &LuaChunk, var_ref_id: &VarRefId, -) -> Option { +) -> Option { let decl_id = match var_ref_id { VarRefId::IndexRef(query_root, _) => query_root.as_decl_id(), _ => var_ref_id.get_decl_id_ref(), @@ -3029,7 +3137,7 @@ fn antecedent_has_relevant_special_call_effect_before_node( flow_node: &FlowNode, var_ref_id: &VarRefId, ) -> bool { - let mut visited = HashSet::new(); + let mut visited = HashSet::default(); match flow_node.antecedent { Some(FlowAntecedent::Single(prev)) => antecedent_has_relevant_special_call_effect( db, @@ -3394,7 +3502,7 @@ fn infer_collection_base_types<'a>( base_type } -fn expr_access_path(expr: &LuaExpr) -> Option { +fn expr_access_path(expr: &LuaExpr) -> Option { match expr { LuaExpr::NameExpr(name_expr) => name_expr.get_access_path(), LuaExpr::IndexExpr(index_expr) => index_expr.get_access_path(), @@ -3514,7 +3622,7 @@ pub fn explicit_param_string_default_reaches_flow( use_flow_id: FlowId, ) -> bool { let var_ref_id = VarRefId::VarRef(decl_id); - let mut visited = HashSet::new(); + let mut visited = HashSet::default(); explicit_default_reaches_inner( db, tree, @@ -3722,7 +3830,7 @@ pub fn inferred_string_default_reaches_flow( default_source_range: rowan::TextRange, ) -> bool { let var_ref_id = VarRefId::VarRef(decl_id); - let mut visited = HashSet::new(); + let mut visited = HashSet::default(); inferred_string_default_reaches_inner( db, tree, diff --git a/crates/glua_code_analysis/src/semantic/infer/narrow/mod.rs b/crates/glua_code_analysis/src/semantic/infer/narrow/mod.rs index 1928f655c..828394e34 100644 --- a/crates/glua_code_analysis/src/semantic/infer/narrow/mod.rs +++ b/crates/glua_code_analysis/src/semantic/infer/narrow/mod.rs @@ -120,25 +120,6 @@ fn var_ref_can_be_narrowed(db: &DbIndex, file_id: &crate::FileId, var_ref_id: &V } } -pub(crate) fn expr_may_have_condition_narrowing( - db: &DbIndex, - cache: &mut LuaInferCache, - expr: LuaExpr, -) -> bool { - let file_id = cache.get_file_id(); - let syntax_id = expr.get_syntax_id(); - let Some(VarRefId::IndexRef(_, path)) = get_var_expr_var_ref_id(db, cache, expr) else { - return false; - }; - let Some(flow_tree) = db.get_flow_index().get_flow_tree(&file_id) else { - return false; - }; - let Some(flow_id) = flow_tree.get_flow_id(syntax_id) else { - return false; - }; - flow_tree.has_condition_path_antecedent(flow_id, &path) -} - pub fn infer_expr_narrow_type( db: &DbIndex, cache: &mut LuaInferCache, diff --git a/crates/glua_code_analysis/src/semantic/infer/test.rs b/crates/glua_code_analysis/src/semantic/infer/test.rs index 9186582e1..90d5ff817 100644 --- a/crates/glua_code_analysis/src/semantic/infer/test.rs +++ b/crates/glua_code_analysis/src/semantic/infer/test.rs @@ -1083,12 +1083,11 @@ mod test { } #[test] - fn test_or_with_local_unknown_does_not_coerce_to_nil() { + fn test_or_with_unknown_left_yields_the_fallback() { let mut ws = VirtualWorkspace::new_with_init_std_lib(); let ty = infer_last_name_expr_type( &mut ws, r#" - ---@type unknown local maybe local result = maybe or {} print(result) @@ -1096,16 +1095,16 @@ mod test { "result", ); - // Both arms survive: the fallback table is real evidence and the - // unresolved left arm is not, so neither erases the other. - let LuaType::Union(union) = &ty else { - panic!("expected a union, got: {ty:?}"); - }; - let arms = union.into_vec(); + // `unknown` on the left is a report that inference could not tell what + // the left arm holds, not a type the expression can evaluate to. Whether + // it could tell depends on how far the batch had run, so carrying it + // into the answer pins the file walk order into the type: the same + // source read `integer` or `integer|unknown` depending on which file + // declared a constant first. The fallback is the only real evidence + // here, so it stands alone. assert!( - arms.iter().any(|arm| matches!(arm, LuaType::TableConst(_))) - && arms.iter().any(|arm| matches!(arm, LuaType::Unknown)), - "expected `unknown | table`, got: {arms:?}" + matches!(ty, LuaType::TableConst(_)), + "expected the fallback table, got: {ty:?}" ); } @@ -1489,6 +1488,34 @@ mod test { ); } + /// `X or false` evaluates to the left arm when it is truthy and to `false` + /// otherwise, so it cannot be nil whatever `X` is. A left arm that is falsy + /// throughout has no truthy half to answer with, and the fallback arm is the + /// only value the expression can produce. + #[test] + fn test_or_with_all_falsy_left_arm_yields_the_fallback_not_nil() { + let mut ws = VirtualWorkspace::new(); + let ty = infer_last_name_expr_type( + &mut ws, + r#" + ---@param flag false|nil + local function pick(flag) + local enabled = flag or false + return enabled + end + local chosen = pick(nil) + print(chosen) + "#, + "chosen", + ); + + assert!( + !ty.is_nil(), + "`X or false` must not infer as nil, got: {}", + ws.humanize_type_detailed(ty) + ); + } + /// A call shape the reader cannot interpret leaves the type unresolved. /// `any` would claim the author opted out of checking instead. #[test] @@ -1509,4 +1536,59 @@ mod test { ); assert_eq!(ws.expr_ty("require(some_computed_path)"), LuaType::Unknown); } + + /// An `__index` metamethod on a super that sits after a diamond in the + /// inheritance graph must still be found. `Leaf : Mid, Root, Store` reaches + /// `Root` twice — directly and through `Mid` — and the operator walk shares + /// one infer guard across siblings, so the second arrival reported recursion. + /// Treating that as fatal dropped the remaining siblings, losing `Store`'s + /// index operator entirely. + #[test] + fn test_index_operator_survives_super_diamond() { + let mut ws = VirtualWorkspace::new(); + ws.def( + r#" + ---@class DiamondRoot + local DiamondRoot = {} + + ---@class DiamondMid : DiamondRoot + local DiamondMid = {} + + ---@class DiamondStore + ---@field [string] number + local DiamondStore = {} + + ---@class DiamondLeaf : DiamondMid, DiamondRoot, DiamondStore + local DiamondLeaf = {} + + ---@type DiamondLeaf + leafValue = nil + "#, + ); + + assert_eq!(ws.expr_ty("leafValue.anythingAtAll"), LuaType::Number); + } + + /// Sibling super branches get their own guard fork, so cycle detection now + /// rests entirely on the guard's parent chain. Mutually recursive classes + /// must still terminate rather than recurse forever, and an absent field + /// stays an absent field rather than degrading to unknown. + #[test] + fn test_mutually_recursive_supers_terminate() { + let mut ws = VirtualWorkspace::new(); + ws.def( + r#" + ---@class CycleFirst : CycleSecond + local CycleFirst = {} + + ---@class CycleSecond : CycleFirst + local CycleSecond = {} + + ---@type CycleFirst + cycleValue = nil + "#, + ); + + assert_eq!(ws.expr_ty("cycleValue.missingField"), LuaType::Nil); + } } diff --git a/crates/glua_code_analysis/src/semantic/member/find_members.rs b/crates/glua_code_analysis/src/semantic/member/find_members.rs index d2eebd877..58a40ee3f 100644 --- a/crates/glua_code_analysis/src/semantic/member/find_members.rs +++ b/crates/glua_code_analysis/src/semantic/member/find_members.rs @@ -555,7 +555,14 @@ fn find_unscoped_owner_members( ) -> FindMembersResult { let mut members = Vec::new(); let member_index = db.get_member_index(); - let owner_members = member_index.get_members(owner)?; + // A by-key search reads the index by key rather than walking the owner's + // whole member list. + let owner_members = match filter { + FindMemberFilter::ByKey { member_key, .. } => { + member_index.get_members_with_key(owner, member_key)? + } + FindMemberFilter::All => member_index.get_members(owner)?, + }; for member in owner_members { let member_key = member.get_key().clone(); @@ -849,11 +856,31 @@ fn find_merged_table_members( continue; }; - let mut component_seen: HashSet = HashSet::new(); + // One component can hold several writers of the same key. Keeping only + // the first discards what the rest assign, and which one comes first is + // the member sort order rather than anything the source says - the + // table then disagrees with the union every other reader of that slot + // gets from `resolve_member_item_type`. Union within the component, + // then merge components as table fragments below. + let mut component_members: HashMap = HashMap::new(); + let mut component_order: Vec = Vec::new(); for member in sub_members { - if !component_seen.insert(member.key.clone()) { - continue; + match component_members.entry(member.key.clone()) { + std::collections::hash_map::Entry::Vacant(entry) => { + component_order.push(member.key.clone()); + entry.insert(member); + } + std::collections::hash_map::Entry::Occupied(mut entry) => { + let unioned = crate::TypeOps::Union.apply(db, &entry.get().typ, &member.typ); + entry.get_mut().typ = unioned; + } } + } + + for key in component_order { + let Some(member) = component_members.remove(&key) else { + continue; + }; match members.entry(member.key.clone()) { std::collections::hash_map::Entry::Vacant(entry) => { diff --git a/crates/glua_code_analysis/src/semantic/member/infer_raw_member.rs b/crates/glua_code_analysis/src/semantic/member/infer_raw_member.rs index 0c5d60670..05f30bb33 100644 --- a/crates/glua_code_analysis/src/semantic/member/infer_raw_member.rs +++ b/crates/glua_code_analysis/src/semantic/member/infer_raw_member.rs @@ -167,7 +167,16 @@ fn infer_owner_raw_member_type( return Err(InferFailReason::FieldNotFound); }; - let Some(owner_members) = db.get_member_index().get_members(&member_owner) else { + // See `infer_owner_raw_member_type_with_realm`: a literal access that got + // past the exact-key lookup can only be answered by an expression-keyed + // member, so the rest of the owner's members are not candidates. + let member_index = db.get_member_index(); + let owner_members = if matches!(member_key, LuaMemberKey::Name(_) | LuaMemberKey::Integer(_)) { + member_index.get_expr_key_members(&member_owner) + } else { + member_index.get_members(&member_owner) + }; + let Some(owner_members) = owner_members else { return Err(InferFailReason::FieldNotFound); }; @@ -217,7 +226,16 @@ pub(crate) fn infer_owner_raw_member_type_with_realm( return Err(InferFailReason::FieldNotFound); }; - let Some(owner_members) = db.get_member_index().get_members(&member_owner) else { + // Two literal keys match only when they are equal, which the exact-key + // lookup above already covered, so a literal access that reaches here can + // only be answered by an expression-keyed member. + let member_index = db.get_member_index(); + let owner_members = if matches!(member_key, LuaMemberKey::Name(_) | LuaMemberKey::Integer(_)) { + member_index.get_expr_key_members(&member_owner) + } else { + member_index.get_members(&member_owner) + }; + let Some(owner_members) = owner_members else { return Err(InferFailReason::FieldNotFound); }; diff --git a/crates/glua_code_analysis/src/semantic/member/mod.rs b/crates/glua_code_analysis/src/semantic/member/mod.rs index 9eeed3ab8..72964b82b 100644 --- a/crates/glua_code_analysis/src/semantic/member/mod.rs +++ b/crates/glua_code_analysis/src/semantic/member/mod.rs @@ -5,6 +5,8 @@ mod infer_raw_member; use std::collections::HashSet; +use rustc_hash::FxHashSet; + use crate::{ DbIndex, FileId, GmodStateMask, InFiled, LuaDecl, LuaMemberFeature, LuaMemberId, LuaMemberKey, LuaMemberOwner, LuaSemanticDeclId, TypeOps, @@ -146,11 +148,26 @@ pub(crate) fn local_class_table_member_ids( let member_index = db.get_member_index(); let mut member_ids = Vec::new(); + // A class declared several times in one file names that file once per + // declaration, and each of them would otherwise scan the same declarations. + let mut scanned_files = FxHashSet::default(); for location in type_decl.get_locations() { + if !scanned_files.insert(location.file_id) { + continue; + } let Some(decl_tree) = db.get_decl_index().get_decl_tree(&location.file_id) else { continue; }; for decl in decl_tree.get_decls().values() { + // `local_table_decl_member_owner` opens with these two conditions + // and they are field reads, so they run before the index lookups. + // Keep them in step with it. + if decl + .get_initializer() + .is_none_or(|initializer| initializer.get_ret_idx() != 0) + { + continue; + } if !decl_binds_type(db, decl, type_id) { continue; } diff --git a/crates/glua_code_analysis/src/semantic/mod.rs b/crates/glua_code_analysis/src/semantic/mod.rs index 3b159606a..cb549d54e 100644 --- a/crates/glua_code_analysis/src/semantic/mod.rs +++ b/crates/glua_code_analysis/src/semantic/mod.rs @@ -15,6 +15,11 @@ mod visibility; use std::collections::HashMap; use std::sync::{Arc, Mutex, MutexGuard}; +/// Test-only work counter, re-exported so scaling guards can assert on the +/// number of walks rather than on how long they took. +#[cfg(test)] +pub(crate) use infer::narrow::get_type_at_flow::BASELINE_FLOW_WALKS; + pub use cache::{CacheEntry, CacheOptions, LuaAnalysisPhase, LuaInferCache, PendingStrTplTypeDecl}; pub use decl::{enum_variable_is_param, parse_require_module_info}; use glua_parser::{ @@ -95,9 +100,7 @@ pub(crate) use infer::is_authoritative_self_receiver_type; pub(crate) use infer::remove_false_or_nil; pub(crate) use infer::type_decl_is_vgui_panel; pub use infer::{SelfRefId, VarRefId, VarRefRootId}; -pub(crate) use infer::{ - contains_gmod_null_type, expr_may_have_condition_narrowing, get_var_expr_var_ref_id, -}; +pub(crate) use infer::{contains_gmod_null_type, get_var_expr_var_ref_id}; pub use infer::{infer_param, infer_param_with_cache}; use overload_resolve::resolve_signature; pub use semantic_info::SemanticDeclLevel; diff --git a/crates/glua_code_analysis/src/semantic/type_check/complex_type/mod.rs b/crates/glua_code_analysis/src/semantic/type_check/complex_type/mod.rs index 821f12b3c..4d33589b0 100644 --- a/crates/glua_code_analysis/src/semantic/type_check/complex_type/mod.rs +++ b/crates/glua_code_analysis/src/semantic/type_check/complex_type/mod.rs @@ -15,7 +15,7 @@ use table_generic_check::check_table_generic_type_compact; use tuple_type_check::check_tuple_type_compact; use crate::{ - LuaObjectType, LuaType, LuaUnionType, TypeSubstitutor, + LuaObjectType, LuaType, LuaUnionType, TypeOps, TypeSubstitutor, semantic::{member::find_members, type_check::type_check_context::TypeCheckContext}, }; @@ -196,10 +196,23 @@ fn check_merged_table_type_compact( fn structural_object_from_members(context: &TypeCheckContext, typ: &LuaType) -> Option { let members = find_members(context.db, typ).unwrap_or_default(); - let fields: BTreeMap<_, _> = members - .into_iter() - .map(|member| (member.key, member.typ)) - .collect(); + // A key several members write to is worth all of them. Collecting straight + // into the map would keep whichever one `find_members` happened to yield + // last, so a field written `nil` in one place and a panel in another could + // read as either - and the same table compared against itself would then + // disagree with the union `resolve_member_item_type` gives the other side. + let mut fields: BTreeMap<_, LuaType> = BTreeMap::new(); + for member in members { + match fields.entry(member.key) { + std::collections::btree_map::Entry::Vacant(entry) => { + entry.insert(member.typ); + } + std::collections::btree_map::Entry::Occupied(mut entry) => { + let merged = TypeOps::Union.apply(context.db, entry.get(), &member.typ); + entry.insert(merged); + } + } + } let mut index_access = Vec::new(); collect_index_access_from_type(typ, &mut index_access); if fields.is_empty() && index_access.is_empty() { diff --git a/crates/glua_ls/src/cmd_args.rs b/crates/glua_ls/src/cmd_args.rs index 2aec452bd..85ecc629e 100644 --- a/crates/glua_ls/src/cmd_args.rs +++ b/crates/glua_ls/src/cmd_args.rs @@ -53,6 +53,8 @@ pub enum LogLevel { Info, /// Debug level Debug, + /// Trace level + Trace, } impl std::str::FromStr for LogLevel { @@ -64,8 +66,9 @@ impl std::str::FromStr for LogLevel { "warn" => Ok(LogLevel::Warn), "info" => Ok(LogLevel::Info), "debug" => Ok(LogLevel::Debug), + "trace" => Ok(LogLevel::Trace), _ => Err(format!( - "Invalid log level: '{}'. Please choose 'error', 'warn', 'info', 'debug'", + "Invalid log level: '{}'. Please choose 'error', 'warn', 'info', 'debug', 'trace'", input )), } diff --git a/crates/glua_ls/src/context/debounced_analysis.rs b/crates/glua_ls/src/context/debounced_analysis.rs index dc94beae9..f8ee1b501 100644 --- a/crates/glua_ls/src/context/debounced_analysis.rs +++ b/crates/glua_ls/src/context/debounced_analysis.rs @@ -1,7 +1,8 @@ use glua_code_analysis::{EmmyLuaAnalysis, FileId}; -use std::collections::HashSet; +use lsp_types::Uri; +use std::collections::{HashMap, HashSet}; use std::sync::Arc; -use std::sync::atomic::{AtomicBool, AtomicUsize, Ordering}; +use std::sync::atomic::{AtomicBool, AtomicU8, AtomicUsize, Ordering}; use std::time::{Duration, Instant}; use tokio::sync::{Mutex, Notify, RwLock}; use tokio_util::sync::CancellationToken; @@ -10,16 +11,62 @@ use super::{ClientProxy, file_diagnostic::SharedDiagnosticDataCache}; const FRESHNESS_STUCK_WARN_AFTER: Duration = Duration::from_secs(5); +/// How long the user must stay idle after a reindex before the whole workspace +/// is re-diagnosed. +const IDLE_WORKSPACE_DIAGNOSTIC_DELAY: Duration = Duration::from_millis(2000); + +/// How long the user must stay idle before the dependency ripple runs. +/// +/// The edited file's own re-index runs on the much shorter debounce, which is +/// all a request positioned inside that file needs. This timer governs only the +/// cross-file settle, which holds the write lock for seconds on a large +/// gamemode — and while it does, no keystroke can even be applied. Starting it +/// after every brief pause is what put a ripple in flight for nearly every +/// completion. +const RIPPLE_QUIET: Duration = Duration::from_millis(1000); + +/// Cap on how long one typing burst can hold the ripple off, so diagnostics +/// still settle during sustained typing. +const MAX_RIPPLE_DEFERRAL: Duration = Duration::from_secs(5); + +/// How long the ripple gives requests released by the self-index to take their +/// read lock before it takes the write lock back. +/// +/// Bounded so a stream of requests cannot starve the ripple: past this, the +/// ripple proceeds and the stragglers wait it out as they did before. +const READER_HANDOFF_GRACE: Duration = Duration::from_millis(250); + /// Debounced analysis: accumulates file IDs from rapid edits and runs `reindex_files` once the user pauses typing. pub struct DebouncedAnalysis { pending_files: Mutex>, reindexing_files: Mutex>, + /// Documents whose own index entries do not match their text yet, either + /// because their edit is still queued or because the batch re-indexing them + /// has not reached them. + /// + /// Holds the URI the *client* used, so a request can be tested against its + /// own params without taking the analysis lock — which the re-index holds + /// for its whole duration, so resolving a file id first would wait out + /// exactly what this exists to avoid. Keyed by file id so entries are + /// cleared by identity rather than by matching that URI against the one the + /// VFS derived from a path, which need not be spelled the same way. + blocked_documents: Mutex>, /// True when document changes have arrived but reindex has not yet completed. /// Set synchronously by `begin_in_flight_change()` (called inline in the /// notification handler, before the didChange task is spawned) so that any /// request handler dispatched afterwards sees the flag immediately. has_pending_changes: AtomicBool, in_flight_changes: AtomicUsize, + /// Requests aimed at one document that are waiting for, or reading against, + /// that document's own index entries. + /// + /// The self-index releases them and then immediately queues the ripple's + /// write lock. A woken request still has to be polled before it can queue + /// its read, and the lock is fair-FIFO, so without this the ripple wins the + /// race every time and the request waits out the whole ripple it was just + /// released from. + pending_readers: AtomicUsize, + readers_idle_notify: Notify, notify: Notify, reindex_notify: Notify, analysis: Arc>, @@ -27,6 +74,12 @@ pub struct DebouncedAnalysis { debounce_duration: Duration, shutdown: CancellationToken, client: Arc, + workspace_diagnostic_level: Arc, + lsp_features: Arc, + /// Entries into [`Self::wait_until_fresh_for`], so a test can synchronise + /// on a handler having reached the wait instead of on a deadline. + #[cfg(test)] + freshness_waits: AtomicUsize, } impl DebouncedAnalysis { @@ -36,12 +89,17 @@ impl DebouncedAnalysis { shutdown: CancellationToken, client: Arc, shared_diagnostic_data_cache: SharedDiagnosticDataCache, + workspace_diagnostic_level: Arc, + lsp_features: Arc, ) -> Self { Self { pending_files: Mutex::new(HashSet::new()), reindexing_files: Mutex::new(HashSet::new()), + blocked_documents: Mutex::new(HashMap::new()), has_pending_changes: AtomicBool::new(false), in_flight_changes: AtomicUsize::new(0), + pending_readers: AtomicUsize::new(0), + readers_idle_notify: Notify::new(), notify: Notify::new(), reindex_notify: Notify::new(), analysis, @@ -49,15 +107,23 @@ impl DebouncedAnalysis { debounce_duration: Duration::from_millis(debounce_ms), shutdown, client, + workspace_diagnostic_level, + lsp_features, + #[cfg(test)] + freshness_waits: AtomicUsize::new(0), } } /// Add a file to the pending reindex set and reset the debounce timer. - pub async fn schedule(&self, file_id: FileId) { + pub async fn schedule(&self, file_id: FileId, uri: Uri) { { let mut pending = self.pending_files.lock().await; pending.insert(file_id); } + { + let mut blocked = self.blocked_documents.lock().await; + blocked.insert(file_id, uri); + } self.has_pending_changes.store(true, Ordering::Release); self.notify.notify_waiters(); } @@ -121,6 +187,57 @@ impl DebouncedAnalysis { self.in_flight_changes.load(Ordering::Acquire) } + #[cfg(test)] + pub(crate) fn freshness_wait_count(&self) -> usize { + self.freshness_waits.load(Ordering::Acquire) + } + + /// Register that a request is waiting on, or reading against, one + /// document's own index entries. + /// + /// Hold the guard until the request has finished reading. The ripple yields + /// to outstanding guards — up to [`READER_HANDOFF_GRACE`] — before it takes + /// the write lock back, so a request the self-index just released is not + /// made to wait out the ripple anyway. + pub fn begin_reader_handoff(self: &Arc) -> ReaderHandoff { + self.pending_readers.fetch_add(1, Ordering::AcqRel); + ReaderHandoff { + analysis: self.clone(), + } + } + + /// Let outstanding [`ReaderHandoff`]s take their read lock before the + /// caller takes the write lock. + /// + /// Returns as soon as none are outstanding, or after + /// [`READER_HANDOFF_GRACE`] so a stream of requests cannot starve the + /// ripple. + async fn await_reader_handoff(&self) { + let deadline = Instant::now() + READER_HANDOFF_GRACE; + + loop { + // Register before testing, or a drop landing in between is lost. + let idle = self.readers_idle_notify.notified(); + tokio::pin!(idle); + idle.as_mut().enable(); + + if self.pending_readers.load(Ordering::Acquire) == 0 { + return; + } + + let remaining = deadline.saturating_duration_since(Instant::now()); + if remaining.is_zero() { + return; + } + + tokio::select! { + _ = idle => {} + _ = tokio::time::sleep(remaining) => return, + _ = self.shutdown.cancelled() => return, + } + } + } + /// Wait until all pending document changes have been reindexed. /// /// Returns `true` when the analysis is fresh, `false` if the cancel token @@ -131,15 +248,15 @@ impl DebouncedAnalysis { cancel_token: &CancellationToken, request_method: &'static str, ) -> bool { + #[cfg(test)] + self.freshness_waits.fetch_add(1, Ordering::AcqRel); + let started_at = Instant::now(); let mut warned_stuck = false; loop { - // Create and enable the Notified future BEFORE checking the - // condition. `enable()` ensures that a `notify_waiters()` call - // between here and the `select!` poll is captured, avoiding a - // missed wakeup (unpolled Notified futures are invisible to - // `notify_waiters` without `enable`). + // Register (`enable()`) before testing the condition: unpolled + // Notified futures are invisible to `notify_waiters()`. let notified = self.reindex_notify.notified(); tokio::pin!(notified); notified.as_mut().enable(); @@ -161,6 +278,78 @@ impl DebouncedAnalysis { } } + /// Wait until the document at `uri` has index entries matching its text. + /// + /// A request positioned inside a file needs that file's entries to line up + /// with the tree it is resolving against — they are keyed by position, so an + /// edit that shifts offsets is what makes them stop matching, and answering + /// from the old ones is what silently returns a thinner list. It does *not* + /// need the edit's dependency ripple to have finished; that settles other + /// files' inferences, and waiting for it costs seconds on a large gamemode + /// for an answer that is already correct. + /// + /// Callers with no URI to aim at want [`wait_until_fresh_for`] instead. + /// + /// [`wait_until_fresh_for`]: Self::wait_until_fresh_for + pub async fn wait_until_file_fresh_for( + &self, + cancel_token: &CancellationToken, + request_method: &'static str, + uri: &Uri, + ) -> bool { + #[cfg(test)] + self.freshness_waits.fetch_add(1, Ordering::AcqRel); + + let started_at = Instant::now(); + let mut warned_stuck = false; + + loop { + let notified = self.reindex_notify.notified(); + tokio::pin!(notified); + notified.as_mut().enable(); + + if self.file_is_answerable(uri).await { + return true; + } + + let remaining = FRESHNESS_STUCK_WARN_AFTER.saturating_sub(started_at.elapsed()); + + tokio::select! { + _ = notified => {} + _ = cancel_token.cancelled() => return false, + _ = tokio::time::sleep(remaining), if !warned_stuck => { + self.log_freshness_stuck(request_method, started_at).await; + warned_stuck = true; + } + } + } + } + + /// Whether a request aimed at `uri` can be answered against entries that + /// match its text. + /// + /// Workspace-wide dirtiness is the wrong question for a request positioned + /// inside one document: an edit to some other file leaves this one's entries + /// matching its own text, so refusing it buys nothing and blanks the answer + /// for every open document whenever anything is typed anywhere. + pub async fn file_is_answerable(&self, uri: &Uri) -> bool { + // An edit whose text has not been applied yet would have the request + // resolve a position against the previous tree. + if self.in_flight_changes.load(Ordering::Acquire) > 0 { + return false; + } + if !self.has_pending_changes.load(Ordering::Acquire) { + return true; + } + // Only ever a handful of documents are mid-edit at once. + !self + .blocked_documents + .lock() + .await + .values() + .any(|blocked| blocked == uri) + } + async fn log_freshness_stuck(&self, request_method: &'static str, started_at: Instant) { let in_flight = self.in_flight_changes.load(Ordering::Acquire); let pending_count = self.pending_files.lock().await.len(); @@ -197,7 +386,43 @@ impl DebouncedAnalysis { } } - async fn reindex_files_without_queuing(&self, file_ids: Vec) -> bool { + /// Re-index the edited files' own entries, and report the dependency + /// expansion the ripple still owes them. + /// + /// The expansion is captured *before* the self-index, because deriving it + /// from a partly-updated index under-expands and leaves dependents holding + /// inferences a cold build would not produce. + /// + /// This takes the write lock and gives it back, which is the whole point: a + /// freshness flag published while the lock is still held buys a waiting + /// request nothing, since it cannot read the index until the lock is free. + async fn self_index_without_queuing(&self, file_ids: Vec) -> Option> { + let analysis = self.analysis.clone(); + let cache = self.shared_diagnostic_data_cache.clone(); + + tokio::select! { + _ = self.shutdown.cancelled() => None, + result = tokio::task::spawn_blocking(move || { + let mut guard = analysis.blocking_write(); + let expansion = guard.expand_reindex_file_ids(file_ids.clone()); + guard.self_index_files(file_ids); + cache.invalidate(); + expansion + }) => match result { + Ok(expansion) => Some(expansion), + Err(err) => { + log::error!("self-index task failed: {}", err); + None + } + } + } + } + + async fn reindex_files_without_queuing( + &self, + file_ids: Vec, + expansion: Vec, + ) -> bool { let analysis = self.analysis.clone(); let cache = self.shared_diagnostic_data_cache.clone(); @@ -207,7 +432,7 @@ impl DebouncedAnalysis { _ = self.shutdown.cancelled() => false, result = tokio::task::spawn_blocking(move || { let mut guard = analysis.blocking_write(); - guard.reindex_files(file_ids); + guard.reindex_expanded_files(file_ids, expansion); // Invalidate under the write lock so no reader can observe the // fresh index next to the stale shared diagnostic data. cache.invalidate(); @@ -221,24 +446,57 @@ impl DebouncedAnalysis { } } + /// Hold the owed ripple until typing has stopped for [`RIPPLE_QUIET`]. + /// + /// Returns `true` when the caller should run the ripple now, `false` when + /// another edit arrived and the loop should self-index that first — the + /// ripple it owes then joins the one already outstanding. + async fn ripple_quiet_elapsed(&self, burst_started_at: Instant) -> bool { + let extra = RIPPLE_QUIET.saturating_sub(self.debounce_duration); + let deferral_left = MAX_RIPPLE_DEFERRAL.saturating_sub(burst_started_at.elapsed()); + if extra.is_zero() || deferral_left.is_zero() { + return true; + } + + tokio::select! { + biased; + _ = self.shutdown.cancelled() => return true, + _ = self.notify.notified() => return false, + _ = tokio::time::sleep(extra.min(deferral_left)) => {} + } + + // A notify landing before the select registered would be lost, so the + // timer expiring is not on its own proof that nothing arrived. + self.pending_files.lock().await.is_empty() + } + /// Background loop: waits for events, debounces, then runs reindex. /// Spawn this once at server startup. pub async fn run(&self) { + let mut idle_workspace_diagnostic_token: Option = None; + // The ripple owed by the self-indexes run so far in this typing burst, + // and the union of the expansions each of them captured. + let mut owed_files: HashSet = HashSet::new(); + let mut owed_expansion: HashSet = HashSet::new(); + let mut burst_started_at: Option = None; loop { - // Wait for the first event, unless files were scheduled during - // the previous reindex (the Notify signal may have been missed - // because there was no active waiter at that point), or - // begin_in_flight_change() was called without a corresponding schedule(). + // Register before testing the condition: `notify_waiters()` stores + // no permit, so a signal landing in between would be lost. + let notified = self.notify.notified(); + tokio::pin!(notified); + notified.as_mut().enable(); + let needs_work = !self.pending_files.lock().await.is_empty() - || self.has_pending_changes.load(Ordering::Acquire); + || self.has_pending_changes.load(Ordering::Acquire) + || !owed_files.is_empty(); if !needs_work { tokio::select! { - _ = self.notify.notified() => {} + _ = notified => {} _ = self.shutdown.cancelled() => return, } } - // Debounce: keep resetting the timer while new events arrive + // Debounce: keep resetting the timer while new events arrive. loop { tokio::select! { biased; @@ -267,25 +525,147 @@ impl DebouncedAnalysis { self.debounce_duration.as_millis() ); - let reindex_completed = self.reindex_files_without_queuing(file_ids.clone()).await; + // Re-index the edited files themselves first and release the + // write lock, so a completion or hover positioned inside one of + // them can be answered against entries that match its text + // instead of waiting out the whole dependency ripple. The + // ripple is by far the larger half — measured on a gamemode + // workspace, 106ms against 5.1s. + let Some(expansion) = self.self_index_without_queuing(file_ids.clone()).await + else { + if self.shutdown.is_cancelled() { + return; + } + // Release the batch, or every request aimed at these + // documents parks until some later edit happens to cover + // them. + let mut reindexing = self.reindexing_files.lock().await; + let mut blocked = self.blocked_documents.lock().await; + for id in &file_ids { + reindexing.remove(id); + blocked.remove(id); + } + drop(blocked); + drop(reindexing); + self.refresh_dirty_state().await; + self.reindex_notify.notify_waiters(); + continue; + }; + + { + let mut blocked = self.blocked_documents.lock().await; + for file_id in &file_ids { + blocked.remove(file_id); + } + } + self.reindex_notify.notify_waiters(); + + // The requests just released still have to be polled before + // they can queue their read. Taking the write lock back now + // would put them behind the whole ripple. + self.await_reader_handoff().await; + + owed_files.extend(file_ids.iter().copied()); + owed_expansion.extend(expansion); + burst_started_at.get_or_insert_with(Instant::now); + } + + if owed_files.is_empty() { + self.refresh_dirty_state().await; + self.reindex_notify.notify_waiters(); + continue; + } + + // Hold the ripple until typing has genuinely stopped. Another edit + // sends us back for its own self-index, and the ripple it owes + // joins this one. + let burst_started_at_instant = burst_started_at.unwrap_or_else(Instant::now); + if !self.ripple_quiet_elapsed(burst_started_at_instant).await { + continue; + } + + { + let ripple_files: Vec = { + let mut ids: Vec = owed_files.iter().copied().collect(); + ids.sort(); + ids + }; + let ripple_expansion: Vec = { + let mut ids: Vec = owed_expansion.iter().copied().collect(); + ids.sort(); + ids + }; + log::info!( + "ripple: {} edited file(s) over {} file(s) after {}ms quiet", + ripple_files.len(), + ripple_expansion.len(), + RIPPLE_QUIET.as_millis() + ); + + let reindex_completed = self + .reindex_files_without_queuing(ripple_files.clone(), ripple_expansion) + .await; { let mut reindexing = self.reindexing_files.lock().await; - for id in &file_ids { + for id in &ripple_files { reindexing.remove(id); } } + owed_files.clear(); + owed_expansion.clear(); + burst_started_at = None; self.reindex_notify.notify_waiters(); if !reindex_completed { - return; + // Only shutdown stops the loop; a panicked reindex must + // fall through so `refresh_dirty_state()` releases waiters. + if self.shutdown.is_cancelled() { + return; + } + log::error!( + "LS_REINDEX_FAILED reindex of {} file(s) did not complete; continuing so freshness waiters are released", + ripple_files.len() + ); } - // Trigger diagnostic and semantic token refresh so the client - // re-pulls with fresh data after the reindex. - self.client.refresh_workspace_diagnostics(); - self.client.refresh_semantic_tokens(); - self.client.refresh_inlay_hints(); + if self.lsp_features.supports_semantic_tokens_refresh() { + self.client.refresh_semantic_tokens(); + } + if self.lsp_features.supports_inlay_hint_refresh() { + self.client.refresh_inlay_hints(); + } + + // Arm an idle workspace diagnostic refresh so closed files hit + // by cross-file changes get re-diagnosed once typing pauses. + if let Some(token) = idle_workspace_diagnostic_token.take() { + token.cancel(); + } + let cancel_token = CancellationToken::new(); + idle_workspace_diagnostic_token = Some(cancel_token.clone()); + + let client = self.client.clone(); + let status = self.workspace_diagnostic_level.clone(); + let lsp_features = self.lsp_features.clone(); + let shutdown = self.shutdown.clone(); + tokio::spawn(async move { + tokio::select! { + _ = tokio::time::sleep(IDLE_WORKSPACE_DIAGNOSTIC_DELAY) => { + if !cancel_token.is_cancelled() && !shutdown.is_cancelled() { + // Raise, never lower: don't drop a pending Slow sweep. + status.fetch_max( + crate::context::WorkspaceDiagnosticLevel::Fast.to_u8(), + Ordering::AcqRel, + ); + if lsp_features.supports_refresh_diagnostic() { + client.refresh_workspace_diagnostics(); + } + } + } + _ = cancel_token.cancelled() => {} + _ = shutdown.cancelled() => {} + } + }); } self.refresh_dirty_state().await; @@ -298,20 +678,44 @@ impl DebouncedAnalysis { } async fn refresh_dirty_state(&self) { - let has_pending_file_work = { - let pending = self.pending_files.lock().await; - if !pending.is_empty() { - true - } else { - let reindexing = self.reindexing_files.lock().await; - !reindexing.is_empty() - } - }; + // Publish while holding both locks, or concurrent callers can + // interleave and store a stale reading. + let pending = self.pending_files.lock().await; + let reindexing = self.reindexing_files.lock().await; + + let has_pending_file_work = !pending.is_empty() || !reindexing.is_empty(); let has_in_flight_changes = self.in_flight_changes.load(Ordering::Acquire) > 0; + self.has_pending_changes.store( has_pending_file_work || has_in_flight_changes, Ordering::Release, ); + + // `in_flight_changes` is not covered by the locks above, so a + // concurrent `begin_in_flight_change()` can land between the load and + // the store and have its `true` overwritten. Re-reading narrows that + // window rather than closing it; what is guaranteed is only that the + // flag ends up `true` for any change whose `fetch_add` is visible by + // the time this second load runs. A change that arrives later still + // sets the flag itself, and `finish_in_flight_changes` calls back here. + if self.in_flight_changes.load(Ordering::Acquire) > 0 { + self.has_pending_changes.store(true, Ordering::Release); + } + } +} + +/// Keeps the ripple off the write lock while one request takes its read lock. +/// +/// See [`DebouncedAnalysis::begin_reader_handoff`]. +pub struct ReaderHandoff { + analysis: Arc, +} + +impl Drop for ReaderHandoff { + fn drop(&mut self) { + if self.analysis.pending_readers.fetch_sub(1, Ordering::AcqRel) == 1 { + self.analysis.readers_idle_notify.notify_waiters(); + } } } @@ -361,24 +765,33 @@ impl Drop for InFlightChangeGuard { #[cfg(test)] mod tests { use std::sync::Arc; - use std::time::Duration; + use std::sync::atomic::AtomicU8; + use std::time::{Duration, Instant}; + + use super::{MAX_RIPPLE_DEFERRAL, READER_HANDOFF_GRACE}; - use glua_code_analysis::{DiagnosticCode, EmmyLuaAnalysis, file_path_to_uri}; + use glua_code_analysis::{DiagnosticCode, EmmyLuaAnalysis, FileId, file_path_to_uri}; use googletest::prelude::*; use lsp_server::Connection; - use lsp_types::{Diagnostic, NumberOrString}; + use lsp_types::Uri; + use lsp_types::{ClientCapabilities, Diagnostic, NumberOrString}; + use std::str::FromStr; use tokio::sync::RwLock; use tokio_util::sync::CancellationToken; - use crate::context::{ClientProxy, FileDiagnostic, StatusBar}; + use crate::context::{ClientProxy, FileDiagnostic, LspFeatures, StatusBar}; use super::DebouncedAnalysis; + fn test_lsp_features() -> Arc { + Arc::new(LspFeatures::new(ClientCapabilities::default())) + } + fn test_debounced_analysis() -> Arc { let analysis = Arc::new(RwLock::new(EmmyLuaAnalysis::new())); let (connection, _peer) = Connection::memory(); let client = Arc::new(ClientProxy::new(connection)); - let status_bar = Arc::new(StatusBar::new(client.clone())); + let status_bar = Arc::new(StatusBar::new(client.clone(), true)); let file_diagnostic = FileDiagnostic::new(analysis.clone(), status_bar, client.clone()); Arc::new(DebouncedAnalysis::new( analysis, @@ -386,6 +799,8 @@ mod tests { CancellationToken::new(), client, file_diagnostic.shared_diagnostic_data_cache(), + Arc::new(AtomicU8::new(0)), + test_lsp_features(), )) } @@ -441,6 +856,153 @@ mod tests { }) } + /// Typing must send the ripple back rather than let it start: while it + /// runs, no keystroke can even be applied. + #[gtest] + fn a_new_edit_sends_the_ripple_back() -> Result<()> { + let runtime = tokio::runtime::Runtime::new().expect("tokio runtime should build"); + runtime.block_on(async { + let debounced_analysis = test_debounced_analysis(); + let uri = Uri::from_str("file:///workspace/edited.lua").expect("uri should parse"); + + let scheduler = debounced_analysis.clone(); + tokio::spawn(async move { + tokio::time::sleep(Duration::from_millis(20)).await; + scheduler.schedule(FileId { id: 1 }, uri).await; + }); + + let run_the_ripple = tokio::time::timeout( + Duration::from_millis(500), + debounced_analysis.ripple_quiet_elapsed(Instant::now()), + ) + .await + .expect("an edit should send the ripple back well inside the quiet window"); + + verify_that!(run_the_ripple, eq(false))?; + Ok(()) + }) + } + + /// Sustained typing must not hold diagnostics off indefinitely. + #[gtest] + fn a_long_burst_cannot_hold_the_ripple_off_forever() -> Result<()> { + let runtime = tokio::runtime::Runtime::new().expect("tokio runtime should build"); + runtime.block_on(async { + let debounced_analysis = test_debounced_analysis(); + let uri = Uri::from_str("file:///workspace/edited.lua").expect("uri should parse"); + debounced_analysis.schedule(FileId { id: 1 }, uri).await; + + let started_at = Instant::now() + .checked_sub(MAX_RIPPLE_DEFERRAL) + .expect("the deferral cap should fit before now"); + + let run_the_ripple = tokio::time::timeout( + Duration::from_millis(100), + debounced_analysis.ripple_quiet_elapsed(started_at), + ) + .await + .expect("the cap should release the ripple immediately"); + + verify_that!(run_the_ripple, eq(true))?; + Ok(()) + }) + } + + /// The ripple must yield to a request the self-index just released, or the + /// request queues behind the write lock and waits out the ripple anyway. + #[gtest] + fn the_ripple_waits_for_an_outstanding_reader() -> Result<()> { + let runtime = tokio::runtime::Runtime::new().expect("tokio runtime should build"); + runtime.block_on(async { + let debounced_analysis = test_debounced_analysis(); + + // Nothing outstanding: the ripple must not pay the grace. + tokio::time::timeout( + Duration::from_millis(50), + debounced_analysis.await_reader_handoff(), + ) + .await + .expect("no readers should let the ripple straight through"); + + let handoff = debounced_analysis.begin_reader_handoff(); + let held = tokio::time::timeout( + Duration::from_millis(50), + debounced_analysis.await_reader_handoff(), + ) + .await; + verify_that!(held.is_err(), eq(true))?; + + drop(handoff); + tokio::time::timeout( + Duration::from_millis(250), + debounced_analysis.await_reader_handoff(), + ) + .await + .expect("dropping the last handoff should release the ripple"); + + Ok(()) + }) + } + + /// A stream of requests must not starve the ripple. + #[gtest] + fn an_outstanding_reader_only_delays_the_ripple_by_the_grace() -> Result<()> { + let runtime = tokio::runtime::Runtime::new().expect("tokio runtime should build"); + runtime.block_on(async { + let debounced_analysis = test_debounced_analysis(); + let _never_dropped = debounced_analysis.begin_reader_handoff(); + + let started_at = Instant::now(); + debounced_analysis.await_reader_handoff().await; + + verify_that!(started_at.elapsed() >= READER_HANDOFF_GRACE, eq(true))?; + verify_that!(started_at.elapsed() < READER_HANDOFF_GRACE * 4, eq(true))?; + Ok(()) + }) + } + + /// The point of the per-file gate: an edit to one document must not park + /// requests aimed at a different one, and must park requests aimed at + /// itself until its own entries have been rebuilt. + #[gtest] + fn a_pending_edit_blocks_only_its_own_document() -> Result<()> { + let runtime = tokio::runtime::Runtime::new().expect("tokio runtime should build"); + runtime.block_on(async { + let debounced_analysis = test_debounced_analysis(); + let edited = Uri::from_str("file:///workspace/edited.lua").expect("uri should parse"); + let untouched = + Uri::from_str("file:///workspace/untouched.lua").expect("uri should parse"); + + debounced_analysis + .schedule(FileId { id: 1 }, edited.clone()) + .await; + + let cancel = CancellationToken::new(); + let untouched_answered = tokio::time::timeout( + Duration::from_millis(250), + debounced_analysis.wait_until_file_fresh_for( + &cancel, + "textDocument/completion", + &untouched, + ), + ) + .await; + verify_that!(untouched_answered.unwrap_or(false), eq(true))?; + + let edited_answered = tokio::time::timeout( + Duration::from_millis(250), + debounced_analysis.wait_until_file_fresh_for( + &cancel, + "textDocument/completion", + &edited, + ), + ) + .await; + verify_that!(edited_answered.is_err(), eq(true))?; + Ok(()) + }) + } + #[gtest] fn finish_in_flight_changes_saturates_underflow() -> Result<()> { let runtime = tokio::runtime::Runtime::new().expect("tokio runtime should build"); @@ -480,7 +1042,7 @@ mod tests { let (connection, _peer) = Connection::memory(); let client = Arc::new(ClientProxy::new(connection)); - let status_bar = Arc::new(StatusBar::new(client.clone())); + let status_bar = Arc::new(StatusBar::new(client.clone(), true)); let analysis = Arc::new(RwLock::new(analysis)); let file_diagnostic = FileDiagnostic::new(analysis.clone(), status_bar.clone(), client.clone()); @@ -514,10 +1076,12 @@ mod tests { CancellationToken::new(), client, file_diagnostic.shared_diagnostic_data_cache(), + Arc::new(AtomicU8::new(0)), + test_lsp_features(), ); verify_that!( debounced_analysis - .reindex_files_without_queuing(vec![api_file_id]) + .reindex_files_without_queuing(vec![api_file_id], vec![api_file_id]) .await, eq(true) )?; diff --git a/crates/glua_ls/src/context/did_change_coalescer.rs b/crates/glua_ls/src/context/did_change_coalescer.rs index c07e87257..29e261c5f 100644 --- a/crates/glua_ls/src/context/did_change_coalescer.rs +++ b/crates/glua_ls/src/context/did_change_coalescer.rs @@ -56,7 +56,11 @@ impl DidChangeCoalescer { // Wait for at least one message. let first = match rx.recv().await { Some(params) => params, - None => return, // channel closed + None => { + // Every sender is gone: shutdown. + log::info!("didChange coalescer stopped: channel closed"); + return; + } }; // Drain remaining messages without blocking. diff --git a/crates/glua_ls/src/context/file_diagnostic.rs b/crates/glua_ls/src/context/file_diagnostic.rs index acb6beadb..3859fa008 100644 --- a/crates/glua_ls/src/context/file_diagnostic.rs +++ b/crates/glua_ls/src/context/file_diagnostic.rs @@ -142,6 +142,10 @@ impl FileDiagnostic { self.shared_diagnostic_data_cache.invalidate(); } + pub fn is_workspace_loaded(&self) -> bool { + self.workspace_loaded_notified.load(Ordering::Acquire) + } + pub fn notify_workspace_loaded(&self) { if self.workspace_loaded_notified.swap(true, Ordering::AcqRel) { return; @@ -292,7 +296,11 @@ impl FileDiagnostic { } } - /// 清除指定文件的诊断信息 + /// Drop the remembered report for a URI without telling the client. + pub async fn forget_cached_file_diagnostics(&self, uri: &Uri) { + self.cached_file_diagnostics.lock().await.remove(uri); + } + pub async fn clear_push_file_diagnostics(&self, uri: lsp_types::Uri) { self.cached_file_diagnostics.lock().await.remove(&uri); @@ -452,11 +460,14 @@ impl FileDiagnostic { valid_file_count, ); } + let in_flight = Arc::new(InFlightDiagnosticFiles::default()); + watchdog_status.set_detail_source(in_flight.detail_source(self.analysis.clone())); let mut rx = spawn_workspace_diagnostic_workers( self.analysis.clone(), main_workspace_file_ids, shared_data, cancel_token.clone(), + in_flight, ); let mut count = 0; @@ -476,6 +487,13 @@ impl FileDiagnostic { } count += 1; let percentage_done = ((count as f32 / valid_file_count as f32) * 100.0) as u32; + // The watchdog tracks every file; only the notification to + // the client is rate-limited on the percentage. + watchdog_status.set_progress( + "Diagnosing workspace files (slow pull)", + count, + valid_file_count, + ); if last_percentage != percentage_done { last_percentage = percentage_done; let message = format!( @@ -487,11 +505,6 @@ impl FileDiagnostic { Some(percentage_done), Some(message), ); - watchdog_status.set_progress( - "Diagnosing workspace files (slow pull)", - count, - valid_file_count, - ); } } } @@ -575,11 +588,14 @@ impl FileDiagnostic { ); } + let in_flight = Arc::new(InFlightDiagnosticFiles::default()); + watchdog_status.set_detail_source(in_flight.detail_source(self.analysis.clone())); let mut rx = spawn_workspace_diagnostic_workers( self.analysis.clone(), main_workspace_file_ids, shared_data, cancel_token.clone(), + in_flight, ); let mut count = 0; @@ -602,6 +618,12 @@ impl FileDiagnostic { count += 1; let percentage_done = ((count as f32 / valid_file_count as f32) * 100.0) as u32; + // See the slow pull above. + watchdog_status.set_progress( + "Diagnosing workspace files (fast pull)", + count, + valid_file_count, + ); if last_percentage != percentage_done { last_percentage = percentage_done; let message = format!( @@ -613,11 +635,6 @@ impl FileDiagnostic { Some(percentage_done), Some(message), ); - watchdog_status.set_progress( - "Diagnosing workspace files (fast pull)", - count, - valid_file_count, - ); } } } @@ -670,6 +687,7 @@ fn spawn_workspace_diagnostic_workers( file_ids: Vec, shared_data: Arc, cancel_token: CancellationToken, + in_flight: Arc, ) -> tokio::sync::mpsc::Receiver { let worker_count = workspace_diagnostic_parallelism().min(file_ids.len()); let file_ids = Arc::new(file_ids); @@ -682,15 +700,22 @@ fn spawn_workspace_diagnostic_workers( let next_file = next_file.clone(); let shared_data = shared_data.clone(); let cancel_token = cancel_token.clone(); + let in_flight = in_flight.clone(); let tx = tx.clone(); + // Per worker, never per file: fern flushes every record, so a line per + // file turns a workspace sweep into a log-I/O hotspot. Per-file + // visibility comes from `InFlightDiagnosticFiles` via the watchdog. tokio::spawn(async move { loop { if cancel_token.is_cancelled() { + log::trace!("workspace diagnostic worker exiting: cancelled"); break; } let Some(file_id) = claim_next_diagnostic_file(&file_ids, &next_file) else { + log::trace!("workspace diagnostic worker exiting: queue drained"); break; }; + in_flight.claim(file_id); let result = diagnose_workspace_file_off_thread( analysis.clone(), file_id, @@ -698,7 +723,9 @@ fn spawn_workspace_diagnostic_workers( cancel_token.clone(), ) .await; + in_flight.release(file_id); if tx.send(result).await.is_err() { + log::trace!("workspace diagnostic worker exiting: receiver gone"); break; } } @@ -708,6 +735,84 @@ fn spawn_workspace_diagnostic_workers( rx } +/// The files the sweep has started and not finished, with when each was +/// claimed, so a watchdog line can name the longest-running one. +#[derive(Default)] +pub struct InFlightDiagnosticFiles { + files: std::sync::Mutex>, +} + +impl InFlightDiagnosticFiles { + fn claim(&self, file_id: FileId) { + if let Ok(mut files) = self.files.lock() { + files.push((file_id, std::time::Instant::now())); + } + } + + fn release(&self, file_id: FileId) { + if let Ok(mut files) = self.files.lock() + && let Some(index) = files.iter().position(|(id, _)| *id == file_id) + { + files.remove(index); + } + } + + /// The in-flight files, longest-running first. + fn oldest_first(&self) -> Vec<(FileId, std::time::Duration)> { + let Ok(files) = self.files.lock() else { + return Vec::new(); + }; + let mut entries = files + .iter() + .map(|(id, started)| (*id, started.elapsed())) + .collect::>(); + entries.sort_by_key(|(_, elapsed)| std::cmp::Reverse(*elapsed)); + entries + } + + /// A watchdog detail source naming the files the sweep is still inside. + pub fn detail_source( + self: &Arc, + analysis: Arc>, + ) -> crate::util::WatchdogDetailSource { + let in_flight = self.clone(); + Arc::new(move || { + let entries = in_flight.oldest_first(); + if entries.is_empty() { + return None; + } + // `try_read`, because a sweep stuck holding the lock must still get + // a line out. Ids are reported bare when the guard is unavailable. + let described = match analysis.try_read() { + Ok(analysis) => { + let vfs = analysis.compilation.get_db().get_vfs(); + entries + .iter() + .take(3) + .map(|(file_id, elapsed)| { + let path = vfs + .get_file_path(file_id) + .map(|path| path.to_string_lossy().to_string()) + .unwrap_or_else(|| format!("{file_id:?}")); + format!("{path} ({}s)", elapsed.as_secs()) + }) + .collect::>() + } + Err(_) => entries + .iter() + .take(3) + .map(|(file_id, elapsed)| format!("{file_id:?} ({}s)", elapsed.as_secs())) + .collect::>(), + }; + Some(format!( + "{} file(s) still being diagnosed, longest first: {}", + entries.len(), + described.join(", ") + )) + }) + } +} + fn claim_next_diagnostic_file(file_ids: &[FileId], next_file: &AtomicUsize) -> Option { let index = next_file.fetch_add(1, Ordering::Relaxed); file_ids.get(index).copied() @@ -799,11 +904,14 @@ async fn push_workspace_diagnostic( watchdog_status.set_progress("Diagnosing workspace files (push)", 0, valid_file_count); } + let in_flight = Arc::new(InFlightDiagnosticFiles::default()); + watchdog_status.set_detail_source(in_flight.detail_source(analysis.clone())); let mut rx = spawn_workspace_diagnostic_workers( analysis, main_workspace_file_ids, shared_data, cancel_token.clone(), + in_flight, ); let mut count = 0; @@ -828,6 +936,12 @@ async fn push_workspace_diagnostic( } count += 1; let percentage_done = ((count as f32 / valid_file_count as f32) * 100.0) as u32; + // See the slow pull above. + watchdog_status.set_progress( + "Diagnosing workspace files (push)", + count, + valid_file_count, + ); if last_percentage != percentage_done { last_percentage = percentage_done; if !silent { @@ -841,11 +955,6 @@ async fn push_workspace_diagnostic( Some(message), ); } - watchdog_status.set_progress( - "Diagnosing workspace files (push)", - count, - valid_file_count, - ); } } } @@ -915,7 +1024,7 @@ mod tests { fn workspace_loaded_notification_does_not_suppress_startup_complete() -> Result<()> { let (connection, peer) = Connection::memory(); let client = Arc::new(ClientProxy::new(connection)); - let status_bar = Arc::new(StatusBar::new(client.clone())); + let status_bar = Arc::new(StatusBar::new(client.clone(), true)); let analysis = Arc::new(RwLock::new(EmmyLuaAnalysis::new())); let file_diagnostic = FileDiagnostic::new(analysis, status_bar, client); @@ -975,7 +1084,7 @@ mod tests { let (connection, _peer) = Connection::memory(); let client = Arc::new(ClientProxy::new(connection)); - let status_bar = Arc::new(StatusBar::new(client.clone())); + let status_bar = Arc::new(StatusBar::new(client.clone(), true)); let analysis = Arc::new(RwLock::new(analysis)); let file_diagnostic = FileDiagnostic::new(analysis.clone(), status_bar, client); diff --git a/crates/glua_ls/src/context/lsp_features.rs b/crates/glua_ls/src/context/lsp_features.rs index 7400a0298..9aec880f8 100644 --- a/crates/glua_ls/src/context/lsp_features.rs +++ b/crates/glua_ls/src/context/lsp_features.rs @@ -24,6 +24,24 @@ impl LspFeatures { false } + /// Gates `window/workDoneProgress/create` and its `$/progress` traffic. + pub fn supports_work_done_progress(&self) -> bool { + self.client_capabilities + .window + .as_ref() + .and_then(|window| window.work_done_progress) + .unwrap_or(false) + } + + /// Gates `workspace/applyEdit`. + pub fn supports_apply_edit(&self) -> bool { + self.client_capabilities + .workspace + .as_ref() + .and_then(|workspace| workspace.apply_edit) + .unwrap_or(false) + } + pub fn supports_config_request(&self) -> bool { if let Some(workspace) = &self.client_capabilities.workspace { if let Some(supports) = workspace.configuration { diff --git a/crates/glua_ls/src/context/mod.rs b/crates/glua_ls/src/context/mod.rs index 80e06a59e..fb01c820f 100644 --- a/crates/glua_ls/src/context/mod.rs +++ b/crates/glua_ls/src/context/mod.rs @@ -20,88 +20,34 @@ use lsp_types::{ClientCapabilities, Uri}; pub use snapshot::ServerContextSnapshot; pub use status_bar::ProgressTask; pub use status_bar::StatusBar; -use std::{collections::HashMap, future::Future, sync::Arc}; +use std::{collections::HashMap, future::Future, sync::Arc, time::Duration}; use tokio::sync::{Mutex, Notify, RwLock}; use tokio_util::sync::CancellationToken; pub use workspace_manager::*; use crate::context::snapshot::ServerContextInner; -// ============================================================================ -// LOCK ORDERING GUIDELINES (CRITICAL - Must Follow to Avoid Deadlocks) -// ============================================================================ -// -// This module uses multiple locks (RwLock and Mutex) for concurrent access to shared state. -// To prevent deadlocks, **ALL code must acquire locks in the following order**: -// -// ## Global Lock Order (Low to High Priority): -// 1. **diagnostic_tokens** (Mutex) - File diagnostic task tokens -// 2. **workspace_diagnostic_token** (Mutex) - Workspace diagnostic task token -// 3. **cached_file_diagnostics** (Mutex) - UI state -// 4. **update_token** (Mutex) - Reindex/config update token -// 5. **analysis** (RwLock - READ) - Read-only access to EmmyLuaAnalysis -// 6. **workspace_manager** (RwLock - READ) - Read-only access to WorkspaceManager -// 7. **workspace_manager** (RwLock - WRITE) - Exclusive access to WorkspaceManager -// 8. **analysis** (RwLock - WRITE) - Exclusive access to EmmyLuaAnalysis -// -// ## Lock Ordering Rules: -// - **NEVER acquire a lower-priority lock while holding a higher-priority lock** -// - **ALWAYS release locks in reverse order (LIFO) or use explicit scope blocks** -// - **NEVER upgrade a read lock to a write lock (release read, then acquire write)** -// - **Minimize lock scope**: only hold locks for the minimum necessary time -// - **Avoid holding locks across `.await` points when possible** -// - **NEVER call async methods that might acquire locks while holding a lock** -// -// ## Examples: -// -// ### ✅ CORRECT - Proper lock ordering: -// ```rust -// // Acquire workspace_manager read lock first, then release before analysis write -// let should_process = { -// let workspace_manager = context.workspace_manager().read().await; -// workspace_manager.is_workspace_file(&uri) -// }; -// if should_process { -// let mut analysis = context.analysis().write().await; -// analysis.update_file(&uri, text); -// } -// ``` -// -// ### ❌ WRONG - ABBA deadlock risk: -// ```rust -// let mut analysis = context.analysis().write().await; // Lock A -// // ... operations ... -// let workspace = context.workspace_manager().write().await; // Lock B (while holding A!) -// // DEADLOCK RISK: Another thread might hold B and wait for A -// ``` -// -// ### ✅ CORRECT - Release before calling async methods: -// ```rust -// let data = { -// let workspace = context.workspace_manager().read().await; -// workspace.get_config().clone() // Clone data -// }; // Lock released -// init_analysis(data).await; // Safe to call async method -// ``` -// -// ### ❌ WRONG - Holding lock while calling async method: -// ```rust -// let workspace = context.workspace_manager().write().await; -// workspace.reload_workspace().await; // May acquire analysis lock internally! -// ``` -// -// ## Atomic Operations (Lock-Free): -// The following atomics can be accessed without lock ordering concerns: -// - `workspace_initialized` (AtomicBool) -// - `workspace_diagnostic_level` (AtomicU8) -// - `workspace_version` (AtomicI64) -// -// ## Notes: -// - Use `drop(lock_guard)` explicitly to release locks early when needed -// - Use scope blocks `{ ... }` to limit lock lifetime -// - When in doubt, release all locks before performing complex operations -// - If you need to modify this ordering, update this documentation AND review all call sites -// ============================================================================ +// LOCK ORDER (acquire low → high; never a lower lock while holding a higher): +// 1. diagnostic_tokens 2. workspace_diagnostic_token 3. cached_file_diagnostics +// 4. update_token 5. analysis(read) 6. workspace_manager(read) +// 7. workspace_manager(write) 8. analysis(write) +// Within `DebouncedAnalysis`: pending_files before reindexing_files, and both +// are released before anything above is taken. +// Leaf: document_versions — a synchronous lock, so it can never be held across +// an `.await`. Never upgrade read→write in place; avoid holding any lock across +// `.await`. Atomics are exempt. + +/// Panics the debounce supervisor will restart before it stops trying. +const DEBOUNCE_RESTART_LIMIT: u32 = 5; + +const DEBOUNCE_RESTART_BACKOFF_BASE: Duration = Duration::from_millis(200); +const DEBOUNCE_RESTART_BACKOFF_MAX: Duration = Duration::from_secs(5); + +fn debounce_restart_backoff(restarts: u32) -> Duration { + DEBOUNCE_RESTART_BACKOFF_BASE + .saturating_mul(1_u32 << restarts.min(16).saturating_sub(1)) + .min(DEBOUNCE_RESTART_BACKOFF_MAX) +} #[derive(Clone)] pub struct RequestTaskMetadata { @@ -123,49 +69,40 @@ struct InFlightRequest { metadata: RequestTaskMetadata, } +// Methods answered with their computed result on cancel instead of an error, +// so the client keeps its current UI state. +// - semantic tokens excluded: relative offsets make a stale set wrong. +// - workspace/diagnostic included: vscode-languageclient permanently stops +// workspace pulls after 6 non-cancellation errors. +// - textDocument/diagnostic excluded: the client rewrites a cancelled pull's +// result to an empty full report; an error reschedules instead. fn keep_stale_editor_data_on_cancel(method: &str) -> bool { - // When these requests are cancelled (typically because a new didChange - // arrived and cancel_all_requests() fired), prefer sending whatever - // result was already computed rather than RequestCanceled. Per the LSP - // spec, "the result even computed on an older state might still be - // useful for the client". Sending RequestCanceled for these methods - // causes brief visual flickering as the client clears its display. - // - // Semantic tokens are deliberately excluded: their result carries no - // version and is encoded as offsets relative to the previous token, so a - // set computed against superseded text does not degrade — every offset - // past the edit lands on the wrong word. They get ContentModified - // instead, via `cancel_error_code`. matches!( method, - "textDocument/codeLens" | "textDocument/inlayHint" | "gluals/annotator" + "textDocument/codeLens" + | "textDocument/inlayHint" + | "gluals/annotator" + | "workspace/diagnostic" ) } -fn cancel_error_code(features: &LspFeatures, method: &str) -> ErrorCode { - // Pull diagnostics are explicitly server-cancellable. LSP 3.17: "A server - // is also allowed to return an error with code `ServerCancelled` - // indicating that the server can't compute the result right now... If no - // data is provided it defaults to `{ retriggerRequest: true }`." That is - // exactly this situation — our own state was invalidated and we want the - // client to ask again — and the default spares us a `data` payload. +/// The error code — and any `data` payload — for a cancelled request. +fn cancel_error(features: &LspFeatures, method: &str) -> (ErrorCode, Option) { + // The client only retriggers when `data` is present; it ignores the + // spec's default-when-absent. if matches!(method, "textDocument/diagnostic" | "workspace/diagnostic") { - return ErrorCode::ServerCancelled; + return ( + ErrorCode::ServerCancelled, + Some(serde_json::json!({ "retriggerRequest": true })), + ); } - // LSP 3.17 implementation considerations: "Use ContentModified only when - // the server's own internal state invalidates an in-flight result." A - // cancelled request is exactly that — `didChange` fired - // `cancel_all_requests_except`, so the text it describes is gone. - // - // Only worth saying to a client that re-sends the request afterwards. - // `retryOnContentModified` is the client's own per-method declaration of - // that; for anything absent from it, ContentModified reads as "no result" - // and clears the feature's UI, so those keep RequestCanceled. + // ContentModified only for methods the client declares it re-sends; + // others read it as "no result" and clear the feature's UI. if features.retries_on_content_modified(method) { - ErrorCode::ContentModified + (ErrorCode::ContentModified, None) } else { - ErrorCode::RequestCanceled + (ErrorCode::RequestCanceled, None) } } @@ -179,9 +116,7 @@ fn should_send_stale_response_on_cancel(method: &str, response: &Response) -> bo } if matches!(method, "textDocument/codeLens" | "textDocument/inlayHint") { - // Returning stale-but-empty results for inlay hints/code lens can clear - // currently rendered UI while typing. Let RequestCanceled keep the - // previous output visible until fresh results are ready. + // A stale-but-empty result would clear rendered UI while typing. return result.as_array().is_some_and(|hints| !hints.is_empty()); } @@ -205,20 +140,25 @@ impl ServerContext { })); let analysis = Arc::new(RwLock::new(EmmyLuaAnalysis::new())); - let status_bar = Arc::new(StatusBar::new(client.clone())); + let lsp_features = Arc::new(LspFeatures::new(client_capabilities)); + let status_bar = Arc::new(StatusBar::new( + client.clone(), + lsp_features.supports_work_done_progress(), + )); let file_diagnostic = Arc::new(FileDiagnostic::new( analysis.clone(), status_bar.clone(), client.clone(), )); - let lsp_features = Arc::new(LspFeatures::new(client_capabilities)); - let workspace_manager = Arc::new(RwLock::new(WorkspaceManager::new( + let workspace_manager_inner = WorkspaceManager::new( analysis.clone(), client.clone(), status_bar.clone(), file_diagnostic.clone(), lsp_features.clone(), - ))); + ); + let workspace_diagnostic_level = workspace_manager_inner.workspace_diagnostic_level_arc(); + let workspace_manager = Arc::new(RwLock::new(workspace_manager_inner)); let debounced_shutdown = CancellationToken::new(); let debounced_analysis = Arc::new(DebouncedAnalysis::new( analysis.clone(), @@ -226,12 +166,50 @@ impl ServerContext { debounced_shutdown.clone(), client.clone(), file_diagnostic.shared_diagnostic_data_cache(), + workspace_diagnostic_level, + lsp_features.clone(), )); - // Spawn the debounced analysis background loop + // Supervise the debounce loop: freshness waiters park on it with no + // deadline, so if it dies the whole server silently goes quiet. { let da = debounced_analysis.clone(); - tokio::spawn(async move { da.run().await }); + let shutdown = debounced_shutdown.clone(); + tokio::spawn(async move { + let mut restarts = 0_u32; + while !shutdown.is_cancelled() { + let task = tokio::spawn({ + let da = da.clone(); + async move { da.run().await } + }); + match task.await { + // `run` only returns on shutdown. + Ok(()) => return, + Err(err) => { + restarts += 1; + if restarts > DEBOUNCE_RESTART_LIMIT { + log::error!( + "LS_DEBOUNCE_LOOP_DEAD debounced analysis loop panicked {} times; giving up, so edits stop being re-indexed and freshness waits park until their request is cancelled: {}", + restarts, + err + ); + return; + } + log::error!( + "LS_DEBOUNCE_LOOP_PANIC debounced analysis loop died, restarting: {}", + err + ); + } + } + + // A panic on entry would otherwise respawn at full CPU, + // and every restart writes a log line. + tokio::select! { + _ = tokio::time::sleep(debounce_restart_backoff(restarts)) => {} + _ = shutdown.cancelled() => return, + } + } + }); } let inner = Arc::new(ServerContextInner { @@ -242,7 +220,7 @@ impl ServerContext { status_bar, lsp_features, debounced_analysis, - document_versions: Arc::new(Mutex::new(HashMap::new())), + document_versions: Arc::new(std::sync::Mutex::new(HashMap::new())), document_version_notify: Arc::new(Notify::new()), }); @@ -276,42 +254,55 @@ impl ServerContext { F: FnOnce(CancellationToken) -> Fut + Send + 'static, Fut: Future> + Send + 'static, { + let sender = self.conn.sender.clone(); let cancel_token = CancellationToken::new(); - let request_method = metadata.method.clone(); - - { - let mut requests = self.requests.lock().await; - requests.insert( - req_id.clone(), - InFlightRequest { - cancel_token: cancel_token.clone(), - metadata, - }, - ); - } + let lsp_features = self.inner.lsp_features.clone(); + let request_method = metadata.method.to_string(); + + let mut requests = self.requests.lock().await; + requests.insert( + req_id.clone(), + InFlightRequest { + metadata, + cancel_token: cancel_token.clone(), + }, + ); + drop(requests); - let sender = self.conn.sender.clone(); let requests = self.requests.clone(); - let lsp_features = self.inner.lsp_features.clone(); tokio::spawn(async move { - let res = exec(cancel_token.clone()).await; + // Own task per handler: a panic must not skip the response or the + // `requests` cleanup below. + let handler_token = cancel_token.clone(); + let res = match tokio::spawn(exec(handler_token)).await { + Ok(res) => res, + Err(err) => { + log::error!( + "LS_REQUEST_PANIC method={} request failed: {}", + request_method, + err + ); + None + } + }; if cancel_token.is_cancelled() { if keep_stale_editor_data_on_cancel(&request_method) && let Some(response) = res && should_send_stale_response_on_cancel(&request_method, &response) { - // Handler completed with a non-null result before/during - // cancellation — send it. Per LSP spec, "the result even - // computed on an older state might still be useful for the - // client." let _ = sender.send(Message::Response(response.clone())); } else { - let response = Response::new_err( - req_id.clone(), - cancel_error_code(&lsp_features, &request_method) as i32, - "cancel".to_string(), - ); + let (code, data) = cancel_error(&lsp_features, &request_method); + let response = Response { + id: req_id.clone(), + result: None, + error: Some(lsp_server::ResponseError { + code: code as i32, + message: "cancel".to_string(), + data, + }), + }; let _ = sender.send(Message::Response(response)); } } else if res.is_none() { @@ -365,15 +356,6 @@ impl ServerContext { } } - pub async fn cancel_requests_by_method(&self, method: &str) { - let requests = self.requests.lock().await; - for request in requests.values() { - if request.metadata.method == method { - request.cancel_token.cancel(); - } - } - } - pub async fn close(&self) { self.debounced_shutdown.cancel(); let mut workspace_manager = self.inner.workspace_manager.write().await; @@ -388,15 +370,23 @@ impl ServerContext { #[cfg(test)] mod tests { use super::{ - LspFeatures, RequestTaskMetadata, ServerContext, cancel_error_code, + DEBOUNCE_RESTART_BACKOFF_MAX, LspFeatures, RequestTaskMetadata, ServerContext, + WorkspaceDiagnosticLevel, cancel_error, debounce_restart_backoff, keep_stale_editor_data_on_cancel, should_send_stale_response_on_cancel, }; use googletest::prelude::*; - use lsp_server::{ErrorCode, RequestId, Response}; + use lsp_server::{Connection, ErrorCode, RequestId, Response}; use lsp_types::ClientCapabilities; use serde_json::json; use std::time::Duration; + #[test] + fn debounce_restart_backoff_doubles_then_saturates() { + assert_eq!(debounce_restart_backoff(1), Duration::from_millis(200)); + assert_eq!(debounce_restart_backoff(3), Duration::from_millis(800)); + assert_eq!(debounce_restart_backoff(60), DEBOUNCE_RESTART_BACKOFF_MAX); + } + #[gtest] fn stale_inlay_and_code_lens_response_requires_non_empty_array() -> Result<()> { let empty = Response::new_ok(1.into(), json!([])); @@ -445,18 +435,19 @@ mod tests { .expect("capabilities should deserialize"), ); verify_that!( - cancel_error_code(&features, "textDocument/semanticTokens/full") as i32, + cancel_error(&features, "textDocument/semanticTokens/full").0 as i32, eq(ErrorCode::ContentModified as i32) )?; verify_that!( - cancel_error_code(&features, "textDocument/inlayHint") as i32, + cancel_error(&features, "textDocument/inlayHint").0 as i32, eq(ErrorCode::RequestCanceled as i32) )?; verify_that!( - cancel_error_code( + cancel_error( &LspFeatures::new(ClientCapabilities::default()), "textDocument/semanticTokens/full" - ) as i32, + ) + .0 as i32, eq(ErrorCode::RequestCanceled as i32) )?; Ok(()) @@ -473,6 +464,77 @@ mod tests { Ok(()) } + #[gtest] + fn a_cancelled_workspace_sweep_restores_the_level_it_claimed() -> Result<()> { + let runtime = tokio::runtime::Runtime::new().expect("tokio runtime should build"); + let (connection, _peer) = Connection::memory(); + + runtime.block_on(async { + let context = ServerContext::new(connection, ClientCapabilities::default()); + let workspace = context.snapshot().workspace_manager_arc(); + let workspace = workspace.read().await; + + workspace.update_workspace_version(WorkspaceDiagnosticLevel::Slow, false); + workspace.update_workspace_version(WorkspaceDiagnosticLevel::Fast, false); + verify_that!( + workspace.claim_workspace_diagnostic_level(), + eq(WorkspaceDiagnosticLevel::Slow) + )?; + + workspace.update_workspace_version(WorkspaceDiagnosticLevel::Slow, false); + + // Claiming empties it, so a second pull finds nothing to do. + verify_that!( + workspace.claim_workspace_diagnostic_level(), + eq(WorkspaceDiagnosticLevel::Slow) + )?; + verify_that!( + workspace.claim_workspace_diagnostic_level(), + eq(WorkspaceDiagnosticLevel::None) + )?; + + // A `Fast` request arriving mid-sweep must not survive as the + // restored value in place of the interrupted `Slow`. + workspace.update_workspace_version(WorkspaceDiagnosticLevel::Fast, false); + workspace.restore_workspace_diagnostic_level(WorkspaceDiagnosticLevel::Slow); + verify_that!( + workspace.claim_workspace_diagnostic_level(), + eq(WorkspaceDiagnosticLevel::Slow) + )?; + + // And restoring never lowers an already-higher pending level. + workspace.update_workspace_version(WorkspaceDiagnosticLevel::Slow, false); + workspace.restore_workspace_diagnostic_level(WorkspaceDiagnosticLevel::Fast); + verify_that!( + workspace.claim_workspace_diagnostic_level(), + eq(WorkspaceDiagnosticLevel::Slow) + )?; + Ok(()) + }) + } + + /// See `keep_stale_editor_data_on_cancel`: cancelled document pulls must + /// answer with an error, workspace pulls with a success. + #[gtest] + fn cancelled_document_diagnostics_answer_with_an_error() -> Result<()> { + verify_that!( + keep_stale_editor_data_on_cancel("textDocument/diagnostic"), + eq(false) + )?; + verify_that!( + keep_stale_editor_data_on_cancel("workspace/diagnostic"), + eq(true) + )?; + + let features = LspFeatures::new(ClientCapabilities::default()); + for method in ["textDocument/diagnostic", "workspace/diagnostic"] { + let (code, data) = cancel_error(&features, method); + verify_that!(code as i32, eq(ErrorCode::ServerCancelled as i32))?; + verify_that!(data, eq(&Some(json!({ "retriggerRequest": true }))))?; + } + Ok(()) + } + #[gtest] fn cancel_all_requests_except_preserves_inlay_and_code_lens_requests() -> Result<()> { let runtime = tokio::runtime::Runtime::new().expect("tokio runtime should build"); @@ -516,7 +578,19 @@ mod tests { ) .await; - let (inlay_token, code_lens_token, hover_token) = { + let diag_id: RequestId = 4.into(); + context + .task( + diag_id.clone(), + RequestTaskMetadata::new("textDocument/diagnostic", None), + |_cancel_token| async move { + tokio::time::sleep(Duration::from_millis(250)).await; + Some(Response::new_ok(diag_id, json!({"kind": "unChanged", "resultId": "abc"}))) + }, + ) + .await; + + let (inlay_token, code_lens_token, hover_token, diag_token) = { let requests = context.requests.lock().await; let inlay = requests .get(&RequestId::from(1)) @@ -533,15 +607,24 @@ mod tests { .expect("hover request should exist") .cancel_token .clone(); - (inlay, code_lens, hover) + let diag = requests + .get(&RequestId::from(4)) + .expect("diagnostic request should exist") + .cancel_token + .clone(); + (inlay, code_lens, hover, diag) }; context - .cancel_all_requests_except(&["textDocument/inlayHint", "textDocument/codeLens"]) + .cancel_all_requests_except(&[ + "textDocument/inlayHint", + "textDocument/codeLens", + ]) .await; verify_that!(inlay_token.is_cancelled(), eq(false))?; verify_that!(code_lens_token.is_cancelled(), eq(false))?; + verify_that!(diag_token.is_cancelled(), eq(true))?; verify_that!(hover_token.is_cancelled(), eq(true))?; Ok(()) }) diff --git a/crates/glua_ls/src/context/snapshot.rs b/crates/glua_ls/src/context/snapshot.rs index ee4e34f85..502eb5a10 100644 --- a/crates/glua_ls/src/context/snapshot.rs +++ b/crates/glua_ls/src/context/snapshot.rs @@ -1,5 +1,8 @@ -use std::{collections::HashMap, sync::Arc}; -use tokio::sync::{Mutex, Notify, RwLock, RwLockReadGuard}; +use std::{ + collections::HashMap, + sync::{Arc, Mutex, MutexGuard}, +}; +use tokio::sync::{Notify, RwLock, RwLockReadGuard}; use tokio_util::sync::CancellationToken; use glua_code_analysis::EmmyLuaAnalysis; @@ -75,8 +78,18 @@ impl ServerContextSnapshot { self.inner.debounced_analysis.clone() } - pub async fn note_document_seen_version(&self, uri: &Uri, version: i32) { - let mut versions = self.inner.document_versions.lock().await; + /// The document version map is a leaf: a synchronous lock, so it can never + /// be held across an `.await` and every caller is free to take it while + /// holding an analysis lock. + fn document_versions(&self) -> MutexGuard<'_, HashMap> { + self.inner + .document_versions + .lock() + .unwrap_or_else(|error| error.into_inner()) + } + + pub fn note_document_seen_version(&self, uri: &Uri, version: i32) { + let mut versions = self.document_versions(); let applied_version = match versions.get(uri).copied() { Some(DocumentVersionState::Open { applied_version, .. @@ -94,15 +107,12 @@ impl ServerContextSnapshot { self.inner.document_version_notify.notify_waiters(); } - pub async fn has_newer_seen_document_version(&self, uri: &Uri, version: i32) -> bool { - is_stale_document_version( - self.inner.document_versions.lock().await.get(uri).copied(), - version, - ) + pub fn has_newer_seen_document_version(&self, uri: &Uri, version: i32) -> bool { + is_stale_document_version(self.document_versions().get(uri).copied(), version) } - pub async fn note_document_applied_version(&self, uri: &Uri, version: i32) { - let mut versions = self.inner.document_versions.lock().await; + pub fn note_document_applied_version(&self, uri: &Uri, version: i32) { + let mut versions = self.document_versions(); let next_state = match versions.get(uri).copied() { Some(DocumentVersionState::Open { seen_version, .. }) => DocumentVersionState::Open { seen_version, @@ -129,7 +139,7 @@ impl ServerContextSnapshot { tokio::pin!(notified); notified.as_mut().enable(); - let is_fresh = match self.inner.document_versions.lock().await.get(uri).copied() { + let is_fresh = match self.document_versions().get(uri).copied() { Some(DocumentVersionState::Open { seen_version, applied_version, @@ -149,18 +159,15 @@ impl ServerContextSnapshot { } } - pub async fn is_document_closed(&self, uri: &Uri) -> bool { + pub fn is_document_closed(&self, uri: &Uri) -> bool { matches!( - self.inner.document_versions.lock().await.get(uri).copied(), + self.document_versions().get(uri).copied(), Some(DocumentVersionState::Closed) ) } - pub async fn mark_document_closed(&self, uri: &Uri) { - self.inner - .document_versions - .lock() - .await + pub fn mark_document_closed(&self, uri: &Uri) { + self.document_versions() .insert(uri.clone(), DocumentVersionState::Closed); self.inner.document_version_notify.notify_waiters(); } @@ -200,6 +207,7 @@ pub struct ServerContextInner { pub status_bar: Arc, pub lsp_features: Arc, pub debounced_analysis: Arc, + /// Leaf lock: see [`ServerContextSnapshot::document_versions`]. pub document_versions: Arc>>, pub document_version_notify: Arc, } @@ -249,8 +257,8 @@ mod tests { let snapshot = context.snapshot(); let uri = Uri::from_str("file:///format.lua").expect("uri should parse"); - snapshot.note_document_seen_version(&uri, 2).await; - snapshot.note_document_applied_version(&uri, 1).await; + snapshot.note_document_seen_version(&uri, 2); + snapshot.note_document_applied_version(&uri, 1); let waiter_snapshot = snapshot.clone(); let waiter_uri = uri.clone(); @@ -266,7 +274,7 @@ mod tests { tokio::time::sleep(Duration::from_millis(10)).await; verify_that!(waiter.is_finished(), eq(false))?; - snapshot.note_document_applied_version(&uri, 2).await; + snapshot.note_document_applied_version(&uri, 2); let completed = tokio::time::timeout(Duration::from_secs(1), waiter) .await diff --git a/crates/glua_ls/src/context/status_bar.rs b/crates/glua_ls/src/context/status_bar.rs index 5a9ca4ae4..2ae2c7128 100644 --- a/crates/glua_ls/src/context/status_bar.rs +++ b/crates/glua_ls/src/context/status_bar.rs @@ -9,8 +9,10 @@ use crate::util::time_cancel_token; use super::ClientProxy; +#[derive(Clone)] pub struct StatusBar { client: Arc, + supports_work_done_progress: bool, } #[derive(Debug, Clone, Copy)] @@ -36,11 +38,19 @@ impl ProgressTask { } impl StatusBar { - pub fn new(client: Arc) -> Self { - Self { client } + pub fn new(client: Arc, supports_work_done_progress: bool) -> Self { + Self { + client, + supports_work_done_progress, + } } pub async fn create_progress_task(&self, task: ProgressTask) { + // create/update/finish all no-op without the client capability. + if !self.supports_work_done_progress { + return; + } + let request_id = self.client.next_id(); let cancel_token = time_cancel_token(std::time::Duration::from_secs(5)); let _ = self @@ -76,6 +86,9 @@ impl StatusBar { percentage: Option, message: Option, ) { + if !self.supports_work_done_progress { + return; + } self.client.send_notification( "$/progress", ProgressParams { @@ -101,6 +114,9 @@ impl StatusBar { } pub fn finish_progress_task(&self, task: ProgressTask, message: Option) { + if !self.supports_work_done_progress { + return; + } self.client.send_notification( "$/progress", ProgressParams { diff --git a/crates/glua_ls/src/context/workspace_manager.rs b/crates/glua_ls/src/context/workspace_manager.rs index d43c71207..4d8cf621d 100644 --- a/crates/glua_ls/src/context/workspace_manager.rs +++ b/crates/glua_ls/src/context/workspace_manager.rs @@ -71,14 +71,31 @@ impl WorkspaceManager { } } - pub fn get_workspace_diagnostic_level(&self) -> WorkspaceDiagnosticLevel { - let value = self.workspace_diagnostic_level.load(Ordering::Acquire); - WorkspaceDiagnosticLevel::from_u8(value) + pub fn workspace_diagnostic_level_arc(&self) -> Arc { + self.workspace_diagnostic_level.clone() } + /// Take the pending diagnostic level and reset it to `None` in one atomic + /// step; a separate load+store pair races with concurrent writers. + pub fn claim_workspace_diagnostic_level(&self) -> WorkspaceDiagnosticLevel { + let previous = self + .workspace_diagnostic_level + .swap(WorkspaceDiagnosticLevel::None.to_u8(), Ordering::AcqRel); + WorkspaceDiagnosticLevel::from_u8(previous) + } + + /// Put a claimed level back after a sweep failed to finish; keeps the + /// higher of it and anything requested since. + pub fn restore_workspace_diagnostic_level(&self, level: WorkspaceDiagnosticLevel) { + self.workspace_diagnostic_level + .fetch_max(level.to_u8(), Ordering::AcqRel); + } + + /// Request at least `level` of workspace diagnostics. A max, not a store: + /// a concurrent request must never downgrade a pending `Slow` sweep. pub fn update_workspace_version(&self, level: WorkspaceDiagnosticLevel, add_version: bool) { self.workspace_diagnostic_level - .store(level.to_u8(), Ordering::Release); + .fetch_max(level.to_u8(), Ordering::AcqRel); if add_version { self.workspace_version.fetch_add(1, Ordering::AcqRel); } @@ -107,6 +124,7 @@ impl WorkspaceManager { let file_diagnostic = self.file_diagnostic.clone(); let lsp_features = self.lsp_features.clone(); let client = self.client.clone(); + let workspace_diagnostic_level = self.workspace_diagnostic_level.clone(); tokio::spawn(async move { cancel_token.wait_for_reindex().await; if cancel_token.is_cancelled() { @@ -140,9 +158,10 @@ impl WorkspaceManager { loaded.workspace_diagnostic_configs, loaded.workspace_emmyrcs, watchdog_status, + workspace_diagnostic_level, ) .await; - if lsp_features.supports_workspace_diagnostic() { + if lsp_features.supports_refresh_diagnostic() { client.refresh_workspace_diagnostics(); } // After completion, remove from HashMap @@ -195,16 +214,17 @@ impl WorkspaceManager { loaded.workspace_diagnostic_configs, loaded.workspace_emmyrcs, watchdog_status, + workspace_diagnostic_status.clone(), ) .await; // Cancel diagnostics and update status without holding analysis lock file_diagnostic.cancel_workspace_diagnostic().await; + // Raise, never lower — a pending `Slow` request must survive. workspace_diagnostic_status - .store(WorkspaceDiagnosticLevel::Fast.to_u8(), Ordering::Release); + .fetch_max(WorkspaceDiagnosticLevel::Fast.to_u8(), Ordering::AcqRel); - // Trigger diagnostics refresh - if lsp_features.supports_workspace_diagnostic() { + if lsp_features.supports_refresh_diagnostic() { client.refresh_workspace_diagnostics(); } else { file_diagnostic @@ -259,12 +279,15 @@ impl WorkspaceManager { // Cancel diagnostics and update status without holding analysis lock file_diagnostic.cancel_workspace_diagnostic().await; workspace_diagnostic_status - .store(WorkspaceDiagnosticLevel::Fast.to_u8(), Ordering::Release); + .fetch_max(WorkspaceDiagnosticLevel::Fast.to_u8(), Ordering::AcqRel); - // Trigger diagnostics refresh - client.refresh_semantic_tokens(); - client.refresh_inlay_hints(); - if lsp_features.supports_workspace_diagnostic() { + if lsp_features.supports_semantic_tokens_refresh() { + client.refresh_semantic_tokens(); + } + if lsp_features.supports_inlay_hint_refresh() { + client.refresh_inlay_hints(); + } + if lsp_features.supports_refresh_diagnostic() { client.refresh_workspace_diagnostics(); } else { file_diagnostic diff --git a/crates/glua_ls/src/handlers/call_hierarchy/build_call_hierarchy.rs b/crates/glua_ls/src/handlers/call_hierarchy/build_call_hierarchy.rs index 330dc6c99..30e817da6 100644 --- a/crates/glua_ls/src/handlers/call_hierarchy/build_call_hierarchy.rs +++ b/crates/glua_ls/src/handlers/call_hierarchy/build_call_hierarchy.rs @@ -204,7 +204,7 @@ fn build_incoming_hierarchy_item( }; let item = CallHierarchyItem { - name: access_path, + name: access_path.to_string(), kind: SymbolKind::FUNCTION, tags: None, detail: None, diff --git a/crates/glua_ls/src/handlers/code_lens/build_code_lens.rs b/crates/glua_ls/src/handlers/code_lens/build_code_lens.rs index 4c0d63c93..0deb29ac1 100644 --- a/crates/glua_ls/src/handlers/code_lens/build_code_lens.rs +++ b/crates/glua_ls/src/handlers/code_lens/build_code_lens.rs @@ -348,7 +348,7 @@ fn add_net_call_code_lens( return Some(()); } let kind_label = match kind { - NetCodeLensCallKind::Define => call_path.clone(), + NetCodeLensCallKind::Define => call_path.to_string(), NetCodeLensCallKind::Start => { resolve_start_kind_label(semantic_model, &call_expr, &call_path, message_arg_idx) } diff --git a/crates/glua_ls/src/handlers/command/commands/emmy_auto_require.rs b/crates/glua_ls/src/handlers/command/commands/emmy_auto_require.rs index 430dd1cfb..9450fb12e 100644 --- a/crates/glua_ls/src/handlers/command/commands/emmy_auto_require.rs +++ b/crates/glua_ls/src/handlers/command/commands/emmy_auto_require.rs @@ -15,6 +15,11 @@ impl CommandSpec for AutoRequireCommand { const COMMAND: &str = "gluals.auto.require"; async fn handle(context: ServerContextSnapshot, args: Vec) -> Option<()> { + if !context.lsp_features().supports_apply_edit() { + log::warn!("auto-require skipped: client does not support workspace/applyEdit"); + return None; + } + let add_to: FileId = serde_json::from_value(args.first()?.clone()).ok()?; let need_require_file_id: FileId = serde_json::from_value(args.get(1)?.clone()).ok()?; let position: Position = serde_json::from_value(args.get(2)?.clone()).ok()?; diff --git a/crates/glua_ls/src/handlers/completion/mod.rs b/crates/glua_ls/src/handlers/completion/mod.rs index 9caedec43..5a66871c2 100644 --- a/crates/glua_ls/src/handlers/completion/mod.rs +++ b/crates/glua_ls/src/handlers/completion/mod.rs @@ -39,25 +39,7 @@ pub async fn on_completion_handler( let uri = params.text_document_position.text_document.uri; let position = params.text_document_position.position; - // For completion, briefly wait for fresh data (up to 50ms) so the user - // sees accurate results. If the reindex takes longer, proceed with - // whatever data is available — a slightly stale completion list is - // better than a multi-second delay. - { - let fresh = tokio::select! { - biased; - _ = cancel_token.cancelled() => return None, - result = context.debounced_analysis().wait_until_fresh_for(&cancel_token, "textDocument/completion") => result, - _ = tokio::time::sleep(std::time::Duration::from_millis(50)) => false, - }; - // If cancelled during wait, bail out - if cancel_token.is_cancelled() { - return None; - } - // `fresh` being false (timeout or cancel) is fine — we proceed - let _ = fresh; - } - + // Freshness is guaranteed by the `wait_for_fresh_index` dispatch arm. let analysis = context.read_analysis(&cancel_token).await?; if cancel_token.is_cancelled() { diff --git a/crates/glua_ls/src/handlers/completion/providers/member_provider.rs b/crates/glua_ls/src/handlers/completion/providers/member_provider.rs index 41b5c18b7..3fda5d6aa 100644 --- a/crates/glua_ls/src/handlers/completion/providers/member_provider.rs +++ b/crates/glua_ls/src/handlers/completion/providers/member_provider.rs @@ -1,6 +1,6 @@ use glua_code_analysis::{ - DbIndex, FileId, GmodRealm, LuaMemberInfo, LuaMemberKey, LuaSemanticDeclId, LuaType, - LuaTypeDeclId, SemanticModel, enum_variable_is_param, get_tpl_ref_extend_type, + DbIndex, FileId, GmodRealm, GmodStateMask, LuaMemberInfo, LuaMemberKey, LuaSemanticDeclId, + LuaType, LuaTypeDeclId, SemanticModel, enum_variable_is_param, get_tpl_ref_extend_type, }; use glua_parser::{ LuaAstNode, LuaAstToken, LuaComment, LuaCommentOwner, LuaDocTag, LuaDocTagRealm, LuaExpr, @@ -108,7 +108,10 @@ fn extend_global_path_members( } } -fn global_expr_access_path(semantic_model: &SemanticModel, expr: &LuaExpr) -> Option { +fn global_expr_access_path( + semantic_model: &SemanticModel, + expr: &LuaExpr, +) -> Option { if !expr_root_is_global(semantic_model, expr) { return None; } @@ -221,7 +224,7 @@ fn gmod_hook_owner_name(prefix_expr: &LuaExpr, prefix_type: &LuaType) -> Option< match prefix_type { LuaType::Ref(owner_type_decl_id) => Some(owner_type_decl_id.get_simple_name().to_string()), _ => match prefix_expr { - LuaExpr::NameExpr(name_expr) => name_expr.get_name_text(), + LuaExpr::NameExpr(name_expr) => name_expr.get_name_text().map(Into::into), _ => None, }, } @@ -275,12 +278,14 @@ fn add_completions_for_members_with_gmod_owner( let mut sorted_entries: Vec<_> = members.iter().collect(); sorted_entries.sort_unstable_by_key(|(name, _)| *name); + let mut realm_filter = RealmFilter::new(builder); for (_, member_infos) in sorted_entries { add_resolve_member_infos( builder, member_infos, completion_status, gmod_fallback_owner, + &mut realm_filter, ); } @@ -292,10 +297,11 @@ fn add_resolve_member_infos( member_infos: &Vec, completion_status: CompletionTriggerStatus, gmod_fallback_owner: Option>, + realm_filter: &mut RealmFilter, ) -> Option<()> { if member_infos.len() == 1 { let member_info = &member_infos[0]; - if !is_member_realm_compatible(builder, member_info) { + if !realm_filter.accepts(builder, member_info) { return Some(()); } let overload_count = match &member_info.typ { @@ -333,7 +339,7 @@ fn add_resolve_member_infos( let resolve_state = get_resolve_state(builder.semantic_model.get_db(), &filtered_member_infos); for member_info in filtered_member_infos { - if !is_member_realm_compatible(builder, member_info) { + if !realm_filter.accepts(builder, member_info) { continue; } @@ -558,28 +564,122 @@ fn is_gmod_hook_member_info(db: &DbIndex, info: &LuaMemberInfo) -> bool { || owner_name.eq_ignore_ascii_case("PLUGIN") } -fn is_member_realm_compatible(builder: &CompletionBuilder, info: &LuaMemberInfo) -> bool { - if !builder.semantic_model.get_emmyrc().gmod.enabled { - return true; +/// Realm filtering state shared by every candidate member of one request. +/// +/// The call-site mask depends only on the request's own position, and a file's +/// `---@realm` annotations are the same for every member declared in it. Both +/// used to be re-derived per member, which meant walking the declaring file's +/// entire syntax tree once per candidate — 98ms of a 200ms completion on a +/// gamemode workspace. The analyzer already indexes those ranges, so prefer its +/// binary search and fall back to one cached walk per file, exactly as the +/// realm-misuse checker does. +struct RealmFilter { + enabled: bool, + call_mask: GmodStateMask, + walked: HashMap>, +} + +impl RealmFilter { + fn new(builder: &CompletionBuilder) -> Self { + let enabled = builder.semantic_model.get_emmyrc().gmod.enabled; + let call_mask = if enabled { + builder + .semantic_model + .get_db() + .get_gmod_infer_index() + .get_state_mask_at_offset( + &builder.semantic_model.get_file_id(), + builder.position_offset, + ) + } else { + GmodStateMask::empty() + }; + Self { + enabled, + call_mask, + walked: HashMap::new(), + } } - let infer_index = builder.semantic_model.get_db().get_gmod_infer_index(); - let call_mask = infer_index.get_state_mask_at_offset( - &builder.semantic_model.get_file_id(), - builder.position_offset, - ); + fn annotation_realm( + &mut self, + semantic_model: &SemanticModel, + file_id: &FileId, + offset: TextSize, + ) -> Option { + let infer_index = semantic_model.get_db().get_gmod_infer_index(); + if infer_index.has_member_realm_ranges(file_id) { + return infer_index.get_member_annotation_realm_at_offset(file_id, offset); + } - let Some(property_owner_id) = &info.property_owner_id else { - return true; - }; - let Some((decl_file_id, decl_offset)) = semantic_decl_position(property_owner_id) else { - return true; - }; + let ranges = match self.walked.get(file_id) { + Some(ranges) => ranges, + None => { + let ranges = collect_decl_annotation_realms(semantic_model, file_id); + self.walked.entry(*file_id).or_insert(ranges) + } + }; + ranges + .iter() + .find(|(range, _)| range.contains(offset)) + .map(|(_, realm)| *realm) + } + + fn accepts(&mut self, builder: &CompletionBuilder, info: &LuaMemberInfo) -> bool { + if !self.enabled { + return true; + } - let decl_mask = resolve_decl_realm(&builder.semantic_model, property_owner_id) - .map(GmodRealm::state_mask) - .unwrap_or_else(|| infer_index.get_state_mask_at_offset(&decl_file_id, decl_offset)); - call_mask.is_compatible_with(decl_mask) + let Some(property_owner_id) = &info.property_owner_id else { + return true; + }; + let Some((decl_file_id, decl_offset)) = semantic_decl_position(property_owner_id) else { + return true; + }; + + let decl_mask = self + .annotation_realm(&builder.semantic_model, &decl_file_id, decl_offset) + .or_else(|| { + resolve_decl_realm_without_annotation(&builder.semantic_model, property_owner_id) + }) + .map(GmodRealm::state_mask) + .unwrap_or_else(|| { + builder + .semantic_model + .get_db() + .get_gmod_infer_index() + .get_state_mask_at_offset(&decl_file_id, decl_offset) + }); + self.call_mask.is_compatible_with(decl_mask) + } +} + +/// Every `---@realm` covered range in a file, in one walk. +fn collect_decl_annotation_realms( + semantic_model: &SemanticModel, + file_id: &FileId, +) -> Vec<(rowan::TextRange, GmodRealm)> { + let Some(tree) = semantic_model.get_db().get_vfs().get_syntax_tree(file_id) else { + return Vec::new(); + }; + let mut ranges = Vec::new(); + for node in tree.get_chunk_node().syntax().descendants() { + if let Some(func_stat) = LuaFuncStat::cast(node.clone()) { + if let Some(comment) = func_stat.get_left_comment() + && let Some(realm) = realm_from_doc_comment(&comment) + { + ranges.push((func_stat.get_range(), realm)); + } + continue; + } + if let Some(local_func_stat) = LuaLocalFuncStat::cast(node) + && let Some(comment) = local_func_stat.get_left_comment() + && let Some(realm) = realm_from_doc_comment(&comment) + { + ranges.push((local_func_stat.get_range(), realm)); + } + } + ranges } fn semantic_decl_position(property_owner_id: &LuaSemanticDeclId) -> Option<(FileId, TextSize)> { @@ -593,17 +693,12 @@ fn semantic_decl_position(property_owner_id: &LuaSemanticDeclId) -> Option<(File } } -fn resolve_decl_realm( +/// The declaration's realm once its `---@realm` annotation has been ruled out. +fn resolve_decl_realm_without_annotation( semantic_model: &SemanticModel, property_owner_id: &LuaSemanticDeclId, ) -> Option { let (decl_file_id, decl_offset) = semantic_decl_position(property_owner_id)?; - if let Some(annotation_realm) = - resolve_decl_annotation_realm_at_offset(semantic_model, &decl_file_id, decl_offset) - { - return Some(annotation_realm); - } - Some( semantic_model .get_db() @@ -612,33 +707,6 @@ fn resolve_decl_realm( ) } -fn resolve_decl_annotation_realm_at_offset( - semantic_model: &SemanticModel, - file_id: &FileId, - offset: TextSize, -) -> Option { - let tree = semantic_model.get_db().get_vfs().get_syntax_tree(file_id)?; - for func_stat in tree.get_chunk_node().descendants::() { - if func_stat.get_range().contains(offset) - && let Some(comment) = func_stat.get_left_comment() - && let Some(realm) = realm_from_doc_comment(&comment) - { - return Some(realm); - } - } - - for local_func_stat in tree.get_chunk_node().descendants::() { - if local_func_stat.get_range().contains(offset) - && let Some(comment) = local_func_stat.get_left_comment() - && let Some(realm) = realm_from_doc_comment(&comment) - { - return Some(realm); - } - } - - None -} - fn realm_from_doc_comment(comment: &LuaComment) -> Option { for tag in comment.get_doc_tags() { if let LuaDocTag::Realm(realm_tag) = tag diff --git a/crates/glua_ls/src/handlers/diagnostic/document_diagnostic.rs b/crates/glua_ls/src/handlers/diagnostic/document_diagnostic.rs index 4188f78af..7b7c6843f 100644 --- a/crates/glua_ls/src/handlers/diagnostic/document_diagnostic.rs +++ b/crates/glua_ls/src/handlers/diagnostic/document_diagnostic.rs @@ -1,7 +1,7 @@ use lsp_types::{ Diagnostic, DocumentDiagnosticParams, DocumentDiagnosticReport, DocumentDiagnosticReportResult, FullDocumentDiagnosticReport, RelatedFullDocumentDiagnosticReport, - RelatedUnchangedDocumentDiagnosticReport, UnchangedDocumentDiagnosticReport, + RelatedUnchangedDocumentDiagnosticReport, UnchangedDocumentDiagnosticReport, Uri, }; use tokio_util::sync::CancellationToken; @@ -27,16 +27,24 @@ fn unchanged_report(result_id: String) -> DocumentDiagnosticReportResult { .into() } -/// Answer without touching what the client already shows. -/// -/// `unchanged` is only legal once the client has an id to compare against, so -/// without one the honest answer is an empty set — which only happens for a -/// document we have no diagnostics for. -fn keep_client_state(previous_result_id: Option) -> DocumentDiagnosticReportResult { - match previous_result_id { - Some(result_id) => unchanged_report(result_id), - None => full_report(None, Vec::new()), +/// Answer without touching what the client already shows: `unchanged` if it +/// has an id, else replay the last report. An empty full report means "clean" +/// to the client and must never stand in for "not ready yet". +async fn keep_client_state( + context: &ServerContextSnapshot, + uri: &Uri, + previous_result_id: Option, +) -> DocumentDiagnosticReportResult { + if let Some(result_id) = previous_result_id { + return unchanged_report(result_id); + } + + if let Some(items) = context.file_diagnostic().cached_file_diagnostics(uri).await { + let result_id = diagnostic_result_id(&items); + return full_report(Some(result_id), items); } + + full_report(None, Vec::new()) } pub async fn on_pull_document_diagnostic( @@ -47,23 +55,14 @@ pub async fn on_pull_document_diagnostic( let uri = params.text_document.uri; let previous_result_id = params.previous_result_id; - // LSP 3.17: "The server must compute document diagnostics against the - // currently synchronized document version." So wait for the reindex - // rather than answering from an older state — a full report replaces - // everything the client shows, so a stale one is a visible repaint, not a - // harmless approximation. - // - // Waiting is safe: until this request resolves the client keeps the - // diagnostics it has and moves their ranges along with the edits itself. + // Correctness, not latency: the index stays stale between didChange and + // the debounced reindex, and diagnostics computed then are wrong. if !context .debounced_analysis() .wait_until_fresh_for(&token, "textDocument/diagnostic") .await { - // Cancelled. The dispatcher turns this into RequestCancelled, which - // the client reschedules without clearing; this value is only a - // fallback if it ever reaches the wire. - return keep_client_state(previous_result_id); + return keep_client_state(&context, &uri, previous_result_id).await; } let Some(diagnostics) = context @@ -71,23 +70,127 @@ pub async fn on_pull_document_diagnostic( .pull_file_diagnostics(uri.clone(), token.clone()) .await else { - return if token.is_cancelled() { - keep_client_state(previous_result_id) + return if token.is_cancelled() || !context.file_diagnostic().is_workspace_loaded() { + keep_client_state(&context, &uri, previous_result_id).await } else { - // The file is not in the index, so it genuinely has no - // diagnostics — reporting `unchanged` here would strand whatever - // the client is still showing for it. + // Not in the index: genuinely no diagnostics. full_report(None, Vec::new()) }; }; - // The push-path cache is deliberately not written here: its only reader is - // gated on `!supports_pull`, so for a pull client this would clone every - // diagnostic on every request for nothing. let result_id = diagnostic_result_id(&diagnostics); if previous_result_id.as_deref() == Some(result_id.as_str()) { return unchanged_report(result_id); } + // Cache for `keep_client_state` replay — but not for a closed document, + // whose final pull would re-insert the entry `didClose` just dropped. + if !context.is_document_closed(&uri) { + context + .file_diagnostic() + .cache_fresh_file_diagnostics(&uri, &diagnostics) + .await; + } + full_report(Some(result_id), diagnostics) } + +#[cfg(test)] +mod tests { + use super::*; + use crate::context::ServerContext; + use googletest::prelude::*; + use lsp_server::Connection; + use lsp_types::{ClientCapabilities, DiagnosticSeverity, Range}; + use std::str::FromStr; + + fn diagnostic(message: &str) -> Diagnostic { + Diagnostic { + message: message.to_string(), + range: Range::default(), + severity: Some(DiagnosticSeverity::WARNING), + ..Default::default() + } + } + + fn as_empty_full_report(result: &DocumentDiagnosticReportResult) -> bool { + let DocumentDiagnosticReportResult::Report(DocumentDiagnosticReport::Full(report)) = result + else { + return false; + }; + report.full_document_diagnostic_report.items.is_empty() + } + + #[gtest] + fn id_less_pull_replays_the_last_report_instead_of_claiming_clean() -> Result<()> { + let runtime = tokio::runtime::Runtime::new().expect("tokio runtime should build"); + let (connection, _peer) = Connection::memory(); + + runtime.block_on(async { + let context = ServerContext::new(connection, ClientCapabilities::default()); + let snapshot = context.snapshot(); + let uri = Uri::from_str("file:///test.lua").unwrap(); + let items = vec![diagnostic("undefined global")]; + + snapshot + .file_diagnostic() + .cache_fresh_file_diagnostics(&uri, &items) + .await; + + let result = keep_client_state(&snapshot, &uri, None).await; + + verify_that!(as_empty_full_report(&result), eq(false))?; + let DocumentDiagnosticReportResult::Report(DocumentDiagnosticReport::Full(report)) = + &result + else { + return fail!("expected a full report replaying the cached diagnostics"); + }; + verify_that!(report.full_document_diagnostic_report.items.len(), eq(1))?; + verify_that!( + report.full_document_diagnostic_report.result_id.is_some(), + eq(true) + )?; + Ok(()) + }) + } + + #[gtest] + fn pull_with_a_result_id_answers_unchanged() -> Result<()> { + let runtime = tokio::runtime::Runtime::new().expect("tokio runtime should build"); + let (connection, _peer) = Connection::memory(); + + runtime.block_on(async { + let context = ServerContext::new(connection, ClientCapabilities::default()); + let snapshot = context.snapshot(); + let uri = Uri::from_str("file:///test.lua").unwrap(); + + let result = keep_client_state(&snapshot, &uri, Some("abc".to_string())).await; + + verify_that!( + matches!( + result, + DocumentDiagnosticReportResult::Report(DocumentDiagnosticReport::Unchanged(_)) + ), + eq(true) + )?; + Ok(()) + }) + } + + #[gtest] + fn unseen_document_may_still_report_empty() -> Result<()> { + let runtime = tokio::runtime::Runtime::new().expect("tokio runtime should build"); + let (connection, _peer) = Connection::memory(); + + runtime.block_on(async { + let context = ServerContext::new(connection, ClientCapabilities::default()); + let snapshot = context.snapshot(); + let uri = Uri::from_str("file:///never-seen.lua").unwrap(); + + let result = keep_client_state(&snapshot, &uri, None).await; + + verify_that!(as_empty_full_report(&result), eq(true))?; + Ok(()) + }) + } +} diff --git a/crates/glua_ls/src/handlers/diagnostic/workspace_diagnostic.rs b/crates/glua_ls/src/handlers/diagnostic/workspace_diagnostic.rs index 431cfd79f..171b6bf8a 100644 --- a/crates/glua_ls/src/handlers/diagnostic/workspace_diagnostic.rs +++ b/crates/glua_ls/src/handlers/diagnostic/workspace_diagnostic.rs @@ -32,13 +32,12 @@ pub async fn on_pull_workspace_diagnostic( let Some(workspace_manager) = context.read_workspace_manager(&token).await else { return WorkspaceDiagnosticReport { items: vec![] }; }; - let status = workspace_manager.get_workspace_diagnostic_level(); + let status = workspace_manager.claim_workspace_diagnostic_level(); if status == WorkspaceDiagnosticLevel::None { return WorkspaceDiagnosticReport { items: vec![] }; } let client_id = workspace_manager.client_config.client_id; let open_files = workspace_manager.current_open_files.clone(); - workspace_manager.update_workspace_version(WorkspaceDiagnosticLevel::None, false); drop(workspace_manager); if client_id.is_vscode() && context.lsp_features().supports_refresh_diagnostic() { @@ -51,16 +50,24 @@ pub async fn on_pull_workspace_diagnostic( WorkspaceDiagnosticLevel::Fast => { context .file_diagnostic() - .pull_workspace_diagnostics_fast(token) + .pull_workspace_diagnostics_fast(token.clone()) .await } WorkspaceDiagnosticLevel::Slow => { context .file_diagnostic() - .pull_workspace_diagnostics_slow(token) + .pull_workspace_diagnostics_slow(token.clone()) .await } }; + + // A cut-short sweep covered only part of the workspace: restore the + // claimed level so the rest is re-swept. + if token.is_cancelled() { + let workspace_manager = context.workspace_manager().read().await; + workspace_manager.restore_workspace_diagnostic_level(status); + } + let analysis = context.analysis().read().await; let vfs = analysis.compilation.get_db().get_vfs(); build_report( diff --git a/crates/glua_ls/src/handlers/document_selection_range/mod.rs b/crates/glua_ls/src/handlers/document_selection_range/mod.rs index c6ae988f1..97774b5c7 100644 --- a/crates/glua_ls/src/handlers/document_selection_range/mod.rs +++ b/crates/glua_ls/src/handlers/document_selection_range/mod.rs @@ -16,6 +16,15 @@ pub async fn on_document_selection_range_handle( cancel_token: CancellationToken, ) -> Option> { let uri = params.text_document.uri; + + // Tree-offset answer: needs the latest document version, not a fresh index. + if !context + .wait_until_latest_document_version_applied(&uri, &cancel_token) + .await + { + return None; + } + let position = params.positions; let analysis = context.read_analysis(&cancel_token).await?; diff --git a/crates/glua_ls/src/handlers/emmy_syntax_tree/mod.rs b/crates/glua_ls/src/handlers/emmy_syntax_tree/mod.rs index 591782441..02e893b9b 100644 --- a/crates/glua_ls/src/handlers/emmy_syntax_tree/mod.rs +++ b/crates/glua_ls/src/handlers/emmy_syntax_tree/mod.rs @@ -20,6 +20,15 @@ pub async fn on_emmy_syntax_tree_handler( cancel_token: CancellationToken, ) -> Option { let uri = Uri::from_str(¶ms.uri).ok()?; + + // Tree-offset answer: needs the latest document version, not a fresh index. + if !context + .wait_until_latest_document_version_applied(&uri, &cancel_token) + .await + { + return None; + } + let analysis = context.read_analysis(&cancel_token).await?; let file_id = analysis.get_file_id(&uri)?; let semantic_model = analysis.compilation.get_semantic_model(file_id)?; diff --git a/crates/glua_ls/src/handlers/fold_range/mod.rs b/crates/glua_ls/src/handlers/fold_range/mod.rs index bb8638c61..077f97b94 100644 --- a/crates/glua_ls/src/handlers/fold_range/mod.rs +++ b/crates/glua_ls/src/handlers/fold_range/mod.rs @@ -33,6 +33,15 @@ pub async fn on_folding_range_handler( return None; } let uri = params.text_document.uri; + + // Tree-offset answer: needs the latest document version, not a fresh index. + if !context + .wait_until_latest_document_version_applied(&uri, &cancel_token) + .await + { + return None; + } + let client_id = context .read_workspace_manager(&cancel_token) .await? diff --git a/crates/glua_ls/src/handlers/initialized/mod.rs b/crates/glua_ls/src/handlers/initialized/mod.rs index e627a923b..871f33b4a 100644 --- a/crates/glua_ls/src/handlers/initialized/mod.rs +++ b/crates/glua_ls/src/handlers/initialized/mod.rs @@ -15,7 +15,7 @@ use crate::{ }, handlers::text_document::register_files_watch, logger::init_logger, - util::{LongRunningWatchdogStatus, spawn_long_running_watchdog}, + util::{AnalysisProgressReporter, LongRunningWatchdogStatus, spawn_long_running_watchdog}, }; pub use client_config::{ClientConfig, get_client_config}; use codestyle::load_editorconfig; @@ -191,6 +191,11 @@ pub async fn initialized_handler( workspace_diagnostic_configs, workspace_emmyrcs, watchdog_status.clone(), + context + .workspace_manager() + .read() + .await + .workspace_diagnostic_level_arc(), ) .await; @@ -211,6 +216,7 @@ pub async fn init_analysis( workspace_diagnostic_configs: HashMap, workspace_emmyrcs: HashMap>, watchdog_status: LongRunningWatchdogStatus, + workspace_diagnostic_level: Arc, ) { if let Ok(emmyrc_json) = serde_json::to_string_pretty(emmyrc.as_ref()) { log::info!("current config : {}", emmyrc_json); @@ -353,7 +359,14 @@ pub async fn init_analysis( watchdog_status.describe(), ); log::info!("analyzing {} Lua files", file_count); + + // `update_files_by_path` blocks for the whole index, so the phases it + // reports are the only progress the client can be given. + let _progress = + AnalysisProgressReporter::install(status_bar.clone(), watchdog_status.clone()); mut_analysis.update_files_by_path(files); + drop(_progress); + watchdog_status.set_progress("Analyzing Lua files", file_count, file_count); status_bar.update_startup_phase( ProgressTask::LoadWorkspace, @@ -432,13 +445,23 @@ pub async fn init_analysis( client.refresh_code_lens(); } - if !lsp_features.supports_workspace_diagnostic() { + if lsp_features.supports_workspace_diagnostic() { + // The pending level is claimed by whichever pull arrives first and + // reset to `None`, so a pull racing startup consumes the workspace's + // one level against a half-built index. Re-arm before asking the + // client to pull again, or the request is a no-op. + workspace_diagnostic_level.fetch_max( + crate::context::WorkspaceDiagnosticLevel::Slow.to_u8(), + std::sync::atomic::Ordering::AcqRel, + ); + if lsp_features.supports_refresh_diagnostic() { + client.refresh_workspace_diagnostics(); + } + } else { log::info!("client does not support workspace diagnostics; scheduling push diagnostics"); file_diagnostic .add_workspace_diagnostic_task(0, false) .await; - } else { - log::info!("client supports workspace diagnostics; waiting for diagnostic pull requests"); } } @@ -560,8 +583,9 @@ mod tests { use googletest::prelude::*; use lsp_server::{Connection, Message}; use lsp_types::{ - ClientCapabilities, CodeLensWorkspaceClientCapabilities, - InlayHintWorkspaceClientCapabilities, SemanticTokensWorkspaceClientCapabilities, + ClientCapabilities, CodeLensWorkspaceClientCapabilities, DiagnosticClientCapabilities, + DiagnosticWorkspaceClientCapabilities, InlayHintWorkspaceClientCapabilities, + SemanticTokensWorkspaceClientCapabilities, TextDocumentClientCapabilities, WorkspaceClientCapabilities, }; use tokio::sync::RwLock; @@ -589,13 +613,23 @@ mod tests { code_lens: Some(CodeLensWorkspaceClientCapabilities { refresh_support: Some(true), }), + diagnostics: Some(DiagnosticWorkspaceClientCapabilities { + refresh_support: Some(true), + }), + ..Default::default() + }), + text_document: Some(TextDocumentClientCapabilities { + diagnostic: Some(DiagnosticClientCapabilities { + dynamic_registration: Some(true), + related_document_support: Some(true), + }), ..Default::default() }), ..Default::default() }; let lsp_features = LspFeatures::new(capabilities); let analysis = Arc::new(RwLock::new(EmmyLuaAnalysis::new())); - let status_bar = Arc::new(StatusBar::new(client.clone())); + let status_bar = Arc::new(StatusBar::new(client.clone(), true)); let file_diagnostic = Arc::new(FileDiagnostic::new( analysis.clone(), status_bar.clone(), @@ -613,10 +647,13 @@ mod tests { HashMap::new(), HashMap::new(), LongRunningWatchdogStatus::new("test"), + Arc::new(std::sync::atomic::AtomicU8::new( + crate::context::WorkspaceDiagnosticLevel::None.to_u8(), + )), )); let mut methods = Vec::new(); - while methods.len() < 3 { + while methods.len() < 4 { let message = peer_connection .receiver .recv_timeout(Duration::from_secs(1)) @@ -633,6 +670,7 @@ mod tests { methods, vec![ "workspace/codeLens/refresh".to_string(), + "workspace/diagnostic/refresh".to_string(), "workspace/inlayHint/refresh".to_string(), "workspace/semanticTokens/refresh".to_string(), ] diff --git a/crates/glua_ls/src/handlers/inlay_hint/build_inlay_hint.rs b/crates/glua_ls/src/handlers/inlay_hint/build_inlay_hint.rs index 5bf99c888..0d75642b1 100644 --- a/crates/glua_ls/src/handlers/inlay_hint/build_inlay_hint.rs +++ b/crates/glua_ls/src/handlers/inlay_hint/build_inlay_hint.rs @@ -352,7 +352,7 @@ fn build_call_args_for_func_type( if let LuaExpr::NameExpr(name_expr) = arg && let Some(param_name) = name_expr.get_name_text() // optimize like rust analyzer - && ¶m_name == name + && param_name == *name { continue; } diff --git a/crates/glua_ls/src/handlers/notification_handler.rs b/crates/glua_ls/src/handlers/notification_handler.rs index 2728d9c19..e37b7a1d8 100644 --- a/crates/glua_ls/src/handlers/notification_handler.rs +++ b/crates/glua_ls/src/handlers/notification_handler.rs @@ -9,7 +9,6 @@ use lsp_types::{ DidChangeWorkspaceFolders, DidCloseTextDocument, DidOpenTextDocument, DidRenameFiles, DidSaveTextDocument, Notification as LspNotification, SetTrace, }, - request::{Request as LspRequest, WorkspaceDiagnosticRequest}, }; use crate::context::{ServerContext, WorkspaceDiagnosticLevel}; @@ -71,17 +70,18 @@ pub async fn on_notification_handler( { let uri = params.text_document.uri.clone(); let snapshot = server_context.snapshot(); - snapshot - .note_document_seen_version(&uri, params.text_document.version) - .await; - if snapshot.lsp_features().supports_workspace_diagnostic() { - let workspace = snapshot.workspace_manager().read().await; - workspace.update_workspace_version(WorkspaceDiagnosticLevel::Fast, true); - } - // Keep stale-aware UI requests alive so they can wait for fresh - // data instead of flickering while typing. + snapshot.note_document_seen_version(&uri, params.text_document.version); + // Exempted requests wait for fresh data instead of being + // cancelled: the client clears a file on a cancelled diagnostic + // pull, and a cancelled executeCommand drops the user's command. server_context - .cancel_all_requests_except(&["textDocument/codeLens", "textDocument/inlayHint"]) + .cancel_all_requests_except(&[ + "textDocument/codeLens", + "textDocument/inlayHint", + "textDocument/diagnostic", + "workspace/diagnostic", + "workspace/executeCommand", + ]) .await; // Mark analysis dirty BEFORE handing the update to the coalescer so // follow-up requests see the stale state immediately. @@ -101,18 +101,15 @@ pub async fn on_notification_handler( { let uri = params.text_document.uri.clone(); let snapshot = server_context.snapshot(); - snapshot - .note_document_seen_version(&uri, params.text_document.version) - .await; + snapshot.note_document_seen_version(&uri, params.text_document.version); { let mut workspace = snapshot.workspace_manager().write().await; workspace.current_open_files.insert(uri.clone()); workspace.update_workspace_version(WorkspaceDiagnosticLevel::Fast, true); } server_context.cancel_text_requests_for_uri(&uri).await; - server_context - .cancel_requests_by_method(WorkspaceDiagnosticRequest::METHOD) - .await; + // The in-flight workspace sweep is deliberately left to finish: + // cancelling restarts it from scratch, which livelocks large scans. let in_flight = snapshot.debounced_analysis_arc().begin_in_flight_change(); let task_snapshot = snapshot.clone(); tokio::spawn(async move { @@ -137,16 +134,15 @@ pub async fn on_notification_handler( { let uri = params.text_document.uri.clone(); let snapshot = server_context.snapshot(); - snapshot.mark_document_closed(&uri).await; + snapshot.mark_document_closed(&uri); { let mut workspace = snapshot.workspace_manager().write().await; workspace.current_open_files.remove(&uri); workspace.update_workspace_version(WorkspaceDiagnosticLevel::Fast, true); } server_context.cancel_text_requests_for_uri(&uri).await; - server_context - .cancel_requests_by_method(WorkspaceDiagnosticRequest::METHOD) - .await; + // The in-flight workspace sweep is deliberately left to finish — + // see the didOpen branch above. let in_flight = snapshot.debounced_analysis_arc().begin_in_flight_change(); let task_snapshot = snapshot.clone(); tokio::spawn(async move { diff --git a/crates/glua_ls/src/handlers/rename/mod.rs b/crates/glua_ls/src/handlers/rename/mod.rs index 828eb4719..a79f905d1 100644 --- a/crates/glua_ls/src/handlers/rename/mod.rs +++ b/crates/glua_ls/src/handlers/rename/mod.rs @@ -43,7 +43,14 @@ pub async fn on_prepare_rename_handler( let uri = params.text_document.uri; let analysis = context.read_analysis(&cancel_token).await?; let file_id = analysis.get_file_id(&uri)?; - let position = params.position; + prepare_rename(&analysis, file_id, params.position) +} + +pub fn prepare_rename( + analysis: &glua_code_analysis::EmmyLuaAnalysis, + file_id: glua_code_analysis::FileId, + position: lsp_types::Position, +) -> Option { let semantic_model = analysis.compilation.get_semantic_model(file_id)?; let root = semantic_model.get_root(); let document = semantic_model.get_document(); @@ -73,6 +80,11 @@ pub async fn on_prepare_rename_handler( token.kind().into(), LuaTokenKind::TkName | LuaTokenKind::TkInt | LuaTokenKind::TkString ) { + // The rename handler refuses these, so offering them would only give + // the user a rename box whose edit never arrives. + if token_is_unrenameable(&semantic_model, &token) { + return None; + } let range = document.to_lsp_range(token.text_range())?; let placeholder = token.text().to_string(); Some(PrepareRenameResponse::RangeWithPlaceholder { range, placeholder }) @@ -115,6 +127,36 @@ pub fn rename( rename_references(&semantic_model, &analysis.compilation, token, new_name) } +fn find_rename_target( + semantic_model: &SemanticModel, + token: &LuaSyntaxToken, +) -> Option { + match get_target_node(token.clone()) { + Some(node) => semantic_model.find_decl(node.into(), SemanticDeclLevel::NoTrace), + None => semantic_model.find_decl(token.clone().into(), SemanticDeclLevel::NoTrace), + } +} + +/// A colon method's `self` is implicit: there is no declaration to carry the +/// new name, so the edit would only break the code. A written `self` — an +/// explicit parameter, or a `local self = self` capture — has one and renames +/// normally. +fn is_unrenameable(semantic_model: &SemanticModel, semantic_decl: &LuaSemanticDeclId) -> bool { + let LuaSemanticDeclId::LuaDecl(decl_id) = semantic_decl else { + return false; + }; + + match semantic_model.get_db().get_decl_index().get_decl(decl_id) { + Some(decl) => decl.is_implicit_self(), + None => true, + } +} + +fn token_is_unrenameable(semantic_model: &SemanticModel, token: &LuaSyntaxToken) -> bool { + find_rename_target(semantic_model, token) + .is_some_and(|semantic_decl| is_unrenameable(semantic_model, &semantic_decl)) +} + #[allow(clippy::mutable_key_type)] fn rename_references( semantic_model: &SemanticModel, @@ -123,10 +165,10 @@ fn rename_references( new_name: String, ) -> Option { let mut result = HashMap::new(); - let semantic_decl = match get_target_node(token.clone()) { - Some(node) => semantic_model.find_decl(node.into(), SemanticDeclLevel::NoTrace), - None => semantic_model.find_decl(token.into(), SemanticDeclLevel::NoTrace), - }?; + let semantic_decl = find_rename_target(semantic_model, &token)?; + if is_unrenameable(semantic_model, &semantic_decl) { + return None; + } match semantic_decl { LuaSemanticDeclId::LuaDecl(decl_id) => { diff --git a/crates/glua_ls/src/handlers/request_handler.rs b/crates/glua_ls/src/handlers/request_handler.rs index 35afd7469..08c1396c7 100644 --- a/crates/glua_ls/src/handlers/request_handler.rs +++ b/crates/glua_ls/src/handlers/request_handler.rs @@ -107,6 +107,8 @@ fn content_modified(id: lsp_server::RequestId) -> Option { macro_rules! dispatch_request { ($request:expr, $context:expr, { $($req_type:ty => $handler:expr),* $(,)? + }, wait_for_fresh_index: { + $($fresh_req_type:ty => $fresh_handler:expr),* $(,)? }, content_modified_if_client_retries: { $($retry_req_type:ty => $retry_handler:expr),* $(,)? }) => { @@ -124,41 +126,120 @@ macro_rules! dispatch_request { } } )* + $( + <$fresh_req_type>::METHOD => { + if let Ok((id, params)) = $request.extract::<<$fresh_req_type as LspRequest>::Params>(<$fresh_req_type>::METHOD) { + let snapshot = $context.snapshot(); + let task_metadata = request_task_metadata(<$fresh_req_type>::METHOD, ¶ms); + let target_uri = task_metadata.uri.clone(); + $context.task(id.clone(), task_metadata, |cancel_token| async move { + // Symbol resolution against a stale index silently + // returns empty; wait for the reindex. A request + // aimed at one file only needs that file's own + // entries to match its text, so it waits for those + // rather than for the edit's whole dependency + // ripple — seconds apart on a large gamemode. + // The handoff is held across the handler so the + // ripple waits for this request to take its read + // lock rather than putting it behind the ripple it + // was just released from. + let (fresh, _handoff) = match target_uri.as_ref() { + Some(uri) => { + let debounced = snapshot.debounced_analysis_arc(); + let handoff = debounced.begin_reader_handoff(); + let fresh = debounced + .wait_until_file_fresh_for( + &cancel_token, + <$fresh_req_type>::METHOD, + uri, + ) + .await; + (fresh, Some(handoff)) + } + None => { + let fresh = snapshot + .debounced_analysis() + .wait_until_fresh_for( + &cancel_token, + <$fresh_req_type>::METHOD, + ) + .await; + (fresh, None) + } + }; + if !fresh { + return None; + } + let result = $fresh_handler(snapshot, params, cancel_token).await; + Some(Response::new_ok(id, result)) + }).await; + return Ok(()); + } + } + )* $( <$retry_req_type>::METHOD => { if let Ok((id, params)) = $request.extract::<<$retry_req_type as LspRequest>::Params>(<$retry_req_type>::METHOD) { let snapshot = $context.snapshot(); let task_metadata = request_task_metadata(<$retry_req_type>::METHOD, ¶ms); + let target_uri = task_metadata.uri.clone(); $context.task(id.clone(), task_metadata, |cancel_token| async move { - // ContentModified is only useful to a client that - // re-sends afterwards. Anyone else reads it as "no - // result" and clears the feature, so they get - // whatever can be computed from the current state. + let debounced = snapshot.debounced_analysis_arc(); + // Aimed at one document, so its own entries are what + // it needs. Waiting on the workspace instead parks it + // behind the whole dependency ripple, and refuses it + // outright while any other file is mid-edit. + let _handoff = target_uri + .as_ref() + .map(|_| debounced.begin_reader_handoff()); + let stale = || async { + match target_uri.as_ref() { + Some(uri) => !debounced.file_is_answerable(uri).await, + None => debounced.is_dirty(), + } + }; + + // A client that doesn't retry ContentModified must + // get a real result: wait for freshness instead. if !snapshot .lsp_features() .retries_on_content_modified(<$retry_req_type>::METHOD) { + let fresh = match target_uri.as_ref() { + Some(uri) => { + debounced + .wait_until_file_fresh_for( + &cancel_token, + <$retry_req_type>::METHOD, + uri, + ) + .await + } + None => { + debounced + .wait_until_fresh_for( + &cancel_token, + <$retry_req_type>::METHOD, + ) + .await + } + }; + if !fresh { + return None; + } let result = $retry_handler(snapshot, params, cancel_token).await; return Some(Response::new_ok(id, result)); } - // A pending reindex means the index still describes - // the previous text, and unresolved symbols are - // silently dropped from the result rather than - // reported. Answering would repaint the file with a - // near-empty result; the refresh after reindex - // drives the corrective re-pull instead. - if snapshot.debounced_analysis().is_dirty() { + if stale().await { return content_modified(id); } let result = $retry_handler(snapshot.clone(), params, cancel_token).await; - // An edit landed while we worked, so this result - // describes neither the text the client asked - // about nor the text it now holds. - if snapshot.debounced_analysis().is_dirty() { + // An edit landed while we worked. + if stale().await { return content_modified(id); } @@ -186,47 +267,53 @@ pub async fn on_request_handler( server_context: &mut ServerContext, ) -> Result<(), Box> { dispatch_request!(req, server_context, { - HoverRequest => on_hover, - DocumentSymbolRequest => on_document_symbol, + // Must not resolve declarations/members/globals through the index — + // those need the `wait_for_fresh_index` arm. FoldingRangeRequest => on_folding_range_handler, - DocumentColor => on_document_color, - ColorPresentationRequest => on_document_color_presentation, - DocumentLinkRequest => on_document_link_handler, - DocumentLinkResolve => on_document_link_resolve_handler, - EmmyGutterRequest => on_emmy_gutter_handler, - EmmyGutterDetailRequest => on_emmy_gutter_detail_handler, EmmySyntaxTreeRequest => on_emmy_syntax_tree_handler, - EmmyAnnotatorRequest => on_emmy_annotator_handler, SelectionRangeRequest => on_document_selection_range_handle, + Formatting => on_formatting_handler, + RangeFormatting => on_range_formatting_handler, + OnTypeFormatting => on_type_formatting_handler, + + // Reads the index but performs its own wait to control the cancel + // response. + EmmyAnnotatorRequest => on_emmy_annotator_handler, + CodeLensRequest => on_code_lens_handler, + InlayHintRequest => on_inlay_hint_handler, + DocumentDiagnosticRequest => on_pull_document_diagnostic, + WorkspaceDiagnosticRequest => on_pull_workspace_diagnostic, + }, wait_for_fresh_index: { Completion => on_completion_handler, ResolveCompletionItem => on_completion_resolve_handler, - InlayHintResolveRequest => on_resolve_inlay_hint, - CodeLensRequest => on_code_lens_handler, + HoverRequest => on_hover, + GluaHoverExpandRequest => on_hover_expand_handler, GotoDefinition => on_goto_definition_handler, GotoImplementation => on_implementation_handler, References => on_references_handler, Rename => on_rename_handler, PrepareRenameRequest => on_prepare_rename_handler, - CodeLensResolve => on_resolve_code_lens_handler, SignatureHelpRequest => on_signature_helper_handler, DocumentHighlightRequest => on_document_highlight_handler, - ExecuteCommand => on_execute_command_handler, + DocumentSymbolRequest => on_document_symbol, + WorkspaceSymbolRequest => on_workspace_symbol_handler, CodeActionRequest => on_code_action_handler, InlineValueRequest => on_inline_values_handler, - WorkspaceSymbolRequest => on_workspace_symbol_handler, - GluaDocSearchRequest => on_doc_search_handler, - GluaHoverExpandRequest => on_hover_expand_handler, - GmodScriptedClassesRequest => on_gmod_scripted_classes_handler, - GmodScriptedClassesV2Request => on_gmod_scripted_classes_v2_handler, - InlayHintRequest => on_inlay_hint_handler, - Formatting => on_formatting_handler, - RangeFormatting => on_range_formatting_handler, - OnTypeFormatting => on_type_formatting_handler, + DocumentColor => on_document_color, + ColorPresentationRequest => on_document_color_presentation, + DocumentLinkRequest => on_document_link_handler, + DocumentLinkResolve => on_document_link_resolve_handler, + CodeLensResolve => on_resolve_code_lens_handler, + InlayHintResolveRequest => on_resolve_inlay_hint, + EmmyGutterRequest => on_emmy_gutter_handler, + EmmyGutterDetailRequest => on_emmy_gutter_detail_handler, CallHierarchyPrepare => on_prepare_call_hierarchy_handler, CallHierarchyIncomingCalls => on_incoming_calls_handler, CallHierarchyOutgoingCalls => on_outgoing_calls_handler, - DocumentDiagnosticRequest => on_pull_document_diagnostic, - WorkspaceDiagnosticRequest => on_pull_workspace_diagnostic, + GluaDocSearchRequest => on_doc_search_handler, + GmodScriptedClassesRequest => on_gmod_scripted_classes_handler, + GmodScriptedClassesV2Request => on_gmod_scripted_classes_v2_handler, + ExecuteCommand => on_execute_command_handler, }, content_modified_if_client_retries: { SemanticTokensFullRequest => on_semantic_token_handler, }); @@ -238,6 +325,7 @@ pub async fn on_request_handler( mod tests { use super::extract_uri_from_value; use glua_code_analysis::LuaDeclId; + use googletest::prelude::*; use rowan::TextSize; use serde_json::json; use std::str::FromStr; @@ -249,6 +337,62 @@ mod tests { completion::{CompletionData, CompletionDataType}, }; + #[gtest] + fn fresh_index_requests_do_not_answer_until_analysis_settles() -> Result<()> { + use super::{Completion, LspRequest, on_request_handler}; + use crate::context::ServerContext; + use lsp_server::{Connection, Message}; + use lsp_types::ClientCapabilities; + use std::time::Duration; + + let runtime = tokio::runtime::Runtime::new().expect("tokio runtime should build"); + let (server_connection, peer) = Connection::memory(); + + runtime.block_on(async { + let mut context = ServerContext::new(server_connection, ClientCapabilities::default()); + let snapshot = context.snapshot(); + let debounced_analysis = snapshot.debounced_analysis_arc(); + + // Mark analysis dirty exactly as a didChange does, before the + // request arrives. + let in_flight = debounced_analysis.begin_in_flight_change(); + + let request = lsp_server::Request::new( + 1.into(), + Completion::METHOD.to_string(), + json!({ + "textDocument": { "uri": "file:///test.lua" }, + "position": { "line": 0, "character": 0 } + }), + ); + on_request_handler(request, &mut context) + .await + .expect("dispatch should succeed"); + + // Wait for the condition, not for a deadline: once the handler is + // inside the freshness wait it cannot leave while the change is + // in flight, so an empty channel here is not a race. + tokio::time::timeout(Duration::from_secs(5), async { + while debounced_analysis.freshness_wait_count() == 0 { + tokio::task::yield_now().await; + } + }) + .await + .expect("the handler should reach the freshness wait"); + verify_that!(peer.receiver.try_recv().is_err(), eq(true))?; + + // Settling the change releases the wait. + in_flight.finish().await; + + let message = peer + .receiver + .recv_timeout(Duration::from_secs(5)) + .expect("a response must arrive once analysis is fresh"); + verify_that!(matches!(message, Message::Response(_)), eq(true))?; + Ok(()) + }) + } + #[test] fn extracts_text_document_uri() { let uri = Uri::from_str("file:///document.lua").expect("uri should parse"); diff --git a/crates/glua_ls/src/handlers/semantic_token/build_semantic_tokens.rs b/crates/glua_ls/src/handlers/semantic_token/build_semantic_tokens.rs index c07b0898e..7df1222e1 100644 --- a/crates/glua_ls/src/handlers/semantic_token/build_semantic_tokens.rs +++ b/crates/glua_ls/src/handlers/semantic_token/build_semantic_tokens.rs @@ -268,6 +268,8 @@ fn build_tokens_semantic_token( | LuaTokenKind::TkTagUsing | LuaTokenKind::TkTagSource | LuaTokenKind::TkTagRealm + | LuaTokenKind::TkTagFileparam + | LuaTokenKind::TkTagOutparam | LuaTokenKind::TkTagReturnCast | LuaTokenKind::TkTagExport | LuaTokenKind::TkLanguage @@ -496,6 +498,18 @@ fn build_node_semantic_token( ); } } + LuaAst::LuaDocTagOutparam(doc_outparam) => { + if let Some(path) = doc_outparam.get_path_token() { + builder.push_with_modifiers( + path.syntax(), + SemanticTokenType::PARAMETER, + &[ + SemanticTokenModifier::DECLARATION, + SemanticTokenModifier::DOCUMENTATION, + ], + ); + } + } LuaAst::LuaDocTagFileparam(doc_fileparam) => { if let Some(name) = doc_fileparam.get_name_token() { builder.push_with_modifiers( @@ -2117,7 +2131,7 @@ fn inferred_alias_target_token_type( } } -fn expr_access_path(value_expr: &LuaExpr) -> Option { +fn expr_access_path(value_expr: &LuaExpr) -> Option { match value_expr { LuaExpr::NameExpr(name_expr) => name_expr.get_access_path(), LuaExpr::IndexExpr(index_expr) => index_expr.get_access_path(), diff --git a/crates/glua_ls/src/handlers/semantic_token/mod.rs b/crates/glua_ls/src/handlers/semantic_token/mod.rs index 92feb757b..64a39ad0b 100644 --- a/crates/glua_ls/src/handlers/semantic_token/mod.rs +++ b/crates/glua_ls/src/handlers/semantic_token/mod.rs @@ -155,7 +155,7 @@ mod tests { // Seen but not yet applied: exactly the window between a didChange // notification and the coalescer applying its preparsed tree. - snapshot.note_document_seen_version(&uri, 2).await; + snapshot.note_document_seen_version(&uri, 2); let handler_snapshot = snapshot.clone(); let params = SemanticTokensParams { @@ -170,7 +170,7 @@ mod tests { tokio::time::sleep(Duration::from_millis(10)).await; verify_that!(handler.is_finished(), eq(false))?; - snapshot.note_document_applied_version(&uri, 2).await; + snapshot.note_document_applied_version(&uri, 2); tokio::time::timeout(Duration::from_secs(1), handler) .await diff --git a/crates/glua_ls/src/handlers/test/hover_test.rs b/crates/glua_ls/src/handlers/test/hover_test.rs index 927c6cb7c..b2e769209 100644 --- a/crates/glua_ls/src/handlers/test/hover_test.rs +++ b/crates/glua_ls/src/handlers/test/hover_test.rs @@ -4404,4 +4404,182 @@ local EscapeStringMap: { Ok(()) } + + /// `self` inside a scripted-class method is an instance of the class the + /// authoring table stands for, so hover must name that class. + /// + /// `ENT` is virtual inside a scripted scope, so it has no decl and the + /// receiver lookup used to fall back to the enclosing method's member. That + /// is the shared semantic decl id, so goto-definition, references, + /// implementation and rename all pointed at the method too; hover is just + /// the cheapest place to observe it. + #[gtest] + fn test_hover_self_in_scripted_entity_method_shows_class() -> Result<()> { + let mut ws = enable_gmod_workspace(); + let mut emmyrc = ws.get_emmyrc(); + emmyrc + .gmod + .scripted_class_scopes + .set_include(vec![legacy_scope("entities/**")]); + ws.update_emmyrc(emmyrc); + + let source = r#" + --- Implement this base class function. + function ENT:OnSeatInput(seatIndex, action, pressed) + local t = SELF_ANCHOR + return t + end + "#; + + let (self_content, self_position) = ProviderVirtualWorkspace::handle_file_content( + &source.replace("SELF_ANCHOR", "self"), + )?; + let self_file = ws.def_file("lua/entities/base_glide_car/init.lua", &self_content); + let self_hover = extract_hover_markdown(&ws, self_file, self_position); + + let (ent_content, ent_position) = ProviderVirtualWorkspace::handle_file_content( + &source + .replace("SELF_ANCHOR", "self") + .replace("function ENT:OnSeatInput", "function ENT:OnSeatInput"), + )?; + let ent_file = ws.def_file("lua/entities/base_glide_boat/init.lua", &ent_content); + let ent_hover = extract_hover_markdown(&ws, ent_file, ent_position); + + assert!( + self_hover.contains("self: base_glide_car") && !self_hover.contains("(method)"), + "hovering `self` must name the class it is an instance of, not the \ + enclosing method, got: {self_hover}" + ); + assert!( + ent_hover.contains("base_glide_boat") && !ent_hover.contains("(method)"), + "hovering the authoring table must name the class, got: {ent_hover}" + ); + + // Goto-definition shares the same semantic decl and used to land on the + // enclosing method's name token, so pin that it no longer does. + let self_def = + crate::handlers::definition::definition(&ws.analysis, self_file, self_position) + .expect("goto-definition on `self`"); + let lsp_types::GotoDefinitionResponse::Scalar(location) = self_def else { + panic!("expected a single goto-definition location, got: {self_def:?}"); + }; + let method_name_start = self_content + .lines() + .nth(location.range.start.line as usize) + .and_then(|line| line.find("OnSeatInput")) + .expect("the method name is on the target line"); + assert!( + (location.range.start.character as usize) < method_name_start, + "goto-definition on `self` must not land on the method name, got: {location:?}" + ); + Ok(()) + } + + /// `PLAYER` and `PLUGIN` author their class through a real local, unlike the + /// virtual `ENT` / `SWEP` / `GM` tables. `self` must still be typed as that + /// file's scripted class in both styles — the local is the authoring + /// mechanism, not a different entity. + #[gtest] + fn test_self_in_player_class_resolves_to_class_like_the_token() -> Result<()> { + let source = r#" + local PLAYER = {} + + function PLAYER:Loadout() + local t = ANCHOR + return t + end + "#; + + let mut ws = enable_gmod_workspace(); + let (self_content, self_position) = + ProviderVirtualWorkspace::handle_file_content(&source.replace("ANCHOR", "self"))?; + let self_file = ws.def_file("lua/gamemode/player_class/player_x.lua", &self_content); + let self_hover = extract_hover_markdown(&ws, self_file, self_position); + + let (token_content, token_position) = ProviderVirtualWorkspace::handle_file_content( + &source + .replace("ANCHOR", "self") + .replace("function PLAYER:Loadout", "function PLAYER:Loadout"), + )?; + let token_file = ws.def_file("lua/gamemode/player_class/player_y.lua", &token_content); + let token_hover = extract_hover_markdown(&ws, token_file, token_position); + + assert!( + self_hover.contains("self: player_x") && !self_hover.contains("(method)"), + "hovering `self` in a player class must name the class, got: {self_hover}" + ); + assert!( + token_hover.contains("player_y"), + "hovering the authoring token must name the class, got: {token_hover}" + ); + Ok(()) + } + + /// A scripted class inherits both the GMod `ENT` annotation chain and its + /// own scripted base, so the super graph is a diamond: `ENT : ENTITY : + /// Entity` sits alongside a direct `Entity` super, and the real base comes + /// last. Walking the siblings with one shared infer guard marked `Entity` + /// visited in the `ENT` branch, so the direct `Entity` sibling failed with + /// `RecursiveInfer` and aborted the rest of the walk — the real base was + /// never searched and the inherited method lost its signature. + #[gtest] + fn test_inherited_scripted_method_survives_super_diamond() -> Result<()> { + let mut ws = enable_gmod_workspace(); + let mut emmyrc = ws.get_emmyrc(); + emmyrc + .gmod + .scripted_class_scopes + .set_include(vec![legacy_scope("entities/**")]); + ws.update_emmyrc(emmyrc); + + ws.def_files(vec![ + ( + "lua/annotations/ent.lua", + r#" +---@class Entity +local Entity = {} +---@class ENTITY : Entity +ENTITY = Entity +---@class ENT : ENTITY +ENT = {} +"#, + ), + ( + "lua/entities/base_glide/shared.lua", + "ENT.Type = \"anim\"\nENT.Base = \"base_anim\"\n", + ), + ( + "lua/entities/base_glide/sv_input.lua", + r#" +--- Get the action's boolean value from a specific seat. +---@param seatIndex number The seat index +function ENT:GetInputBool(seatIndex) + return false +end +"#, + ), + ( + "lua/entities/base_glide_car/shared.lua", + "ENT.Type = \"anim\"\nENT.Base = \"base_glide\"\n", + ), + ]); + + let (content, position) = ProviderVirtualWorkspace::handle_file_content( + r#" +function ENT:OnSeatInput() + local held = self:GetInputBool(1) + return held +end +"#, + )?; + let file = ws.def_file("lua/entities/base_glide_car/init.lua", &content); + let hover = extract_hover_markdown(&ws, file, position); + + assert!( + hover.contains("(method) base_glide:GetInputBool"), + "an inherited scripted-class method must keep its signature when the \ + super graph forms a diamond, got: {hover}" + ); + Ok(()) + } } diff --git a/crates/glua_ls/src/handlers/test/rename_test.rs b/crates/glua_ls/src/handlers/test/rename_test.rs index 2c78cf11a..d6b36f0af 100644 --- a/crates/glua_ls/src/handlers/test/rename_test.rs +++ b/crates/glua_ls/src/handlers/test/rename_test.rs @@ -1,9 +1,59 @@ #[cfg(test)] mod tests { + use crate::handlers::rename::{prepare_rename, rename}; use crate::handlers::test_lib::{ProviderVirtualWorkspace, check}; use googletest::prelude::*; use lsp_types::{Position, Range, TextEdit}; + /// A written `self` has a declaration to carry the new name; only a colon + /// method's implicit receiver does not. + #[gtest] + fn test_rename_self_only_refused_for_the_implicit_receiver() -> Result<()> { + let mut ws = ProviderVirtualWorkspace::new(); + check!(ws.check_rename( + r#" + local self = 1 + print(self) + "#, + "captured".to_string(), + vec![( + "virtual_0.lua".to_string(), + vec![ + TextEdit { + range: Range::new(Position::new(1, 22), Position::new(1, 26)), + new_text: "captured".to_string(), + }, + TextEdit { + range: Range::new(Position::new(2, 22), Position::new(2, 26)), + new_text: "captured".to_string(), + }, + ], + )] + )); + + let mut ws = ProviderVirtualWorkspace::new(); + let (content, position) = ProviderVirtualWorkspace::handle_file_content( + r#" + local Class = {} + function Class:Method() + return self + end + "#, + )?; + let file_id = ws.def(&content); + verify_that!( + rename(&ws.analysis, file_id, position, "renamed".to_string()).is_none(), + eq(true) + )?; + // prepareRename must agree, or the client opens a rename box for an + // edit that never arrives. + verify_that!( + prepare_rename(&ws.analysis, file_id, position).is_none(), + eq(true) + )?; + Ok(()) + } + #[gtest] fn test_int_key() -> Result<()> { let mut ws = ProviderVirtualWorkspace::new(); diff --git a/crates/glua_ls/src/handlers/test/semantic_token_test.rs b/crates/glua_ls/src/handlers/test/semantic_token_test.rs index 7c5f8e999..caad10a10 100644 --- a/crates/glua_ls/src/handlers/test/semantic_token_test.rs +++ b/crates/glua_ls/src/handlers/test/semantic_token_test.rs @@ -30,12 +30,9 @@ mod tests { let mut result = Vec::new(); let mut line = 0; let mut col = 0; - for chunk in data.chunks_exact(5) { - let delta_line = chunk[0]; - let delta_start = chunk[1]; - let length = chunk[2]; - let token_type = chunk[3]; - let token_modifiers = chunk[4]; + let (chunks, _) = data.as_chunks::<5>(); + for chunk in chunks { + let [delta_line, delta_start, length, token_type, token_modifiers] = *chunk; if delta_line > 0 { line += delta_line; @@ -163,6 +160,50 @@ local x = 1 Ok(()) } + #[gtest] + fn test_doc_tag_outparam_highlights_tag_and_path() -> Result<()> { + let mut ws = ProviderVirtualWorkspace::new(); + let main = ws.def_file( + "main.lua", + r#"---@outparam config.output string +---@param config table +function fill(config) end +"#, + ); + + let data = ws.get_semantic_token_data_for_file(main)?; + let tokens = decode(&data); + let doc_modifiers = &[ + SemanticTokenModifier::DECLARATION, + SemanticTokenModifier::DOCUMENTATION, + ]; + + verify_that!( + has_token( + &tokens, + 0, + 4, + 8, + SemanticTokenType::KEYWORD, + &[SemanticTokenModifier::DOCUMENTATION] + ), + eq(true) + )?; + verify_that!( + has_token( + &tokens, + 0, + 13, + 13, + SemanticTokenType::PARAMETER, + doc_modifiers + ), + eq(true) + )?; + + Ok(()) + } + #[gtest] fn test_string_literal_segments_use_utf16_lengths() -> Result<()> { let mut ws = ProviderVirtualWorkspace::new(); diff --git a/crates/glua_ls/src/handlers/text_document/text_document_handler.rs b/crates/glua_ls/src/handlers/text_document/text_document_handler.rs index d90b750b1..c0a1e7b55 100644 --- a/crates/glua_ls/src/handlers/text_document/text_document_handler.rs +++ b/crates/glua_ls/src/handlers/text_document/text_document_handler.rs @@ -23,12 +23,12 @@ fn spawn_deferred_drop(deferred_drop: DeferredVfsDrop) { tokio::task::spawn_blocking(move || drop(deferred_drop)); } -async fn should_drop_stale_version( +fn should_drop_stale_version( context: &ServerContextSnapshot, uri: &lsp_types::Uri, version: i32, ) -> bool { - context.has_newer_seen_document_version(uri, version).await + context.has_newer_seen_document_version(uri, version) } async fn apply_document_update_without_queuing( @@ -39,67 +39,62 @@ async fn apply_document_update_without_queuing( mut preparsed: Option, trigger_reindex: bool, ) -> Option { - let mut pending_text = Some(text); - let mut retries = 0u32; - - loop { - if should_drop_stale_version(context, uri, version).await { - return None; - } - - if let Ok(mut analysis) = context.analysis().try_write() { - let text = pending_text - .take() - .expect("document text should still be available"); - let (file_id, deferred_drop) = if let Some(preparsed) = preparsed.take() { - if trigger_reindex { - ( - analysis.update_file_preparsed( - uri.clone(), - Some(text), - preparsed.tree, - preparsed.line_index, - Some(version), - true, - ), - None, - ) - } else { - let (file_id, deferred_drop) = analysis.update_file_preparsed_deferred( - uri.clone(), - Some(text), - preparsed.tree, - preparsed.line_index, - Some(version), - )?; - (Some(file_id), Some(deferred_drop)) - } - } else if trigger_reindex { - (analysis.update_file_by_uri(uri, Some(text)), None) - } else { - (analysis.update_file_text_only(uri, text), None) - }; - if file_id.is_some() { - context - .file_diagnostic() - .invalidate_shared_diagnostic_data(); - } - drop(analysis); + if should_drop_stale_version(context, uri, version) { + return None; + } - if let Some(deferred_drop) = deferred_drop { - spawn_deferred_drop(deferred_drop); - } + // Fair-queued `write().await`, not a `try_write` spin, which can starve + // for seconds under a stream of readers. + let mut analysis = context.analysis().write().await; - return file_id; - } + // The lock wait is unbounded, so re-check staleness now that we hold it. + if should_drop_stale_version(context, uri, version) { + return None; + } - retries += 1; - if retries <= 20 { - tokio::task::yield_now().await; + let (file_id, deferred_drop) = if let Some(preparsed) = preparsed.take() { + if trigger_reindex { + ( + analysis.update_file_preparsed( + uri.clone(), + Some(text), + preparsed.tree, + preparsed.line_index, + Some(version), + true, + ), + None, + ) } else { - tokio::time::sleep(Duration::from_millis(2)).await; + let (file_id, deferred_drop) = analysis.update_file_preparsed_deferred( + uri.clone(), + Some(text), + preparsed.tree, + preparsed.line_index, + Some(version), + )?; + (Some(file_id), Some(deferred_drop)) } + } else if trigger_reindex { + (analysis.update_file_by_uri(uri, Some(text)), None) + } else { + (analysis.update_file_text_only(uri, text), None) + }; + + // Text-only updates leave the index alone; the debounced reindex + // invalidates under its own write lock. + if file_id.is_some() && trigger_reindex { + context + .file_diagnostic() + .invalidate_shared_diagnostic_data(); + } + drop(analysis); + + if let Some(deferred_drop) = deferred_drop { + spawn_deferred_drop(deferred_drop); } + + file_id } async fn check_schema_update(context: &ServerContextSnapshot) { @@ -236,11 +231,11 @@ pub async fn on_did_open_text_document( }; if !should_process { - context.mark_document_closed(&uri).await; + context.mark_document_closed(&uri); return None; } - if should_drop_stale_version(&context, &uri, version).await { + if should_drop_stale_version(&context, &uri, version) { return Some(()); } @@ -250,7 +245,7 @@ pub async fn on_did_open_text_document( }; let interval = emmyrc.diagnostics.diagnostic_interval.unwrap_or(500); let preparsed = preparse_document(text.clone(), emmyrc).await; - if should_drop_stale_version(&context, &uri, version).await { + if should_drop_stale_version(&context, &uri, version) { return Some(()); } @@ -261,7 +256,7 @@ pub async fn on_did_open_text_document( let file_id = apply_document_update_without_queuing(&context, &uri, text, version, preparsed, true).await; if file_id.is_some() { - context.note_document_applied_version(&uri, version).await; + context.note_document_applied_version(&uri, version); if context.lsp_features().supports_semantic_tokens_refresh() { context.client().refresh_semantic_tokens(); } @@ -362,11 +357,11 @@ pub async fn on_did_change_text_document( } if !should_process { - context.mark_document_closed(&uri).await; + context.mark_document_closed(&uri); return None; } - if should_drop_stale_version(&context, &uri, version).await { + if should_drop_stale_version(&context, &uri, version) { return Some(()); } @@ -375,7 +370,7 @@ pub async fn on_did_change_text_document( let syntax_diagnostics = preparsed .as_ref() .map_or_else(Vec::new, |parsed| parsed.syntax_diagnostics.clone()); - if should_drop_stale_version(&context, &uri, version).await { + if should_drop_stale_version(&context, &uri, version) { return Some(()); } @@ -383,10 +378,10 @@ pub async fn on_did_change_text_document( apply_document_update_without_queuing(&context, &uri, text, version, preparsed, false) .await; if file_id.is_some() { - context.note_document_applied_version(&uri, version).await; + context.note_document_applied_version(&uri, version); } - if should_drop_stale_version(&context, &uri, version).await { + if should_drop_stale_version(&context, &uri, version) { return Some(()); } @@ -407,7 +402,10 @@ pub async fn on_did_change_text_document( // Schedule debounced reindex — rapid edits into a single reindex if let Some(file_id) = file_id { - context.debounced_analysis().schedule(file_id).await; + context + .debounced_analysis() + .schedule(file_id, uri.clone()) + .await; } // Handle reindex without holding locks @@ -435,6 +433,15 @@ pub async fn on_did_close_document( ) -> Option<()> { let uri = ¶ms.text_document.uri; let lsp_features = context.lsp_features(); + + // A closed document has no reader for its cached replay report. + if lsp_features.supports_pull_diagnostic() { + context + .file_diagnostic() + .forget_cached_file_diagnostics(uri) + .await; + } + let (encoding, interval) = { let analysis = context.analysis().read().await; let emmyrc = analysis.get_emmyrc(); @@ -452,13 +459,13 @@ pub async fn on_did_close_document( if let Some(file_path) = uri_to_file_path(uri) { if file_path.exists() { if let Some(text) = read_file_with_encoding(&file_path, &encoding) { - if !context.is_document_closed(uri).await { + if !context.is_document_closed(uri) { return Some(()); } let file_id = { let mut analysis = context.analysis().write().await; - if !context.is_document_closed(uri).await { + if !context.is_document_closed(uri) { return Some(()); } let file_id = analysis.update_file_by_uri(uri, Some(text)); @@ -473,7 +480,7 @@ pub async fn on_did_close_document( if !lsp_features.supports_pull_diagnostic() && let Some(file_id) = file_id { - if !context.is_document_closed(uri).await { + if !context.is_document_closed(uri) { return Some(()); } context @@ -487,11 +494,11 @@ pub async fn on_did_close_document( } } } else { - if !context.is_document_closed(uri).await { + if !context.is_document_closed(uri) { return Some(()); } let mut mut_analysis = context.analysis().write().await; - if !context.is_document_closed(uri).await { + if !context.is_document_closed(uri) { return Some(()); } mut_analysis.remove_file_by_uri(uri); @@ -611,7 +618,7 @@ mod tests { .update_file_by_uri(&uri, Some("local x = 1".to_string())); // Mark a newer version as seen so the version 1 is considered stale - snapshot.note_document_seen_version(&uri, 2).await; + snapshot.note_document_seen_version(&uri, 2); on_did_open_text_document( snapshot.clone(), diff --git a/crates/glua_ls/src/handlers/text_document/watched_file_handler.rs b/crates/glua_ls/src/handlers/text_document/watched_file_handler.rs index a1bda5367..141f5cda0 100644 --- a/crates/glua_ls/src/handlers/text_document/watched_file_handler.rs +++ b/crates/glua_ls/src/handlers/text_document/watched_file_handler.rs @@ -105,6 +105,14 @@ pub async fn on_did_change_watched_files( .clear_push_file_diagnostics(uri.clone()) .await; } + } else { + // Never replay a report for a file that no longer exists. + for uri in &deleted_lua_uris { + context + .file_diagnostic() + .forget_cached_file_diagnostics(uri) + .await; + } } context diff --git a/crates/glua_ls/src/handlers/workspace/did_rename_files.rs b/crates/glua_ls/src/handlers/workspace/did_rename_files.rs index 38776d19e..27e9b5f6d 100644 --- a/crates/glua_ls/src/handlers/workspace/did_rename_files.rs +++ b/crates/glua_ls/src/handlers/workspace/did_rename_files.rs @@ -18,6 +18,11 @@ pub async fn on_did_rename_files_handler( context: ServerContextSnapshot, params: RenameFilesParams, ) -> Option<()> { + if !context.lsp_features().supports_apply_edit() { + log::warn!("rename import update skipped: client does not support workspace/applyEdit"); + return None; + } + let mut all_renames: Vec = vec![]; let analysis = context.analysis().read().await; diff --git a/crates/glua_ls/src/logger/mod.rs b/crates/glua_ls/src/logger/mod.rs index e5807a5c7..a682693bf 100644 --- a/crates/glua_ls/src/logger/mod.rs +++ b/crates/glua_ls/src/logger/mod.rs @@ -1,4 +1,5 @@ mod best_log_path; +mod non_blocking_stderr; use std::{env, fs, path::PathBuf}; @@ -7,18 +8,50 @@ use chrono::Local; use fern::Dispatch; use glua_code_analysis::file_path_to_uri; use log::{LevelFilter, info}; +use non_blocking_stderr::NonBlockingStderr; use crate::cmd_args::{CmdArgs, LogLevel}; const CRATE_NAME: &str = env!("CARGO_PKG_NAME"); const CRATE_VERSION: &str = env!("CARGO_PKG_VERSION"); +/// Work is spread over a thread pool, so interleaved debug/trace lines are only +/// correlatable if each one says which thread wrote it. +fn thread_tag(level: log::Level) -> String { + if level < log::Level::Debug { + return String::new(); + } + // Pool threads all share one name, so the id is what actually distinguishes them. + let current = std::thread::current(); + match current.name() { + Some(name) => format!(" {name}#{:?}", current.id()), + None => format!(" {:?}", current.id()), + } +} + +/// Applied on the root dispatch so the log file and stderr carry identical text. +fn format_record( + out: fern::FormatCallback<'_>, + message: &std::fmt::Arguments<'_>, + record: &log::Record<'_>, +) { + out.finish(format_args!( + "[{} {} {}{}] {}", + Local::now().format("%Y-%m-%d %H:%M:%S %:z"), + record.level(), + record.target(), + thread_tag(record.level()), + message + )) +} + pub fn init_logger(root: Option<&str>, cmd_args: &CmdArgs) { let level = match cmd_args.log_level { LogLevel::Error => LevelFilter::Error, LogLevel::Warn => LevelFilter::Warn, LogLevel::Info => LevelFilter::Info, LogLevel::Debug => LevelFilter::Debug, + LogLevel::Trace => LevelFilter::Trace, }; let cmd_log_path = cmd_args.log_path.clone(); @@ -71,20 +104,14 @@ pub fn init_logger(root: Option<&str>, cmd_args: &CmdArgs) { } }; + // Stderr as well as the file: an editor that starts the server as a child + // process shows stderr in its own output panel. It must be the + // non-blocking sink, or a client that never reads it stalls analysis. let logger = Dispatch::new() - .format(|out, message, record| { - out.finish(format_args!( - "[{} {} {}] {}", - Local::now().format("%Y-%m-%d %H:%M:%S %:z"), - record.level(), - record.target(), - message - )) - }) - // set level + .format(format_record) .level(level) - // set output - .chain(log_file); + .chain(log_file) + .chain(Box::new(NonBlockingStderr::new()) as Box); if let Err(e) = logger.apply() { eprintln!("Failed to apply logger: {:?}", e); @@ -98,19 +125,9 @@ pub fn init_logger(root: Option<&str>, cmd_args: &CmdArgs) { fn init_stderr_logger(level: LevelFilter) { let logger = Dispatch::new() - .format(|out, message, record| { - out.finish(format_args!( - "[{} {} {}] {}", - Local::now().format("%Y-%m-%d %H:%M:%S %:z"), - record.level(), - record.target(), - message - )) - }) - // set level + .format(format_record) .level(level) - // set output - .chain(std::io::stderr()); + .chain(Box::new(NonBlockingStderr::new()) as Box); if let Err(e) = logger.apply() { eprintln!("Failed to apply logger: {:?}", e); diff --git a/crates/glua_ls/src/logger/non_blocking_stderr.rs b/crates/glua_ls/src/logger/non_blocking_stderr.rs new file mode 100644 index 000000000..7465e5d05 --- /dev/null +++ b/crates/glua_ls/src/logger/non_blocking_stderr.rs @@ -0,0 +1,179 @@ +//! A stderr sink that drops whole log records rather than blocking the server. +//! +//! A client that does not read the server's stderr lets the pipe fill, and a +//! full pipe blocks the writer, which here is whichever analysis thread logged. +//! A startup writes more than a pipe buffer holds, so records go to a +//! background thread through a bounded queue and are dropped when it is full. +//! +//! `fern` splits one record over several `write` calls, so fragments are +//! accumulated here and queued only once a newline arrives. That keeps the queue +//! a count of lines rather than of format pieces, so a full queue drops whole +//! lines instead of cutting one in half. A record whose own message spans +//! several lines still queues one entry per line, and can lose some of them. + +use std::io::{self, Write}; +use std::sync::Arc; +use std::sync::atomic::{AtomicUsize, Ordering}; +use std::sync::mpsc::{Receiver, SyncSender, TrySendError, sync_channel}; + +/// Records allowed to queue before new ones are dropped. +const QUEUE_CAPACITY: usize = 4096; + +pub struct NonBlockingStderr { + /// `None` once the background thread is known to be gone. + sender: Option>>, + /// The fragments of the record being written, up to its newline. + record: Vec, + dropped: Arc, +} + +impl NonBlockingStderr { + pub fn new() -> Self { + let (sender, receiver) = sync_channel::>(QUEUE_CAPACITY); + let dropped = Arc::new(AtomicUsize::new(0)); + + let drain_dropped = dropped.clone(); + let sender = match std::thread::Builder::new() + .name("gluals-stderr".to_string()) + .spawn(move || drain(receiver, drain_dropped)) + { + Ok(_handle) => Some(sender), + Err(error) => { + // The logger is not up yet, so this is the only way to say it. + eprintln!( + "gluals: could not start the stderr log thread ({error}); stderr logging is disabled" + ); + None + } + }; + + Self { + sender, + record: Vec::new(), + dropped, + } + } + + #[cfg(test)] + fn with_capacity(capacity: usize) -> (Self, Receiver>) { + let (sender, receiver) = sync_channel::>(capacity); + ( + Self { + sender: Some(sender), + record: Vec::new(), + dropped: Arc::new(AtomicUsize::new(0)), + }, + receiver, + ) + } + + fn queue(&mut self, record: Vec) { + let Some(sender) = self.sender.as_ref() else { + return; + }; + + match sender.try_send(record) { + Ok(()) => {} + Err(TrySendError::Full(_)) => { + self.dropped.fetch_add(1, Ordering::Relaxed); + } + Err(TrySendError::Disconnected(_)) => self.sender = None, + } + } +} + +/// Writes queued records, prefixing however many were dropped while the queue +/// was full so the loss is visible in the output it interrupted. +fn drain(receiver: Receiver>, dropped: Arc) { + let stderr = io::stderr(); + for record in receiver { + let missing = dropped.swap(0, Ordering::Relaxed); + let mut handle = stderr.lock(); + if missing != 0 { + let _ = writeln!( + handle, + "gluals: dropped {missing} log record(s); stderr is not being read fast enough" + ); + } + let _ = handle.write_all(&record); + let _ = handle.flush(); + } +} + +impl Write for NonBlockingStderr { + fn write(&mut self, buf: &[u8]) -> io::Result { + self.record.extend_from_slice(buf); + while let Some(end) = self.record.iter().position(|byte| *byte == b'\n') { + let record = self.record.drain(..=end).collect(); + self.queue(record); + } + + // A dropped record is the intended outcome, so the write reports success. + Ok(buf.len()) + } + + /// Queues whatever has been written without a terminating newline. + /// + /// It cannot wait for the queue to drain: `fern` flushes after every + /// record, so a flush that blocked until the background thread caught up + /// would put the calling thread back behind the stderr pipe, which is the + /// stall this sink exists to avoid. + fn flush(&mut self) -> io::Result<()> { + if !self.record.is_empty() { + let record = std::mem::take(&mut self.record); + self.queue(record); + } + + Ok(()) + } +} + +#[cfg(test)] +mod tests { + use super::*; + + #[test] + fn writes_report_success_and_never_block() { + let (mut sink, _receiver) = NonBlockingStderr::with_capacity(4); + // Far more than the queue holds. If a full queue blocked or errored, + // this would hang or fail rather than run to completion. + for _ in 0..(QUEUE_CAPACITY * 2) { + let written = sink.write(b"line\n").expect("write should not fail"); + assert_eq!(written, 5); + } + sink.flush().expect("flush should not fail"); + } + + #[test] + fn a_record_queues_once_however_many_writes_it_takes() { + let (mut sink, receiver) = NonBlockingStderr::with_capacity(4); + + // Exactly the shape fern writes a record with; each format piece + // reaches `write` on its own. + let line_sep = "\n"; + write!( + sink, + "{}{}", + format_args!("[{}] {}", "INFO", "hello"), + line_sep + ) + .expect("write should not fail"); + sink.flush().expect("flush should not fail"); + + assert_eq!(receiver.try_recv().expect("one record"), b"[INFO] hello\n"); + assert!(receiver.try_recv().is_err()); + } + + #[test] + fn a_full_queue_drops_whole_records_and_counts_them() { + let (mut sink, receiver) = NonBlockingStderr::with_capacity(1); + + sink.write_all(b"first\n").expect("write should not fail"); + sink.write_all(b"second\n").expect("write should not fail"); + sink.write_all(b"third\n").expect("write should not fail"); + + assert_eq!(receiver.try_recv().expect("one record"), b"first\n"); + assert!(receiver.try_recv().is_err()); + assert_eq!(sink.dropped.load(Ordering::Relaxed), 2); + } +} diff --git a/crates/glua_ls/src/util/analysis_progress.rs b/crates/glua_ls/src/util/analysis_progress.rs new file mode 100644 index 000000000..30f5b83d9 --- /dev/null +++ b/crates/glua_ls/src/util/analysis_progress.rs @@ -0,0 +1,340 @@ +//! Forwards analysis phase reports to the status bar, the watchdog and the log. + +use std::collections::HashMap; +use std::sync::atomic::{AtomicBool, AtomicU64, Ordering}; +use std::sync::mpsc::{SyncSender, sync_channel}; +use std::sync::{Arc, Mutex}; +use std::thread::JoinHandle; +use std::time::{Duration, Instant}; + +use glua_code_analysis::progress; + +use crate::context::{ProgressTask, StatusBar}; +use crate::util::LongRunningWatchdogStatus; + +/// A phase has to run this long before its count updates are forwarded. Phase +/// changes always go through; this only rate-limits the counter inside one. +const MIN_UPDATE_INTERVAL: Duration = Duration::from_millis(100); + +/// How many phases the closing summary names. +const SUMMARY_PHASE_COUNT: usize = 5; + +/// A phase is logged on its own only if it ran at least this long. +const NOTABLE_PHASE: Duration = Duration::from_millis(250); + +/// Status messages allowed to queue before new ones are dropped. +const STATUS_QUEUE_CAPACITY: usize = 64; + +/// Which reporter owns the process-global sink. `progress` keeps one slot, so +/// two overlapping analyses share it and only the newest may clear it. +static SINK_GENERATION: AtomicU64 = AtomicU64::new(0); + +/// Installs a progress sink for as long as it is alive, and logs a summary of +/// where the time went when it is dropped. +pub struct AnalysisProgressReporter { + state: Arc>, + started: Instant, + generation: u64, + /// Cleared on drop, so a queued message cannot reach the client after + /// whatever the caller reports next. + forwarding: Arc, + /// Dropping this ends the forwarding thread. + status_sender: Option>, + /// Joined on drop, so a message already being delivered lands before the + /// caller's next report rather than racing it. + status_thread: Option>, +} + +struct ReporterState { + phase: String, + phase_started: Instant, + last_update: Instant, + /// Total time per phase. A phase repeats once per workspace group. + totals: HashMap, +} + +impl ReporterState { + /// Close off the running phase, adding its time to the totals. + fn finish_phase(&mut self, now: Instant) { + if self.phase.is_empty() { + return; + } + let elapsed = now.duration_since(self.phase_started); + // Only the slow ones: a phase repeats per workspace group. + if elapsed >= NOTABLE_PHASE { + log::info!("analysis phase '{}' took {:?}", self.phase, elapsed); + } + *self + .totals + .entry(std::mem::take(&mut self.phase)) + .or_default() += elapsed; + } +} + +/// Forwards status-bar messages off the analysis threads. +/// +/// The status bar reaches `lsp_server`'s rendezvous channel, so a send parks +/// the caller until the writer thread — itself parked on stdout — accepts it. A +/// client that stops reading stdout would otherwise stall every analysis worker +/// that reports progress. A progress update nobody sees costs nothing, so the +/// hop drops rather than blocks. +fn spawn_status_forwarder( + status_bar: StatusBar, + forwarding: Arc, +) -> Option<(SyncSender, JoinHandle<()>)> { + let (sender, receiver) = sync_channel::(STATUS_QUEUE_CAPACITY); + + match std::thread::Builder::new() + .name("gluals-progress".to_string()) + .spawn(move || { + for message in receiver { + if !forwarding.load(Ordering::Acquire) { + continue; + } + status_bar.update_startup_phase(ProgressTask::LoadWorkspace, None, message); + } + }) { + Ok(handle) => Some((sender, handle)), + Err(error) => { + log::error!("could not start the progress forwarding thread: {error}"); + None + } + } +} + +impl AnalysisProgressReporter { + pub fn install(status_bar: StatusBar, watchdog_status: LongRunningWatchdogStatus) -> Self { + let now = Instant::now(); + let state = Arc::new(Mutex::new(ReporterState { + phase: String::new(), + phase_started: now, + last_update: now - MIN_UPDATE_INTERVAL, + totals: HashMap::new(), + })); + + let generation = SINK_GENERATION.fetch_add(1, Ordering::AcqRel) + 1; + let forwarding = Arc::new(AtomicBool::new(true)); + let (status_sender, status_thread) = + match spawn_status_forwarder(status_bar, forwarding.clone()) { + Some((sender, handle)) => (Some(sender), Some(handle)), + None => (None, None), + }; + let sink_sender = status_sender.clone(); + let sink_state = state.clone(); + progress::set_sink(Arc::new(move |progress: progress::PhaseProgress<'_>| { + let progress::PhaseProgress { + phase, + done, + total, + unit, + } = progress; + let Ok(mut state) = sink_state.lock() else { + return; + }; + let now = Instant::now(); + if state.phase != phase { + state.finish_phase(now); + state.phase.push_str(phase); + state.phase_started = now; + } else if now.duration_since(state.last_update) < MIN_UPDATE_INTERVAL { + return; + } + state.last_update = now; + drop(state); + + // A pass counts its own batch, not the whole workspace. + let message = if total > 1 { + format!("{phase} ({done}/{total} {unit})") + } else { + phase.to_string() + }; + watchdog_status.set_phase(message.clone()); + if let Some(sender) = sink_sender.as_ref() { + let _ = sender.try_send(message); + } + })); + + Self { + state, + started: now, + generation, + forwarding, + status_sender, + status_thread, + } + } +} + +impl Drop for AnalysisProgressReporter { + fn drop(&mut self) { + self.forwarding.store(false, Ordering::Release); + + // A later reporter has taken the sink over; clearing would blind it. + if SINK_GENERATION.load(Ordering::Acquire) == self.generation { + progress::clear_sink(); + } + + // Closing the channel ends the loop; joining then waits out the one + // message that may already be inside `update_startup_phase`, so it + // cannot land after whatever the caller reports next. Everything still + // queued is discarded by the flag above, so this waits on at most one + // send — on the same channel the caller is about to use anyway. + // + // Safe to block here because the reporter is dropped after the analysis + // it wraps has finished, on the thread that started it rather than on a + // worker. A caller that installs a reporter around work whose threads + // outlive it would be blocking one of them here. + self.status_sender = None; + if let Some(handle) = self.status_thread.take() { + let _ = handle.join(); + } + + let Ok(mut state) = self.state.lock() else { + return; + }; + let now = Instant::now(); + state.finish_phase(now); + let mut totals = state.totals.drain().collect::>(); + drop(state); + + // Ties broken on the name so the order is stable. + totals.sort_by(|left, right| right.1.cmp(&left.1).then_with(|| left.0.cmp(&right.0))); + let slowest = totals + .iter() + .take(SUMMARY_PHASE_COUNT) + .map(|(phase, elapsed)| format!("{phase} {:.2}s", elapsed.as_secs_f64())) + .collect::>(); + + if slowest.is_empty() { + return; + } + log::info!( + "workspace analysis finished in {:.2}s; slowest phases: {}", + now.duration_since(self.started).as_secs_f64(), + slowest.join(", ") + ); + } +} + +#[cfg(test)] +mod tests { + use super::*; + + /// The sink is process-global, so these tests cannot run alongside each + /// other. + static SINK: Mutex<()> = Mutex::new(()); + + #[test] + fn clearing_the_sink_stops_reports() { + let _guard = SINK.lock().unwrap_or_else(|error| error.into_inner()); + // The reporter owns the global sink, so dropping it must clear it. + progress::clear_sink(); + assert!(!progress::is_active()); + + let counter = Arc::new(std::sync::atomic::AtomicUsize::new(0)); + let seen = counter.clone(); + progress::set_sink(Arc::new(move |_: progress::PhaseProgress<'_>| { + seen.fetch_add(1, std::sync::atomic::Ordering::Relaxed); + })); + assert!(progress::is_active()); + progress::enter_phase("phase", 0, "files"); + assert_eq!(counter.load(std::sync::atomic::Ordering::Relaxed), 1); + + progress::clear_sink(); + progress::enter_phase("phase", 0, "files"); + assert_eq!(counter.load(std::sync::atomic::Ordering::Relaxed), 1); + } + + #[test] + fn an_overlapping_reporter_keeps_the_sink_until_the_newest_one_goes() { + let _guard = SINK.lock().unwrap_or_else(|error| error.into_inner()); + progress::clear_sink(); + + let (connection, _peer) = lsp_server::Connection::memory(); + let status_bar = StatusBar::new( + Arc::new(crate::context::ClientProxy::new(connection)), + false, + ); + let watchdog = LongRunningWatchdogStatus::new("test"); + + let older = AnalysisProgressReporter::install(status_bar.clone(), watchdog.clone()); + let newer = AnalysisProgressReporter::install(status_bar, watchdog); + + drop(older); + assert!(progress::is_active()); + + drop(newer); + assert!(!progress::is_active()); + } + + /// Progress is forwarded off the analysis threads, so a report can still be + /// in flight when the reporter goes. Whatever the caller says next has to be + /// the last thing the client hears, or a finished workspace is left showing + /// a phase name. A report still queued at that point is discarded outright; + /// one already being delivered is waited out by the join in `Drop`. + #[test] + fn no_forwarded_report_arrives_after_the_caller_closes_the_task() { + let _guard = SINK.lock().unwrap_or_else(|error| error.into_inner()); + progress::clear_sink(); + + let (connection, peer) = lsp_server::Connection::memory(); + // The status bar drops every notification unless the client asked for + // work-done progress. + let status_bar = + StatusBar::new(Arc::new(crate::context::ClientProxy::new(connection)), true); + let watchdog = LongRunningWatchdogStatus::new("test"); + + // The client side is a rendezvous channel, so someone has to be reading + // it for a send to complete at all. + let reader = std::thread::spawn(move || { + let mut messages = Vec::new(); + for message in &peer.receiver { + let lsp_server::Message::Notification(notification) = message else { + continue; + }; + if let Some(text) = notification.params["value"]["message"].as_str() { + messages.push(text.to_string()); + } + } + messages + }); + + let reporter = AnalysisProgressReporter::install(status_bar.clone(), watchdog); + progress::enter_phase("Indexing", 0, "files"); + drop(reporter); + + status_bar.update_startup_phase(ProgressTask::LoadWorkspace, Some(100), "done"); + drop(status_bar); + + let messages = reader.join().expect("reader thread"); + assert_eq!( + messages.last().map(String::as_str), + Some("done"), + "the closing update must be the last thing the client hears, got {messages:?}" + ); + } + + #[test] + fn phase_totals_accumulate_across_repeats() { + // Phases repeat per workspace group, so the summary adds the repeats. + let now = Instant::now(); + let mut state = ReporterState { + phase: String::new(), + phase_started: now, + last_update: now, + totals: HashMap::new(), + }; + + state.phase.push_str("Inferring types"); + state.phase_started = now - Duration::from_secs(2); + state.finish_phase(now); + + state.phase.push_str("Inferring types"); + state.phase_started = now - Duration::from_secs(3); + state.finish_phase(now); + + assert_eq!(state.totals.len(), 1); + assert!(state.totals["Inferring types"] >= Duration::from_secs(5)); + assert!(state.phase.is_empty()); + } +} diff --git a/crates/glua_ls/src/util/long_running_watchdog.rs b/crates/glua_ls/src/util/long_running_watchdog.rs index b83f3fe0d..ceab5e825 100644 --- a/crates/glua_ls/src/util/long_running_watchdog.rs +++ b/crates/glua_ls/src/util/long_running_watchdog.rs @@ -44,15 +44,38 @@ impl LongRunningWatchdogSnapshot { } } -#[derive(Debug, Clone)] +/// Names what the task is currently working on. Called only when the watchdog +/// logs, so it may do real work. +pub type WatchdogDetailSource = Arc Option + Send + Sync>; + +#[derive(Clone)] pub struct LongRunningWatchdogStatus { snapshot: Arc>, + /// Kept beside the snapshot rather than in it so the snapshot stays a + /// plain value that can be cloned and logged. + detail_source: Arc>>, +} + +impl std::fmt::Debug for LongRunningWatchdogStatus { + fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { + f.debug_struct("LongRunningWatchdogStatus") + .field("snapshot", &self.snapshot) + .finish_non_exhaustive() + } } impl LongRunningWatchdogStatus { pub fn new(phase: impl Into) -> Self { Self { snapshot: Arc::new(Mutex::new(LongRunningWatchdogSnapshot::new(phase))), + detail_source: Arc::new(Mutex::new(None)), + } + } + + /// Attach something that can name what the task is currently working on. + pub fn set_detail_source(&self, source: WatchdogDetailSource) { + if let Ok(mut slot) = self.detail_source.lock() { + *slot = Some(source); } } @@ -79,6 +102,22 @@ impl LongRunningWatchdogStatus { .unwrap_or_else(|_| "status unavailable".to_string()) } + /// [`Self::describe`] plus whatever the detail source can add. For the + /// watchdog log, not the client-facing progress message. + pub fn describe_verbose(&self) -> String { + let described = self.describe(); + let detail = self + .detail_source + .lock() + .ok() + .and_then(|slot| slot.as_ref().map(|source| source())) + .flatten(); + match detail { + Some(detail) => format!("{described}; {detail}"), + None => described, + } + } + fn update(&self, update: impl FnOnce(&mut LongRunningWatchdogSnapshot)) { if let Ok(mut snapshot) = self.snapshot.lock() { update(&mut snapshot); @@ -148,7 +187,7 @@ pub fn spawn_long_running_watchdog( "{} still running after {}s: {}", task_name, elapsed.as_secs(), - status.describe() + status.describe_verbose() ); } } diff --git a/crates/glua_ls/src/util/mod.rs b/crates/glua_ls/src/util/mod.rs index 3729a05e3..afa0e01a1 100644 --- a/crates/glua_ls/src/util/mod.rs +++ b/crates/glua_ls/src/util/mod.rs @@ -1,8 +1,10 @@ +mod analysis_progress; mod desc; mod long_running_watchdog; mod module_name_convert; mod time_cancel_token; +pub use analysis_progress::AnalysisProgressReporter; pub use desc::*; pub use long_running_watchdog::*; pub use module_name_convert::{ diff --git a/crates/glua_parser/Cargo.toml b/crates/glua_parser/Cargo.toml index 588af8d63..c453d75c5 100644 --- a/crates/glua_parser/Cargo.toml +++ b/crates/glua_parser/Cargo.toml @@ -16,5 +16,7 @@ workspace = true [dependencies] rowan.workspace = true +rustc-hash.workspace = true +smol_str.workspace = true serde.workspace = true diff --git a/crates/glua_parser/src/syntax/mod.rs b/crates/glua_parser/src/syntax/mod.rs index b65e0398c..21ac2f434 100644 --- a/crates/glua_parser/src/syntax/mod.rs +++ b/crates/glua_parser/src/syntax/mod.rs @@ -11,7 +11,6 @@ use std::marker::PhantomData; use rowan::{Language, TextRange, TextSize}; use crate::kind::{LuaKind, LuaSyntaxKind, LuaTokenKind}; -pub use node::*; pub use traits::*; pub use tree::{LuaSyntaxTree, LuaTreeBuilder}; @@ -63,6 +62,66 @@ impl From for LuaTokenKind { } } +/// Per-thread memo for [`LuaSyntaxId::to_node_from_root`], keyed by root +/// (MRU, a few roots kept). Holding each root alive keeps its green tree +/// alive, which is what makes identity comparison sound. +mod node_memo { + use super::{LuaSyntaxId, LuaSyntaxNode}; + use rustc_hash::FxHashMap; + + const MAX_ROOTS: usize = 4; + /// Each entry pins a red node, which holds an rc on its whole ancestor + /// chain, so a long-lived thread would otherwise retain most of a large + /// tree. Clearing beats evicting: the memo only pays off within one + /// traversal, so a fresh map costs a re-walk, not a lasting miss. + const MAX_ENTRIES_PER_ROOT: usize = 8192; + + #[derive(Default)] + pub(super) struct NodeMemo { + roots: Vec<(LuaSyntaxNode, FxHashMap>)>, + } + + impl NodeMemo { + pub(super) fn resolve( + &mut self, + id: LuaSyntaxId, + root: &LuaSyntaxNode, + ) -> Option { + let found = self.roots.iter().position(|(cached, _)| cached == root); + let index = match found { + Some(0) => 0, + Some(index) => { + self.roots.swap(0, index); + 0 + } + None => { + if self.roots.len() == MAX_ROOTS { + self.roots.pop(); + } + self.roots.insert(0, (root.clone(), FxHashMap::default())); + 0 + } + }; + + if let Some(hit) = self.roots[index].1.get(&id) { + return hit.clone(); + } + let resolved = id.walk_from_root(root); + let entries = &mut self.roots[index].1; + if entries.len() >= MAX_ENTRIES_PER_ROOT { + entries.clear(); + } + entries.insert(id, resolved.clone()); + resolved + } + } +} + +thread_local! { + static NODE_MEMO: std::cell::RefCell = + std::cell::RefCell::new(node_memo::NodeMemo::default()); +} + #[derive(Debug, Clone, Copy, PartialEq, Eq, Hash)] pub struct LuaSyntaxId { kind: LuaKind, @@ -123,7 +182,13 @@ impl LuaSyntaxId { self.to_node_from_root(&root) } + /// Resolve this id to its node, memoized per thread: an uncached walk + /// allocates a red node per nesting level. pub fn to_node_from_root(&self, root: &LuaSyntaxNode) -> Option { + NODE_MEMO.with(|memo| memo.borrow_mut().resolve(*self, root)) + } + + fn walk_from_root(&self, root: &LuaSyntaxNode) -> Option { successors(Some(root.clone()), |node| { node.child_or_token_at_range(self.range)?.into_node() }) diff --git a/crates/glua_parser/src/syntax/node/doc/tag.rs b/crates/glua_parser/src/syntax/node/doc/tag.rs index d8a0b5e49..a3f927ec8 100644 --- a/crates/glua_parser/src/syntax/node/doc/tag.rs +++ b/crates/glua_parser/src/syntax/node/doc/tag.rs @@ -1611,6 +1611,10 @@ impl LuaAstNode for LuaDocTagOutparam { impl LuaDocDescriptionOwner for LuaDocTagOutparam {} impl LuaDocTagOutparam { + pub fn get_path_token(&self) -> Option { + self.token() + } + pub fn get_path(&self) -> Option { let mut path = String::new(); for child in self.syntax.children_with_tokens() { diff --git a/crates/glua_parser/src/syntax/node/lua/expr.rs b/crates/glua_parser/src/syntax/node/lua/expr.rs index 7bca5c591..6352fc926 100644 --- a/crates/glua_parser/src/syntax/node/lua/expr.rs +++ b/crates/glua_parser/src/syntax/node/lua/expr.rs @@ -8,6 +8,8 @@ use crate::{ }, }; +use smol_str::SmolStr; + use super::{ LuaBlock, LuaCallArgList, LuaIndexKey, LuaParamList, LuaTableField, path_trait::PathTrait, }; @@ -236,9 +238,15 @@ impl LuaNameExpr { self.token() } - pub fn get_name_text(&self) -> Option { + /// The identifier's text. + /// + /// Returns `SmolStr` rather than `String`: the token's text is already + /// `&str`, so building a `String` allocated for every name read. `SmolStr` + /// stores up to 22 bytes inline, which covers essentially every Lua + /// identifier, so the common case allocates nothing. + pub fn get_name_text(&self) -> Option { self.get_name_token() - .map(|it| it.get_name_text().to_string()) + .map(|it| SmolStr::new(it.get_name_text())) } } diff --git a/crates/glua_parser/src/syntax/node/lua/path_trait.rs b/crates/glua_parser/src/syntax/node/lua/path_trait.rs index b7c902058..13c910791 100644 --- a/crates/glua_parser/src/syntax/node/lua/path_trait.rs +++ b/crates/glua_parser/src/syntax/node/lua/path_trait.rs @@ -1,10 +1,30 @@ use crate::LuaAstNode; +use smol_str::SmolStr; use super::{LuaExpr, LuaIndexKey}; +/// Join path segments with `.` into a single `SmolStr`, sizing the buffer once. +fn join_path(paths: &[SmolStr]) -> SmolStr { + let width = paths.iter().map(|part| part.len() + 1).sum::(); + let mut joined = String::with_capacity(width); + for (index, part) in paths.iter().enumerate() { + if index > 0 { + joined.push('.'); + } + joined.push_str(part); + } + SmolStr::new(joined) +} + pub trait PathTrait: LuaAstNode { - fn get_access_path(&self) -> Option { - let mut paths = Vec::new(); + /// The dotted access path of this expression, e.g. `foo.bar.baz`. + /// + /// Returns `SmolStr` because paths are short and this is one of the hottest + /// allocation sites in analysis. A bare name — by far the common case — + /// returns without allocating at all: `paths` stays empty, so its backing + /// buffer is never allocated, and a name of 22 bytes or fewer lives inline. + fn get_access_path(&self) -> Option { + let mut paths: Vec = Vec::new(); let mut current_node = self.syntax().clone(); loop { match LuaExpr::cast(current_node)? { @@ -15,7 +35,7 @@ pub trait PathTrait: LuaAstNode { } else { paths.push(name); paths.reverse(); - return Some(paths.join(".")); + return Some(join_path(&paths)); } } LuaExpr::CallExpr(call_expr) => { @@ -25,21 +45,19 @@ pub trait PathTrait: LuaAstNode { LuaExpr::IndexExpr(index_expr) => { match index_expr.get_index_key()? { LuaIndexKey::String(s) => { - paths.push(s.get_value()); + paths.push(SmolStr::new(s.get_value())); } LuaIndexKey::Name(name) => { - paths.push(name.get_name_text().to_string()); + paths.push(SmolStr::new(name.get_name_text())); } LuaIndexKey::Integer(i) => { - paths.push(i.get_number_value().to_string()); + paths.push(SmolStr::new(i.get_number_value().to_string())); } LuaIndexKey::Expr(expr) => { - let text = format!("[{}]", expr.syntax().text()); - paths.push(text); + paths.push(SmolStr::new(format!("[{}]", expr.syntax().text()))); } LuaIndexKey::Idx(idx) => { - let text = format!("[{}]", idx); - paths.push(text); + paths.push(SmolStr::new(format!("[{}]", idx))); } } diff --git a/crates/glua_parser/src/syntax/node/mod.rs b/crates/glua_parser/src/syntax/node/mod.rs index fd50386d3..f314a4aa2 100644 --- a/crates/glua_parser/src/syntax/node/mod.rs +++ b/crates/glua_parser/src/syntax/node/mod.rs @@ -94,6 +94,7 @@ pub enum LuaAst { LuaDocTagAttribute(LuaDocTagAttribute), LuaDocTagAttributeUse(LuaDocTagAttributeUse), LuaDocTagFileparam(LuaDocTagFileparam), + LuaDocTagOutparam(LuaDocTagOutparam), // doc description LuaDocDescription(LuaDocDescription), @@ -187,6 +188,7 @@ impl LuaAstNode for LuaAst { LuaAst::LuaDocTagAttribute(node) => node.syntax(), LuaAst::LuaDocTagAttributeUse(node) => node.syntax(), LuaAst::LuaDocTagFileparam(node) => node.syntax(), + LuaAst::LuaDocTagOutparam(node) => node.syntax(), LuaAst::LuaDocTagLanguage(node) => node.syntax(), LuaAst::LuaDocDescription(node) => node.syntax(), LuaAst::LuaDocNameType(node) => node.syntax(), @@ -304,6 +306,7 @@ impl LuaAstNode for LuaAst { | LuaSyntaxKind::TypeMultiLineUnion | LuaSyntaxKind::DocAttributeUse | LuaSyntaxKind::DocTagFileparam + | LuaSyntaxKind::DocTagOutparam ) } @@ -480,6 +483,9 @@ impl LuaAstNode for LuaAst { LuaSyntaxKind::DocTagFileparam => { LuaDocTagFileparam::cast(syntax).map(LuaAst::LuaDocTagFileparam) } + LuaSyntaxKind::DocTagOutparam => { + LuaDocTagOutparam::cast(syntax).map(LuaAst::LuaDocTagOutparam) + } _ => None, } } diff --git a/crates/glua_parser_desc/src/markdown/mod.rs b/crates/glua_parser_desc/src/markdown/mod.rs index 66ef0bd21..8c3e23b7c 100644 --- a/crates/glua_parser_desc/src/markdown/mod.rs +++ b/crates/glua_parser_desc/src/markdown/mod.rs @@ -1665,17 +1665,14 @@ impl MarkdownParser { let is_right_flanking = !left_is_ws && (!left_is_punct || (right_is_ws || right_is_punct)); - let can_start_highlight; - let can_end_highlight; - if ch == '*' { - can_start_highlight = is_left_flanking; - can_end_highlight = is_right_flanking; + let (can_start_highlight, can_end_highlight) = if ch == '*' { + (is_left_flanking, is_right_flanking) } else { - can_start_highlight = - is_left_flanking && (!is_right_flanking || left_is_punct); - can_end_highlight = - is_right_flanking && (!is_left_flanking || right_is_punct); - } + ( + is_left_flanking && (!is_right_flanking || left_is_punct), + is_right_flanking && (!is_left_flanking || right_is_punct), + ) + }; if can_start_highlight && can_end_highlight { if self.has_highlight(ch, n_chars) { diff --git a/crates/glua_parser_desc/src/markdown_rst/mod.rs b/crates/glua_parser_desc/src/markdown_rst/mod.rs index b92c2a95b..d6701ec4f 100644 --- a/crates/glua_parser_desc/src/markdown_rst/mod.rs +++ b/crates/glua_parser_desc/src/markdown_rst/mod.rs @@ -273,16 +273,12 @@ impl MarkdownRstParser { // 1) Line // (1) Line - let line; - let next_line; - if start + 1 < lines.len() { + let (line, next_line) = if start + 1 < lines.len() { let [got_line, got_next_line] = lines.get_disjoint_mut([start, start + 1]).unwrap(); - line = got_line; - next_line = Some(got_next_line); + (got_line, Some(got_next_line)) } else { - line = &mut lines[start]; - next_line = None; - } + (&mut lines[start], None) + }; let bt = BacktrackPoint::new(self, line); let scope_start = line.current_range().start_offset; diff --git a/docs/mintlify/annotations/outparam.mdx b/docs/mintlify/annotations/outparam.mdx index 04350c4b4..3a13a62e4 100644 --- a/docs/mintlify/annotations/outparam.mdx +++ b/docs/mintlify/annotations/outparam.mdx @@ -15,14 +15,37 @@ Some functions write results into a table you pass in, instead of returning them ```lua ---@outparam paramName.fieldPath Type +---@outparam paramName Type ``` - **`paramName`** — a parameter on the same function (must match an `@param`). -- **`fieldPath`** — the field the function writes to. Use dots for nested fields. +- **`fieldPath`** — the field the function writes to. Use dots for nested fields. Leave it out to type the argument itself. - **`Type`** — the type the field will have after the call. --- +## Typing the argument itself + +Leave off the field path when the function fills in the table you pass, instead of a field inside it: + +```lua +---@class HUDStackLayout +---@field rowHeight integer + +---@outparam out HUDStackLayout +---@param out table Table owned by the caller. +function Glide.GetHUDStackLayout(out) end +``` + +```lua +local layout = {} +Glide.GetHUDStackLayout(layout) + +local h = layout.rowHeight -- ✅ integer +``` + +--- + ## Basic usage ```lua diff --git a/tools/benchmark/src/main.rs b/tools/benchmark/src/main.rs index 535405814..af836a678 100644 --- a/tools/benchmark/src/main.rs +++ b/tools/benchmark/src/main.rs @@ -140,6 +140,25 @@ fn run_incremental_edits( let mut total = std::time::Duration::ZERO; let mut worst = std::time::Duration::ZERO; let mut edited = 0usize; + // `BENCH_EDIT_REPEAT=N` edits each file N times, which both fills a + // sampling profiler's edit window and separates first-edit cache warming + // from the steady-state cost of typing. + let repeats: usize = std::env::var("BENCH_EDIT_REPEAT") + .ok() + .and_then(|value| value.parse().ok()) + .unwrap_or(1) + .max(1); + let sample: Vec<(FileId, usize)> = sample + .into_iter() + .flat_map(|entry| std::iter::repeat_n(entry, repeats)) + .collect(); + if std::env::var_os("BENCH_IDEMPOTENCY").is_some() { + if let Some((file_id, _)) = sample.first().copied() { + report_reindex_idempotency(analysis, file_id); + } + return None; + } + for (file_id, expansion) in sample { let Some(uri) = analysis.compilation.get_db().get_vfs().get_uri(&file_id) else { continue; @@ -169,9 +188,46 @@ fn run_incremental_edits( .map_or(0.0, |s| s.elapsed().as_secs_f64()) ); } - let t = Instant::now(); - analysis.update_file_by_uri(&uri, Some(edited_text)); - let reindex = t.elapsed(); + // `BENCH_EDIT_STAGED=1` splits the keystroke into the two halves a + // position-based request actually depends on: re-indexing the edited + // file alone, then the dependency ripple. It reports what a handler + // gated on the edited file's own freshness would wait for. + let reindex = if std::env::var_os("BENCH_EDIT_STAGED").is_some() { + // The expansion has to be captured before the edit lands, exactly as + // the production edit path does; recomputing it after the edited + // file has been re-indexed under-expands. + let expansion = analysis.expand_reindex_file_ids(vec![file_id]); + analysis.update_file_text_only(&uri, edited_text); + // Just the edited file's own entries — no cross-file stabilization + // and no expansion. This is the floor a position-based request has + // to wait for if it is gated on its own file rather than on the + // whole ripple. + let t = Instant::now(); + analysis.compilation.remove_index(vec![file_id]); + analysis.compilation.update_index(vec![file_id]); + let self_only = t.elapsed(); + // `BENCH_EDIT_SELF_ONLY=1` stops after the edited file's own + // entries, so a profile of the run contains nothing but the cost a + // per-file freshness gate would pay. + let ripple = if std::env::var_os("BENCH_EDIT_SELF_ONLY").is_some() { + std::time::Duration::ZERO + } else { + let t = Instant::now(); + analysis.reindex_expanded_files(vec![file_id], expansion); + t.elapsed() + }; + eprintln!( + " [incremental] {name} staged: {:.3}s self-only + {:.3}s ripple", + self_only.as_secs_f64(), + ripple.as_secs_f64() + ); + self_only + ripple + } else { + let t = Instant::now(); + analysis.update_file_by_uri(&uri, Some(edited_text)); + t.elapsed() + }; + let t = Instant::now(); let shared = analysis.precompute_diagnostic_shared_data(); analysis.diagnose_file_with_shared(file_id, CancellationToken::new(), shared); @@ -186,7 +242,18 @@ fn run_incremental_edits( reindex.as_secs_f64(), diagnostics.as_secs_f64() ); - analysis.update_file_by_uri(&uri, Some(text)); + // Reverting through the full path costs a whole ripple per iteration — + // several times the self-index being measured, and untimed, so it + // would dominate any profile of this loop. `BENCH_EDIT_SELF_ONLY` + // exists to leave nothing but the self-index in the profile, so the + // revert has to match it. + if std::env::var_os("BENCH_EDIT_SELF_ONLY").is_some() { + analysis.update_file_text_only(&uri, text); + analysis.compilation.remove_index(vec![file_id]); + analysis.compilation.update_index(vec![file_id]); + } else { + analysis.update_file_by_uri(&uri, Some(text)); + } } if edited == 0 { return None; @@ -200,6 +267,98 @@ fn run_incremental_edits( Some(worst) } +/// Everything a *consumer* of a file could observe from it: the members it +/// attaches and the types it has inferred. +fn contribution_entries(analysis: &EmmyLuaAnalysis, file_id: FileId) -> Vec { + let db = analysis.compilation.get_db(); + let member_index = db.get_member_index(); + let mut entries = Vec::new(); + for (owner, cache) in db.get_type_index().iter_type_caches() { + if owner.get_file_id() == file_id { + entries.push(format!("type {owner:?} = {:?}", cache.as_type())); + } + } + for member in member_index.get_file_members(file_id) { + entries.push(format!( + "member {:?} owner={:?}", + member.get_key(), + member_index.get_member_owner(&member.get_id()) + )); + } + entries.sort(); + entries +} + +/// `BENCH_IDEMPOTENCY=1` re-indexes the target with its text untouched and +/// reports what the workspace disagrees with itself about afterwards. +/// +/// Re-analysing a file whose text and inputs are unchanged ought to reproduce +/// exactly what was already there. Where it does not, every "has this actually +/// changed?" optimisation downstream is dead on arrival, because every file +/// reports itself as changed. +fn report_reindex_idempotency(analysis: &mut EmmyLuaAnalysis, file_id: FileId) { + let expansion = analysis.expand_reindex_file_ids(vec![file_id]); + let before = expansion + .iter() + .map(|id| (*id, contribution_entries(analysis, *id))) + .collect::>(); + + analysis.reindex_files(vec![file_id]); + + let mut changed = 0usize; + let mut shown = 0usize; + for (id, was) in &before { + let now = contribution_entries(analysis, *id); + if &now == was { + continue; + } + changed += 1; + let show_limit: usize = std::env::var("BENCH_IDEMPOTENCY_SHOW") + .ok() + .and_then(|value| value.parse().ok()) + .unwrap_or(2); + if shown >= show_limit { + continue; + } + shown += 1; + let name = analysis + .compilation + .get_db() + .get_vfs() + .get_file_path(id) + .map(|path| path.display().to_string()) + .unwrap_or_else(|| format!("{id:?}")); + eprintln!(" [idempotency] {name}"); + let count = |lines: &[String]| { + let mut counts = std::collections::BTreeMap::::new(); + for line in lines { + *counts.entry(line.clone()).or_default() += 1; + } + counts + }; + let (was_counts, now_counts) = (count(was), count(&now)); + let mut shown_lines = 0; + for (line, was_n) in &was_counts { + let now_n = now_counts.get(line).copied().unwrap_or(0); + if *was_n != now_n && shown_lines < 6 { + shown_lines += 1; + eprintln!(" {was_n} -> {now_n}: {line}"); + } + } + for (line, now_n) in &now_counts { + if !was_counts.contains_key(line) && shown_lines < 6 { + shown_lines += 1; + eprintln!(" 0 -> {now_n}: {line}"); + } + } + } + eprintln!( + " [idempotency] no-op reindex of {} files changed {} of them", + expansion.len(), + changed + ); +} + fn discover_config_files(root: &Path) -> Vec { let gluarc = root.join(".gluarc.json"); if gluarc.exists() { @@ -215,8 +374,25 @@ fn discover_config_files(root: &Path) -> Vec { .collect() } -#[tokio::main] -async fn main() { +/// Analysis recurses over deeply nested syntax. The server does that work on +/// spawned threads, which get a far larger stack than a process main thread does +/// on Windows, so the tools have to ask for one explicitly. +fn main() { + std::thread::Builder::new() + .stack_size(256 * 1024 * 1024) + .spawn(|| { + tokio::runtime::Builder::new_multi_thread() + .enable_all() + .build() + .expect("tokio runtime should build") + .block_on(run()); + }) + .expect("benchmark worker thread should spawn") + .join() + .expect("benchmark worker thread should not panic"); +} + +async fn run() { let _ = PROCESS_START.set(Instant::now()); #[allow(unused_mut, unused_assignments, unused_variables)] let mut alloc_mark = (0u64, 0u64); @@ -279,6 +455,34 @@ async fn main() { // Add annotations as library workspace analysis.add_library_workspace(annotations_path.clone()); + // The server resolves the gamemode base and loads it as a library, so a + // benchmark without those roots re-indexes a much smaller dependency + // expansion than a keystroke really pays for. `BENCH_LIBS` lists the extra + // library roots (e.g. the `sandbox` and `base` gamemodes) so the harness + // measures the workspace the editor actually has open. + let extra_libraries = std::env::var("BENCH_LIBS") + .ok() + .into_iter() + .flat_map(|libs| { + libs.split(',') + .map(str::trim) + .filter(|lib| !lib.is_empty()) + .map(PathBuf::from) + .collect::>() + }) + .collect::>(); + for library in &extra_libraries { + if !library.exists() { + eprintln!( + "ERROR: BENCH_LIBS path does not exist: {}", + library.display() + ); + std::process::exit(1); + } + eprintln!("Library: {}", library.display()); + analysis.add_library_workspace(library.clone()); + } + // Add main workspace analysis.add_main_workspace(large_path.clone()); results.push(BenchmarkResult { @@ -288,10 +492,13 @@ async fn main() { // Phase 3: Collect files let t = Instant::now(); - let mut workspace_folders = vec![ - WorkspaceFolder::new(annotations_path.clone(), true), - WorkspaceFolder::new(large_path.clone(), false), - ]; + let mut workspace_folders = vec![WorkspaceFolder::new(annotations_path.clone(), true)]; + workspace_folders.extend( + extra_libraries + .iter() + .map(|library| WorkspaceFolder::new(library.clone(), true)), + ); + workspace_folders.push(WorkspaceFolder::new(large_path.clone(), false)); // Add library paths from config for lib in &emmyrc.workspace.library { @@ -363,7 +570,12 @@ async fn main() { // the ranking pass, which costs ~27s and swamps a CPU profile. let explicit_targets = std::env::var("BENCH_EDIT_TARGETS").ok(); if let Some(targets) = &explicit_targets { - let wanted: Vec<&str> = targets.split(',').map(str::trim).collect(); + // A target is either a bare file name or a path suffix, so that a + // common name like `shared.lua` can be pinned to one entity. + let wanted: Vec = targets + .split(',') + .map(|target| target.trim().replace('\\', "/").to_lowercase()) + .collect(); let sample: Vec<(FileId, usize)> = main_ids .iter() .filter(|id| { @@ -372,10 +584,12 @@ async fn main() { .get_db() .get_vfs() .get_file_path(id) - .and_then(|p| p.file_name().map(|n| n.to_string_lossy().into_owned())) - .is_some_and(|name| wanted.iter().any(|w| name == *w)) + .map(|path| path.to_string_lossy().replace('\\', "/").to_lowercase()) + .is_some_and(|path| { + wanted.iter().any(|wanted| path.ends_with(wanted.as_str())) + }) }) - .map(|id| (*id, 0)) + .map(|id| (*id, analysis.expand_reindex_file_ids(vec![*id]).len())) .collect(); #[cfg(feature = "dhat-heap")] let dhat_edit = dhat::Profiler::builder() @@ -393,6 +607,36 @@ async fn main() { .map(|id| (*id, analysis.expand_reindex_file_ids(vec![*id]).len())) .collect(); ranked.sort_by_key(|(_, n)| std::cmp::Reverse(*n)); + { + // What an edit costs depends almost entirely on how many files + // it drags in, so the shape of that distribution matters more + // than the worst case the sample below reports. + let sizes: Vec = ranked.iter().map(|(_, n)| *n).collect(); + let total = sizes.len(); + let pct = |p: usize| sizes[(total.saturating_sub(1)) * (100 - p) / 100]; + let buckets = [1usize, 5, 20, 100, 500, usize::MAX]; + let mut counts = vec![0usize; buckets.len()]; + for n in &sizes { + for (idx, limit) in buckets.iter().enumerate() { + if n <= limit { + counts[idx] += 1; + break; + } + } + } + eprintln!( + " [incremental] expansion distribution over {total} files: median {} p75 {} p90 {} p99 {} max {}", + pct(50), + pct(75), + pct(90), + pct(99), + sizes.first().copied().unwrap_or(0) + ); + eprintln!( + " [incremental] <=1: {} | <=5: {} | <=20: {} | <=100: {} | <=500: {} | >500: {}", + counts[0], counts[1], counts[2], counts[3], counts[4], counts[5] + ); + } eprintln!( " [incremental] ranked {} files by reindex expansion in {:.3}s", ranked.len(), diff --git a/tools/determinism/Cargo.toml b/tools/determinism/Cargo.toml index 4acda98e3..da350ec7d 100644 --- a/tools/determinism/Cargo.toml +++ b/tools/determinism/Cargo.toml @@ -11,6 +11,10 @@ glua_parser.workspace = true tokio-util.workspace = true lsp_types.workspace = true mimalloc.workspace = true +# Allocation sampling (DET_ALLOC_SAMPLE): capture raw frame addresses cheaply +# during the run and resolve them only when reporting. Resolving per sample is +# orders of magnitude slower — slow enough to never finish a full run. +backtrace = "0.3" [[bin]] name = "determinism" diff --git a/tools/determinism/src/main.rs b/tools/determinism/src/main.rs index 1890f36c8..882c0a89b 100644 --- a/tools/determinism/src/main.rs +++ b/tools/determinism/src/main.rs @@ -24,9 +24,9 @@ //! preserves state — it cannot verify re-analysis, because its edit pair is //! exactly what that gate rejects as meaningless. `realedit` is the gate for //! re-analysis itself: its edit changes what the file means, so the incremental -//! result has to land where a cold build of the edited source lands. It is a -//! real gate and it does pass — measured IDENTICAL on CityRP at 11,655 entries -//! — so a divergence there is a regression, not a known hole. +//! result has to land where a cold build of the edited source lands. It does +//! not reach zero today — see AGENTS.md for the known divergence and its cause +//! — so measure it before and after a change and treat any *growth* as yours. //! //! `mainreindex`, `exact`, `editmid` and `split:N` are **bisect stages** — diagnostic //! instruments, not gates, and they are expected to diverge. Both `mainreindex` @@ -64,6 +64,25 @@ //! and skipping must preserve state. It does not //! exercise re-analysis — `realedit` gates that, //! and `editmid` bisects it. +//! indexrepeat +//! re-index each DET_TARGETS entry with its text +//! untouched and require the INDEX to come back +//! identical. The diagnostic gates cannot see +//! this: re-analysing a file can attach different +//! members, or settle a decl's type differently +//! from the cold build, and still produce the same +//! diagnostics — `repeat` and `noopedit` both +//! report IDENTICAL while the index underneath has +//! drifted. That drift is why no incremental work +//! can be skipped: every "did this actually +//! change?" test answers yes. Does **not** pass +//! today (CityRP: 82 type caches, 3 signatures and +//! 11 class members change), and it is a real +//! defect rather than a harness artefact, so treat +//! any *growth* in those counts as yours. Listed +//! last in the default set because it re-indexes +//! in place and leaves that warm state behind, so +//! an in-place stage after it inherits it. //! editmid offset-shifting no-op edit pair (newline at the //! front of the file, then removed): the semantic //! no-op gate cannot fire, so both edits run the @@ -75,6 +94,21 @@ //! inherits that state and its diff is //! meaningless. Honours DET_INDEX_DIFF (cold index //! snapshot vs the state each target leaves behind). +//! editrevert +//! apply a real edit through the single-file +//! update path and then take it back out. The +//! source ends where it started, so the INDEX and +//! the diagnostics have to as well — a difference +//! is drift the edit path introduced, not a fact +//! about the code. Needs DET_EDIT_FIND; without it +//! the stage skips. Runs before `realedit`, which +//! leaves the edited file re-indexed behind it. +//! Covers what the other stages cannot: +//! `indexrepeat` re-indexes with the text +//! untouched and so never exercises an edit's +//! invalidation, and `noopedit`'s pair is +//! semantically neutral, so the update path skips +//! the re-index outright. //! realedit apply a real edit (DET_EDIT_FIND replaced by //! DET_EDIT_REPLACE) to each DET_TARGETS entry and //! compare the incremental result against a cold @@ -86,9 +120,32 @@ //! Always diffs the index, DET_INDEX_DIFF or not. //! Needs DET_EDIT_FIND; without it the stage //! skips loudly instead of gating anything. +//! burst three edits per DET_TARGETS entry, each one +//! self-indexed on its own, then ONE ripple over +//! the union of the three separately-captured +//! expansions — the sequence a debounce that +//! defers the ripple behind a longer idle timer +//! produces. Gates that union against a cold build +//! of the final text. An expansion recomputed +//! after a self-index under-expands badly (739 +//! files collapsed to 8), so union is the only +//! shape that can work; this measures whether it +//! does. Needs DET_EDIT_FIND; without it the stage +//! skips loudly instead of gating anything. //! exact reindex DET_TARGETS with no text change and no //! dependency expansion (bisects which file's //! re-analysis perturbs a fact) +//! perfile remove and re-add every file ON ITS OWN, so +//! each one is re-derived against the complete +//! settled workspace instead of the prefix the +//! cold walk had built when it reached that file. +//! Says whether the cold answer is simply one +//! computed from an incomplete view. It is, and it +//! converges: on CityRP round 1 moves 4277 type +//! caches away from cold and round 2 moves +//! nothing. Honours DET_PERFILE_ROUNDS (default +//! 2). Slow -- ~830s a round -- because every file +//! pays the whole pipeline. //! reindex full clear + rebuild, the ground truth //! order rebuild with the file list reversed //! split:N rebuild in N batches instead of one @@ -123,6 +180,10 @@ //! DET_SHOW_EXPANSION print the reindex expansion set for each edit //! DET_DUMP write the cold diagnostic set to this path //! DET_DUMP_FILE_IDS print the main-workspace file id table +//! DET_DUMP_EXPANSION list every file `indexrepeat` re-indexes, so a drifting +//! entry can be told apart from one merely near the batch: +//! whether the writer of a fact sits inside or outside it is +//! what decides whether the reader saw it settled //! DET_DUMP_CLASS print the member list of this class at each snapshot //! DET_LIMIT max diff lines printed per bucket (default 40) //! DET_FILTER substring an index-diff entry line must contain to be @@ -131,9 +192,286 @@ //! hides behind the first 40 unrelated entries. use mimalloc::MiMalloc; +use std::alloc::{GlobalAlloc, Layout}; + +/// mimalloc, plus a count of every allocation it hands out. +/// +/// A sampling profiler blames the allocator, never the code that asked for the +/// memory, so it cannot answer "is this phase allocation-bound?". Counting can: +/// `GLUALS_PROFILE=1` prints allocations alongside each phase's cost, and +/// dividing by the phase's unit of work gives allocations-per-step directly. +struct CountingMiMalloc; + +/// Sample one in every `DET_ALLOC_SAMPLE` allocations and record where it came +/// from. This is a poor-man's allocation profiler: it attributes allocations to +/// source locations, which a CPU sampling profiler cannot do (it blames the +/// allocator) — and it works even where external profilers fail to read the PDB. +mod alloc_sample { + use std::collections::HashMap; + use std::sync::Mutex; + use std::sync::atomic::{AtomicU64, AtomicUsize, Ordering}; + + const MAX_FRAMES: usize = 64; + + // Documented in WinBase.h; returns the number of frames written. + #[cfg(windows)] + unsafe extern "system" { + fn RtlCaptureStackBackTrace( + frames_to_skip: u32, + frames_to_capture: u32, + back_trace: *mut *mut std::ffi::c_void, + back_trace_hash: *mut u32, + ) -> u16; + } + + /// Raw instruction pointers for the current stack, innermost first. Returns + /// how many of `buffer` were filled. + /// + /// `backtrace::trace` goes through dbghelp's StackWalkEx on Windows, which + /// takes a process-wide lock and costs milliseconds per capture: a full run + /// never finished. `RtlCaptureStackBackTrace` unwinds via the x64 unwind + /// tables instead and costs microseconds. Elsewhere the crate's own unwinder + /// is already cheap enough. + #[cfg(windows)] + fn capture(buffer: &mut [*mut std::ffi::c_void]) -> usize { + let captured = unsafe { + RtlCaptureStackBackTrace( + 1, + buffer.len() as u32, + buffer.as_mut_ptr(), + std::ptr::null_mut(), + ) + }; + captured as usize + } + + #[cfg(not(windows))] + fn capture(buffer: &mut [*mut std::ffi::c_void]) -> usize { + let mut filled = 0; + backtrace::trace(|frame| { + if filled >= buffer.len() { + return false; + } + buffer[filled] = frame.ip(); + filled += 1; + true + }); + filled + } + + static SAMPLE_RATE: AtomicUsize = AtomicUsize::new(0); + static PHASE_SCOPED: std::sync::atomic::AtomicBool = std::sync::atomic::AtomicBool::new(false); + static TICK: AtomicU64 = AtomicU64::new(0); + /// Frame address -> number of sampled allocations whose stack contained it. + /// Addresses are resolved to names once, at report time. + static FRAMES: Mutex>> = Mutex::new(None); + /// Ordered stacks (innermost first) -> count. Kept alongside FRAMES because + /// "which of our functions asked for this memory" needs frame order, which + /// the flat per-frame tally throws away. + type StackCounts = HashMap, u64>; + static STACKS: Mutex> = Mutex::new(None); + + thread_local! { + /// Capturing a backtrace allocates; without this guard the sampler + /// would recurse into itself. + static SAMPLING: std::cell::Cell = const { std::cell::Cell::new(false) }; + } + + pub fn init() { + let rate = std::env::var("DET_ALLOC_SAMPLE") + .ok() + .and_then(|v| v.parse::().ok()) + .unwrap_or(0); + SAMPLE_RATE.store(rate, Ordering::Relaxed); + PHASE_SCOPED.store( + std::env::var_os("GLUALS_PROFILE_SAMPLE").is_some(), + Ordering::Relaxed, + ); + } + + #[inline] + pub fn maybe_sample() { + let rate = SAMPLE_RATE.load(Ordering::Relaxed); + if rate == 0 { + return; + } + // Sample one phase only (GLUALS_PROFILE_SAMPLE), when asked to. + if PHASE_SCOPED.load(Ordering::Relaxed) + && !glua_code_analysis::profile::sample_phase_active() + { + return; + } + if !TICK + .fetch_add(1, Ordering::Relaxed) + .is_multiple_of(rate as u64) + { + return; + } + SAMPLING.with(|sampling| { + if sampling.get() { + return; + } + sampling.set(true); + // Raw instruction pointers only — no symbol resolution here. + let mut buffer = [std::ptr::null_mut::(); MAX_FRAMES]; + let captured = capture(&mut buffer); + let mut ips: Vec = buffer[..captured].iter().map(|&ip| ip as usize).collect(); + { + let mut stacks = STACKS.lock().unwrap_or_else(|p| p.into_inner()); + *stacks + .get_or_insert_with(HashMap::new) + .entry(ips.as_slice().into()) + .or_insert(0) += 1; + } + let mut frames = FRAMES.lock().unwrap_or_else(|p| p.into_inner()); + let frames = frames.get_or_insert_with(HashMap::new); + // Count each frame once per sample, so the number reads as "share of + // sampled allocations made underneath this function". + ips.sort_unstable(); + ips.dedup(); + for ip in ips { + *frames.entry(ip).or_insert(0) += 1; + } + sampling.set(false); + }); + } + + /// Attribute each sampled allocation to the innermost frame belonging to our + /// own crates. The inclusive tally says an allocation happened *somewhere* + /// under a function; this says which of our functions actually asked for the + /// memory, which is the line you can go and change. + fn nearest_caller_report(top: usize) { + let stacks = STACKS.lock().unwrap_or_else(|p| p.into_inner()); + let Some(stacks) = stacks.as_ref() else { + return; + }; + let total: u64 = stacks.values().sum(); + if total == 0 { + return; + } + + let mut names: HashMap> = HashMap::new(); + let mut resolve = |ip: usize| -> Option { + names + .entry(ip) + .or_insert_with(|| { + let mut found = None; + backtrace::resolve(ip as *mut _, |symbol| { + if found.is_none() { + found = symbol.name().map(|name| name.to_string()); + } + }); + found + }) + .clone() + }; + + let mut by_caller: HashMap = HashMap::new(); + for (stack, count) in stacks { + let owner = stack.iter().find_map(|&ip| { + let name = resolve(ip)?; + (name.starts_with("glua_") && !name.contains("::profile::")).then_some(name) + }); + let owner = owner.unwrap_or_else(|| "".to_string()); + let owner = owner + .rsplit_once("::h") + .filter(|(_, hash)| hash.len() == 16) + .map_or(owner.as_str(), |(head, _)| head) + .to_string(); + *by_caller.entry(owner).or_insert(0) += count; + } + + let mut rows: Vec<_> = by_caller.into_iter().collect(); + rows.sort_unstable_by_key(|(_, count)| std::cmp::Reverse(*count)); + eprintln!("\n=== allocations by nearest owning function ({total} samples) ==="); + for (name, count) in rows.into_iter().take(top) { + eprintln!("{:>6.2}% {name}", (count as f64 / total as f64) * 100.0); + } + } + + /// Print the functions that appear most often across sampled allocations. + pub fn report(top: usize) { + // Symbol resolution allocates, so a sample taken while reporting would + // re-enter and block on a lock this thread already holds. + SAMPLE_RATE.store(0, Ordering::Relaxed); + + let frames = FRAMES.lock().unwrap_or_else(|p| p.into_inner()); + let Some(frames) = frames.as_ref() else { + return; + }; + // Samples, not frame hits: the most-hit frame is the allocator entry, + // which every sample passes through. + let total = frames.values().copied().max().unwrap_or(0); + if total == 0 { + return; + } + + // Resolve once, then fold the per-address counts into per-function ones + // (a function inlined or spread over several addresses is one entry). + let mut by_name: HashMap = HashMap::new(); + for (&ip, &count) in frames { + backtrace::resolve(ip as *mut _, |symbol| { + let Some(name) = symbol.name() else { return }; + let name = name.to_string(); + // Strip the trailing hash rustc appends to monomorphized names. + let name = name + .rsplit_once("::h") + .filter(|(_, hash)| hash.len() == 16) + .map_or(name.as_str(), |(head, _)| head) + .to_string(); + *by_name.entry(name).or_insert(0) += count; + }); + } + + nearest_caller_report(top); + + let mut rows: Vec<_> = by_name + .into_iter() + .filter(|(name, _)| { + !name.starts_with("core::") + && !name.starts_with("alloc::") + && !name.starts_with("std::") + && !name.contains("hashbrown") + && !name.contains("mi_") + && !name.contains("CountingMiMalloc") + && !name.contains("alloc_sample") + }) + .collect(); + rows.sort_unstable_by_key(|(_, count)| std::cmp::Reverse(*count)); + eprintln!("\n=== sampled allocation frames ({total} samples) ==="); + eprintln!("share of sampled allocations made underneath each function:"); + for (name, count) in rows.into_iter().take(top) { + eprintln!("{:>6.2}% {name}", (count as f64 / total as f64) * 100.0); + } + } +} + +// SAFETY: every method forwards to MiMalloc with the same arguments; the +// counters are plain relaxed atomics and do not affect allocation behavior. +unsafe impl GlobalAlloc for CountingMiMalloc { + unsafe fn alloc(&self, layout: Layout) -> *mut u8 { + glua_code_analysis::profile::record_alloc(layout.size()); + alloc_sample::maybe_sample(); + unsafe { MiMalloc.alloc(layout) } + } + + unsafe fn dealloc(&self, ptr: *mut u8, layout: Layout) { + unsafe { MiMalloc.dealloc(ptr, layout) } + } + + unsafe fn alloc_zeroed(&self, layout: Layout) -> *mut u8 { + glua_code_analysis::profile::record_alloc(layout.size()); + unsafe { MiMalloc.alloc_zeroed(layout) } + } + + unsafe fn realloc(&self, ptr: *mut u8, layout: Layout, new_size: usize) -> *mut u8 { + glua_code_analysis::profile::record_alloc(new_size); + unsafe { MiMalloc.realloc(ptr, layout, new_size) } + } +} #[global_allocator] -static GLOBAL: MiMalloc = MiMalloc; +static GLOBAL: CountingMiMalloc = CountingMiMalloc; use std::collections::{BTreeMap, BTreeSet, HashSet}; use std::path::{Path, PathBuf}; @@ -350,6 +688,17 @@ fn collect(analysis: &EmmyLuaAnalysis, label: &str) -> BTreeSet { /// Snapshot of the derived index state that diagnostics read from. struct IndexSnapshot { type_caches: BTreeMap, + /// How each cached type was reached, under `DET_PROVENANCE`. Diffed apart + /// from the type so a value that stayed put while the reasoning behind it + /// moved is still visible. + type_facts: BTreeMap, + /// The assignment-contribution store, under `DET_PROVENANCE`: which writers + /// each owner/key group merges. A member's type is a function of this, so a + /// group that gains or loses a writer explains a drifting type directly. + contribution_groups: BTreeMap, + /// Where each member is currently homed, under `DET_PROVENANCE`. Class + /// member lists and contribution groups are both keyed off this. + member_owners: BTreeMap, members: BTreeSet, net_flows: Vec, inferred_params: BTreeMap, @@ -365,10 +714,58 @@ fn collect_index(analysis: &EmmyLuaAnalysis, label: &str) -> IndexSnapshot { let db = analysis.compilation.get_db(); let type_index = db.get_type_index(); let mut type_caches = BTreeMap::new(); + let mut type_facts: BTreeMap = BTreeMap::new(); + // `DET_PROVENANCE=1` records how each cached type was reached, not just what + // it is. Drift entries then carry the pass that produced them on both sides, + // so the whole set can be grouped by cause in one run rather than traced one + // at a time. + let with_provenance = std::env::var_os("DET_PROVENANCE").is_some(); for (owner, cache) in type_index.iter_type_caches() { + let value = format!("{:?}", cache.as_type()); + if with_provenance { + let fact = type_index.get_type_fact(owner); + let (confidence, base, steps) = match &fact { + Some(fact) => ( + format!("{:?}", fact.confidence()), + format!("{:?}", fact.base_provenance_kind()), + fact.provenance() + .iter() + .map(|step| format!("{:?}", step.event.kind)) + .collect::>() + .join("+"), + ), + None => ("-".into(), "-".into(), String::new()), + }; + let doc = if cache.is_doc() { "doc" } else { "infer" }; + // How many writers this member's value is merged from, and whether + // any of them contributed an unsettled type. If drift tracks these + // rather than the individual site, the cause is the merge, not the + // sites. + let writers = match owner { + glua_code_analysis::LuaTypeOwner::Member(member_id) => { + let member_index = db.get_member_index(); + member_index + .get_member(member_id) + .zip(member_index.get_member_owner(member_id)) + .and_then(|(member, member_owner)| { + member_index + .member_assignment_contributions() + .contributions(&(member_owner.clone(), member.get_key().clone())) + .map(|group| group.len()) + }) + .map(|len| format!("w{len}")) + .unwrap_or_else(|| "w-".into()) + } + _ => "w-".into(), + }; + type_facts.insert( + format!("{}|{owner:?}", file_label(analysis, owner.get_file_id())), + format!("{doc}/{confidence}/{base}/[{steps}]/{writers}"), + ); + } type_caches.insert( format!("{}|{owner:?}", file_label(analysis, owner.get_file_id())), - format!("{:?}", cache.as_type()), + value, ); } @@ -481,8 +878,60 @@ fn collect_index(analysis: &EmmyLuaAnalysis, label: &str) -> IndexSnapshot { members.len(), net_flows.len() ); + // Where every member ended up. Contribution groups and class member lists + // are both keyed off this, so a member homed differently explains drift in + // either without having to trace them apart. + let mut member_owners = BTreeMap::new(); + if with_provenance { + for file_id in db.get_vfs().get_all_file_ids() { + for member in member_index.get_file_members(file_id) { + member_owners.insert( + format!( + "{}:{:?}|{:?}", + file_id.id, + u32::from(member.get_id().get_position()), + member.get_key() + ), + format!("{:?}", member_index.get_current_owner(&member.get_id())), + ); + } + } + } + + // Every assignment-contribution group, so a writer that landed under the + // wrong owner is visible as a group that moved rather than as a member + // whose type merely changed. + let mut contribution_groups = BTreeMap::new(); + if with_provenance { + let all_files = db.get_vfs().get_all_file_ids().into_iter().collect(); + let store = member_index.member_assignment_contributions(); + for group_key in store.keys_for_files(&all_files) { + let Some(group) = store.contributions(&group_key) else { + continue; + }; + let mut ids = group + .keys() + .map(|member_id| { + format!( + "{}:{:?}", + member_id.file_id.id, + u32::from(member_id.get_position()) + ) + }) + .collect::>(); + ids.sort(); + contribution_groups.insert( + format!("{:?}|{:?}", group_key.0, group_key.1), + ids.join(","), + ); + } + } + let snapshot = IndexSnapshot { type_caches, + type_facts, + contribution_groups, + member_owners, members, net_flows, inferred_params, @@ -590,6 +1039,19 @@ fn diff_index(base_label: &str, base: &IndexSnapshot, label: &str, other: &Index for (name, prefix, base_map, other_map) in [ ("type_caches", "TC", &base.type_caches, &other.type_caches), + ("type_facts", "TF", &base.type_facts, &other.type_facts), + ( + "contribution_groups", + "CG", + &base.contribution_groups, + &other.contribution_groups, + ), + ( + "member_owners", + "MO", + &base.member_owners, + &other.member_owners, + ), ( "super_types", "SUPER", @@ -712,6 +1174,72 @@ fn diff(base_label: &str, base: &BTreeSet, label: &str, other: &BTreeS } } +/// Re-index the targets without touching their text and require the index to +/// come back byte-identical. +/// +/// The diagnostic gates cannot see this: re-analysing a file can attach members +/// or settle a decl's type differently from the cold build and still produce the +/// same diagnostics, so `repeat` and `noopedit` both report IDENTICAL while the +/// index underneath has drifted. That drift is what makes incremental work +/// impossible to skip — every "did this actually change?" test reports yes — so +/// it needs a gate of its own. +/// +/// Like `editrevert` it builds its own analysis, for the same reason: sharing +/// one with the other index gate lets whichever runs second measure against an +/// already-converged index and report a clean 0. +fn run_index_repeat(codebase: &Path, annotations: &Path, targets: &[String]) { + let analysis = &mut build_analysis(codebase, annotations); + for target in targets { + let path = codebase.join(target.replace('/', std::path::MAIN_SEPARATOR_STR)); + let Some(uri) = glua_code_analysis::file_path_to_uri(&path) else { + eprintln!("[indexrepeat] cannot build uri for {}", path.display()); + continue; + }; + let Some(file_id) = analysis.get_file_id(&uri) else { + eprintln!("[indexrepeat] file not indexed: {}", path.display()); + continue; + }; + + let expanded = analysis.expand_reindex_file_ids(vec![file_id]); + eprintln!( + "[indexrepeat] {} re-indexes {} files with no text change", + target, + expanded.len() + ); + if std::env::var_os("DET_DUMP_EXPANSION").is_some() { + for id in &expanded { + if let Some(path) = analysis.compilation.get_db().get_vfs().get_file_path(id) { + eprintln!("[expansion] {}", path.display()); + } + } + } + + // Re-indexing the same unchanged file again asks whether the index is + // converging on a fixed point or just oscillating. Round 1 measures the + // cold build against a re-index; every later round measures a re-index + // against the one before it, so a shrinking count means the cold build + // had simply not settled, while a steady one means each pass invents a + // fresh answer. + let rounds = std::env::var("DET_INDEXREPEAT_ROUNDS") + .ok() + .and_then(|raw| raw.parse::().ok()) + .unwrap_or(1); + let mut before = collect_index(analysis, "before_indexrepeat"); + for round in 1..=rounds { + analysis.reindex_files(vec![file_id]); + let label = format!("after_indexrepeat[{target}]#{round}"); + let after = collect_index(analysis, &label); + diff_index( + &format!("before_indexrepeat#{round}"), + &before, + &label, + &after, + ); + before = after; + } + } +} + /// Applies a semantically-neutral edit pair and lets the analysis settle. /// /// `at_front` decides which kind: appending a trailing newline is a semantic @@ -990,6 +1518,76 @@ fn expand_why(analysis: &EmmyLuaAnalysis, codebase: &Path, targets: &[String]) { } } +/// Edit a file through the single-file update path, then put it back. +/// +/// The source ends up exactly as it started, so the index has to as well. Any +/// difference is drift the edit path introduced rather than a fact about the +/// code: state the edit added and the revert did not take away, or state the +/// edit dropped and the revert did not restore. +/// +/// This is the update-path counterpart of `indexrepeat`. That stage re-indexes +/// with the text untouched, so it never exercises the invalidation an edit +/// triggers; this one does, and unlike `realedit` it needs no ground-truth +/// build, because the pre-edit index *is* the truth. `noopedit` does not cover +/// it either: its edit pair is semantically neutral, so the update path skips +/// the re-index outright and nothing is invalidated. +/// +/// It builds its own analysis rather than sharing the caller's. Both index +/// gates re-index in place and leave a converged index behind, so whichever ran +/// second would measure drift against the other's converged state instead of +/// against a cold build and report a clean 0 — the drift does not go away, it +/// stops being visible. +fn edit_revert(codebase: &Path, annotations: &Path, targets: &[String]) { + let Ok(find) = std::env::var("DET_EDIT_FIND") else { + eprintln!("[editrevert] SKIPPED: DET_EDIT_FIND is not set, so no drift gate ran"); + return; + }; + let replace = std::env::var("DET_EDIT_REPLACE").unwrap_or_default(); + let analysis = &mut build_analysis(codebase, annotations); + + for target in targets { + let path = codebase.join(target.replace('/', std::path::MAIN_SEPARATOR_STR)); + let Some(uri) = glua_code_analysis::file_path_to_uri(&path) else { + continue; + }; + let Some(file_id) = analysis.get_file_id(&uri) else { + eprintln!("[editrevert] file not indexed: {}", path.display()); + continue; + }; + let Some(original) = analysis + .compilation + .get_db() + .get_vfs() + .get_file_content(&file_id) + .cloned() + else { + continue; + }; + if !original.contains(find.as_str()) { + eprintln!("[editrevert] {find:?} not present in {}", path.display()); + continue; + } + let edited = original.replace(find.as_str(), replace.as_str()); + + let before_index = collect_index(analysis, "before_editrevert"); + let before = collect(analysis, "before_editrevert"); + + let t = Instant::now(); + analysis.update_file_by_uri(&uri, Some(edited)); + analysis.update_file_by_uri(&uri, Some(original)); + eprintln!( + "[editrevert] {target} edited and reverted ({:.2}s)", + t.elapsed().as_secs_f64() + ); + + let label = format!("after_editrevert[{target}]"); + let after_index = collect_index(analysis, &label); + let after = collect(analysis, &label); + diff_index("before_editrevert", &before_index, &label, &after_index); + diff("before_editrevert", &before, &label, &after); + } +} + /// Applies a **real** edit and compares the incremental result against a cold /// build of the same edited source. /// @@ -1074,6 +1672,173 @@ fn real_edit( } } +/// Gate the deferred ripple: several self-indexes, then one re-index over the +/// union of their separately-captured expansions. +/// +/// The LSP debounce runs the edited file's own re-index on a short timer and +/// owes the dependency ripple afterwards. Deferring that ripple behind a longer +/// idle timer means a typing burst produces several self-indexes before one +/// ripple, so the ripple has to run against a *union* of expansions each +/// captured at a different point. +/// +/// That union is the whole risk. An expansion recomputed after a self-index is +/// known to under-expand badly — 739 files collapsed to 8 — which is why the +/// production path captures before self-indexing. Union survives that, because +/// a collapsed later capture only ever adds files and the burst's first capture +/// is taken before any self-index, exactly as today. What it cannot rule out by +/// argument is a dependent present in *no* capture, and that is what this +/// stage measures: the burst's result against a cold build of the final text. +fn burst_edit( + analysis: &mut EmmyLuaAnalysis, + codebase: &Path, + annotations: &Path, + targets: &[String], + cold: &BTreeSet, +) { + let Ok(find) = std::env::var("DET_EDIT_FIND") else { + eprintln!("[burst] SKIPPED: DET_EDIT_FIND is not set, so no burst gate ran"); + return; + }; + let replace = std::env::var("DET_EDIT_REPLACE").unwrap_or_default(); + let cold_index = collect_index(analysis, "cold"); + + struct Target { + path: std::path::PathBuf, + uri: lsp_types::Uri, + file_id: FileId, + original: String, + } + + let mut resolved = Vec::new(); + for target in targets { + let path = codebase.join(target.replace('/', std::path::MAIN_SEPARATOR_STR)); + let Some(uri) = glua_code_analysis::file_path_to_uri(&path) else { + continue; + }; + let Some(file_id) = analysis.get_file_id(&uri) else { + eprintln!("[burst] file not indexed: {}", path.display()); + continue; + }; + let Some(original) = analysis + .compilation + .get_db() + .get_vfs() + .get_file_content(&file_id) + .cloned() + else { + continue; + }; + if !original.contains(find.as_str()) { + eprintln!("[burst] {find:?} not present in {}", path.display()); + continue; + } + resolved.push(Target { + path, + uri, + file_id, + original, + }); + } + + if resolved.is_empty() { + eprintln!("[burst] SKIPPED: no target matched, so no burst gate ran"); + return; + } + + // Three keystroke groups, chosen to cover what a burst can do that a single + // edit cannot: change meaning, shift offsets only, and introduce a class + // definition partway through — the case where reusing the first capture + // would miss the new dependents outright. + let step_text = |target: &Target, step: usize| -> String { + let mut text = target.original.replace(find.as_str(), replace.as_str()); + if step >= 1 { + text.push_str("\n-- burst\n"); + } + if step >= 2 { + text.push_str("\n---@class DetBurstClass\nlocal DetBurst = {}\n"); + } + text + }; + + let mut owed_files: BTreeSet = BTreeSet::new(); + let mut owed_expansion: BTreeSet = BTreeSet::new(); + + let t = Instant::now(); + for step in 0..3 { + for target in &resolved { + // Production order: didChange installs the text, then phase A + // captures the expansion and self-indexes under one lock. + analysis.update_file_text_only(&target.uri, step_text(target, step)); + let expansion = analysis.expand_reindex_file_ids(vec![target.file_id]); + analysis.self_index_files(vec![target.file_id]); + owed_files.insert(target.file_id); + owed_expansion.extend(expansion); + } + } + + let self_indexed = t.elapsed(); + let t = Instant::now(); + analysis.reindex_expanded_files( + owed_files.iter().copied().collect(), + owed_expansion.iter().copied().collect(), + ); + eprintln!( + "[burst] {} file(s) x 3 edits: {} self-index(es) in {:.2}s, then one ripple over {} files ({:.2}s)", + resolved.len(), + resolved.len() * 3, + self_indexed.as_secs_f64(), + owed_expansion.len(), + t.elapsed().as_secs_f64() + ); + + let warm_index = collect_index(analysis, "warm"); + let warm = collect(analysis, "warm"); + + // The control: the same three edits through today's path, one ripple each. + // Deferral is only a regression if it drifts further than this does — the + // incremental path already drifts from cold on its own, so comparing the + // burst against cold alone would charge it for drift it did not cause. + for target in &resolved { + analysis.update_file_by_uri(&target.uri, Some(target.original.clone())); + } + let t = Instant::now(); + for step in 0..3 { + for target in &resolved { + analysis.update_file_by_uri(&target.uri, Some(step_text(target, step))); + } + } + eprintln!( + "[burst] control: same edits through {} separate ripples ({:.2}s)", + resolved.len() * 3, + t.elapsed().as_secs_f64() + ); + let control_index = collect_index(analysis, "control"); + let control = collect(analysis, "control"); + + let overrides = resolved + .iter() + .map(|target| (target.path.clone(), step_text(target, 2))) + .collect::>(); + let ground_truth = build_analysis_with(codebase, annotations, Order::Natural, 1, &overrides); + let truth_index = collect_index(&ground_truth, "cold_burst"); + let truth = collect(&ground_truth, "cold_burst"); + + // What the burst genuinely moves, so a reader can tell a real miss from a + // change the edits were always going to make. + diff_index("cold", &cold_index, "cold_burst", &truth_index); + diff("cold", cold, "cold_burst", &truth); + // What today's path already gets wrong about it. + diff_index("cold_burst", &truth_index, "control", &control_index); + diff("cold_burst", &truth, "control", &control); + // The gate: deferring the ripple must not get more wrong than the control. + diff_index("cold_burst", &truth_index, "warm", &warm_index); + diff("cold_burst", &truth, "warm", &warm); + + for target in &resolved { + analysis.update_file_by_uri(&target.uri, Some(target.original.clone())); + } +} + fn reindex_exact(analysis: &mut EmmyLuaAnalysis, codebase: &Path, relatives: &[String]) -> bool { let mut file_ids = Vec::new(); for relative in relatives { @@ -1100,14 +1865,27 @@ fn reindex_exact(analysis: &mut EmmyLuaAnalysis, codebase: &Path, relatives: &[S true } +/// Analysis recurses over deeply nested syntax. The server does that work on +/// spawned threads, which get a far larger stack than a process main thread does +/// on Windows, so the tools have to ask for one explicitly. fn main() { + std::thread::Builder::new() + .stack_size(256 * 1024 * 1024) + .spawn(run) + .expect("determinism worker thread should spawn") + .join() + .expect("determinism worker thread should not panic"); +} + +fn run() { + alloc_sample::init(); let codebase = PathBuf::from(std::env::var("DET_CODEBASE").expect("DET_CODEBASE env var is required")); let annotations = PathBuf::from( std::env::var("DET_ANNOTATIONS").expect("DET_ANNOTATIONS env var is required"), ); let stages = std::env::var("DET_STAGES") - .unwrap_or_else(|_| "repeat,noopedit,realedit".to_string()) + .unwrap_or_else(|_| "repeat,noopedit,editrevert,realedit,indexrepeat".to_string()) .split(',') .map(|s| s.trim().to_string()) .filter(|s| !s.is_empty()) @@ -1268,6 +2046,43 @@ fn main() { } } + // Remove and re-add each file ON ITS OWN, so every file is re-derived + // against the complete settled workspace rather than against the prefix the + // cold walk had built when it reached that file. This is the "fully + // informed" fixed point: if the index converges here, the cold answer is + // simply one computed from an incomplete view, and a subset re-index — which + // also sees a settled workspace — is agreeing with the informed answer + // rather than drifting. + if stages.iter().any(|s| s == "perfile") { + let all_ids = analysis.compilation.get_db().get_vfs().get_all_file_ids(); + let rounds = std::env::var("DET_PERFILE_ROUNDS") + .ok() + .and_then(|raw| raw.parse::().ok()) + .unwrap_or(2); + let mut previous_index = collect_index(&analysis, "cold"); + let mut previous_diagnostics = cold.clone(); + let mut previous_label = "cold".to_string(); + for round in 1..=rounds { + let t = Instant::now(); + for file_id in &all_ids { + analysis.reindex_files_without_expansion(vec![*file_id]); + } + let label = format!("after_perfile_{round}"); + eprintln!( + "[perfile] round {round} over {} files ({:.2}s)", + all_ids.len(), + t.elapsed().as_secs_f64() + ); + let after_index = collect_index(&analysis, &label); + diff_index(&previous_label, &previous_index, &label, &after_index); + let after = collect(&analysis, &label); + diff(&previous_label, &previous_diagnostics, &label, &after); + previous_index = after_index; + previous_diagnostics = after; + previous_label = label; + } + } + // The production incremental path: `reindex_files` runs the same re-analysis // as `mainreindex` but first widens the set through `expand_reindex_file_ids`. // Comparing the two says whether the expansion is what closes the gap, i.e. @@ -1305,10 +2120,22 @@ fn main() { refresh_faithfulness(&analysis); } + if stages.iter().any(|s| s == "editrevert") { + edit_revert(&codebase, &annotations, &targets); + } + if stages.iter().any(|s| s == "realedit") { real_edit(&mut analysis, &codebase, &annotations, &targets, &cold); } + if stages.iter().any(|s| s == "burst") { + burst_edit(&mut analysis, &codebase, &annotations, &targets, &cold); + } + + if stages.iter().any(|s| s == "indexrepeat") { + run_index_repeat(&codebase, &annotations, &targets); + } + if stages.iter().any(|s| s == "fresh") { let fresh_analysis = build_analysis(&codebase, &annotations); let fresh = collect(&fresh_analysis, "fresh_process"); @@ -1334,4 +2161,11 @@ fn main() { let split = collect(&split_analysis, "split_batches"); diff("cold", &cold, "split_batches", &split); } + + alloc_sample::report( + std::env::var("DET_ALLOC_TOP") + .ok() + .and_then(|v| v.parse().ok()) + .unwrap_or(30), + ); } diff --git a/tools/lsp_latency.js b/tools/lsp_latency.js new file mode 100644 index 000000000..ff969cbb0 --- /dev/null +++ b/tools/lsp_latency.js @@ -0,0 +1,614 @@ +// Interactive latency harness for the language server. +// +// Drives a real `glua_ls` binary over stdio with the capabilities and the +// cancellation behaviour vscode-languageclient actually uses, then reports how +// long the requests a user waits on take. Unit tests cannot catch what this +// measures: the cost of a request is dominated by whether a reindex is pending, +// which only shows up against a real workspace. +// +// Usage: +// LSP_CODEBASE=/path/to/workspace \ +// LSP_ANNOTATIONS=/path/to/annotations/output \ +// node tools/lsp_latency.js [--json] [--runs N] [--file relative/path.lua] +// [--edit code|comment] +// [--edit-find TEXT --edit-replace TEXT] +// [--completion-find TEXT] +// +// --edit-find/--edit-replace swap one snippet back and forth on every edit, so +// a specific real change to real code can be reproduced. --completion-find puts +// the cursor immediately after a given snippet instead of at the first member +// access in the file. Together they reproduce one user's exact complaint. +// +// --edit code (default) inserts a global function declaration, so each edit +// really changes what the file exports and the full dependency ripple runs. +// --edit comment inserts a comment line instead: cheaper, and useful for +// isolating offset-shift cost, but an optimisation that only helps this case +// has not made real typing any faster. +// +// LSP_SERVER overrides the binary (default: target/dist/glua_ls[.exe] if built, +// else target/release/glua_ls[.exe] — see defaultServerPath, and prefer `dist`). +// LSP_SERVER_ARGS passes extra space-separated arguments to the server, e.g. +// LSP_SERVER_ARGS='--log-level debug' to profile a slow path. +// --file defaults to the largest .lua file in the workspace, which is the +// pessimistic case and keeps runs comparable without naming a file per repo. +// +// Exits non-zero on a correctness check, not on a latency number: a cancelled +// diagnostic pull answered with an empty full report, or a mid-edit completion +// that disagrees with the settled one. +'use strict'; + +const { spawn } = require('child_process'); +const fs = require('fs'); +const path = require('path'); + +// LSP ContentModified. The client is expected to re-send rather than wait. +const CONTENT_MODIFIED = -32801; + +// ---------------------------------------------------------------- config --- + +function parseArgs(argv) { + const opts = { + json: false, runs: 5, file: null, edit: 'code', + editFind: null, editReplace: null, completionFind: null, + }; + for (let i = 2; i < argv.length; i++) { + const a = argv[i]; + if (a === '--json') opts.json = true; + else if (a === '--runs') opts.runs = Number(argv[++i]); + else if (a === '--file') opts.file = argv[++i]; + else if (a === '--edit') opts.edit = argv[++i]; + else if (a === '--edit-find') opts.editFind = argv[++i]; + else if (a === '--edit-replace') opts.editReplace = argv[++i]; + else if (a === '--completion-find') opts.completionFind = argv[++i]; + else throw new Error(`unknown argument: ${a}`); + } + if (opts.edit !== 'code' && opts.edit !== 'comment') { + throw new Error('--edit must be "code" or "comment"'); + } + if ((opts.editFind === null) !== (opts.editReplace === null)) { + throw new Error('--edit-find and --edit-replace must be given together'); + } + if (opts.editFind === '') throw new Error('--edit-find must not be empty'); + if (!Number.isFinite(opts.runs) || opts.runs < 1) { + throw new Error('--runs must be a positive integer'); + } + return opts; +} + +/** + * Prefers the `dist` profile, which is what ships. `release` lacks its thin LTO + * and single codegen unit, so measuring it reports numbers no user experiences — + * an easy mistake to make for a whole session before noticing. + */ +function defaultServerPath() { + const exe = process.platform === 'win32' ? 'glua_ls.exe' : 'glua_ls'; + const dist = path.resolve(__dirname, '..', 'target', 'dist', exe); + if (fs.existsSync(dist)) return dist; + return path.resolve(__dirname, '..', 'target', 'release', exe); +} + +function requireDir(value, name) { + if (!value) throw new Error(`${name} is required`); + if (!fs.existsSync(value)) throw new Error(`${name} does not exist: ${value}`); + return path.resolve(value); +} + +function largestLuaFile(root) { + let best = null; + const walk = (dir) => { + let entries; + try { entries = fs.readdirSync(dir, { withFileTypes: true }); } catch { return; } + for (const e of entries) { + if (e.name === '.git' || e.name === 'node_modules') continue; + const p = path.join(dir, e.name); + if (e.isDirectory()) walk(p); + else if (e.name.endsWith('.lua')) { + const size = fs.statSync(p).size; + if (!best || size > best.size) best = { path: p, size }; + } + } + }; + walk(root); + if (!best) throw new Error(`no .lua files found under ${root}`); + return best.path; +} + +function fileUri(p) { + const resolved = path.resolve(p).replace(/\\/g, '/'); + const withSlash = resolved.startsWith('/') ? resolved : `/${resolved}`; + return `file://${encodeURI(withSlash).replace(/#/g, '%23').replace(/\?/g, '%3F')}`; +} + +// ------------------------------------------------------------ lsp client --- + +class LspClient { + constructor(proc) { + this.proc = proc; + this.buffer = Buffer.alloc(0); + this.nextId = 1; + this.pending = new Map(); + this.onNotification = null; + this.serverRequests = new Set(); + proc.stdout.on('data', (chunk) => this._receive(chunk)); + // The server's phase profiler writes to stderr, so LSP_SERVER_STDERR + // makes those numbers reachable instead of dropping them on the floor. + const stderrPath = process.env.LSP_SERVER_STDERR; + if (stderrPath) fs.writeFileSync(stderrPath, ''); + proc.stderr.on('data', (chunk) => { + if (stderrPath) fs.appendFileSync(stderrPath, chunk); + }); + } + + _receive(chunk) { + this.buffer = Buffer.concat([this.buffer, chunk]); + for (;;) { + const headerEnd = this.buffer.indexOf('\r\n\r\n'); + if (headerEnd < 0) return; + const header = this.buffer.slice(0, headerEnd).toString('ascii'); + const match = /Content-Length: (\d+)/i.exec(header); + if (!match) return; + const length = Number(match[1]); + const bodyStart = headerEnd + 4; + if (this.buffer.length < bodyStart + length) return; + const body = this.buffer.slice(bodyStart, bodyStart + length).toString('utf8'); + this.buffer = this.buffer.slice(bodyStart + length); + try { this._dispatch(JSON.parse(body)); } catch { /* ignore malformed frame */ } + } + } + + _dispatch(message) { + if (message.id !== undefined && message.method) { + // Server-initiated request. Record it and answer so the server is + // never left waiting on us. `workspace/configuration` has to come + // back as one entry per requested item; null is not a valid result + // and would have the server fall back to something a real client + // never makes it use. + this.serverRequests.add(message.method); + const result = message.method === 'workspace/configuration' + ? ((message.params && message.params.items) || []).map(() => ({})) + : null; + this._write({ jsonrpc: '2.0', id: message.id, result }); + return; + } + if (message.id !== undefined) { + const resolve = this.pending.get(message.id); + if (resolve) { this.pending.delete(message.id); resolve(message); } + return; + } + if (this.onNotification) this.onNotification(message); + } + + _write(payload) { + const body = Buffer.from(JSON.stringify(payload), 'utf8'); + this.proc.stdin.write(`Content-Length: ${body.length}\r\n\r\n`); + this.proc.stdin.write(body); + } + + notify(method, params) { + this._write({ jsonrpc: '2.0', method, params }); + } + + /** Returns a promise carrying the response, the elapsed ms, and its id. */ + request(method, params) { + const id = this.nextId++; + const startedAt = Date.now(); + const promise = new Promise((resolve) => { + this.pending.set(id, (message) => + resolve({ message, ms: Date.now() - startedAt, id })); + }); + this._write({ jsonrpc: '2.0', id, method, params }); + return Object.assign(promise, { id }); + } + + cancel(id) { + this.notify('$/cancelRequest', { id }); + } +} + +const sleep = (ms) => new Promise((resolve) => setTimeout(resolve, ms)); + +/** + * The subset of VS Code's capabilities that changes server behaviour on the + * paths this tool measures. Keep in step with vscode-languageclient: the point + * is to exercise what the real client exercises. + */ +function clientCapabilities() { + return { + general: { + staleRequestSupport: { + cancel: true, + retryOnContentModified: [ + 'textDocument/semanticTokens/full', + 'textDocument/semanticTokens/range', + 'textDocument/semanticTokens/full/delta', + ], + }, + }, + window: { workDoneProgress: true }, + workspace: { + applyEdit: true, + configuration: true, + workspaceFolders: true, + diagnostics: { refreshSupport: true }, + semanticTokens: { refreshSupport: true }, + inlayHint: { refreshSupport: true }, + codeLens: { refreshSupport: true }, + didChangeWatchedFiles: { dynamicRegistration: true }, + }, + textDocument: { + synchronization: { didSave: true }, + diagnostic: { dynamicRegistration: true, relatedDocumentSupport: true }, + completion: { completionItem: { tagSupport: { valueSet: [1] } } }, + hover: {}, + definition: {}, + semanticTokens: { + requests: { full: true }, + tokenTypes: [], tokenModifiers: [], formats: ['relative'], + }, + }, + }; +} + +// ------------------------------------------------------------- reporting --- + +function summarise(samples) { + if (samples.length === 0) return null; + const sorted = [...samples].sort((a, b) => a - b); + const at = (q) => sorted[Math.min(sorted.length - 1, Math.floor(q * sorted.length))]; + return { runs: sorted.length, min: sorted[0], median: at(0.5), max: sorted[sorted.length - 1] }; +} + +function describeReport(result) { + if (!result) return { kind: 'none' }; + if (result.kind === 'unchanged') return { kind: 'unchanged', resultId: result.resultId }; + return { kind: 'full', count: (result.items || []).length, resultId: result.resultId }; +} + +// ------------------------------------------------------------- scenarios --- + +/** + * Finds a position just after a `.` on a member access, which is the case that + * matters most: it forces the server to resolve a receiver type through the + * index rather than listing globals. + * + * `self.` is preferred because resolving it exercises the class/type indexes + * rather than a module table, which is where a stale index shows up first. + */ +function memberAccessPosition(text) { + const lines = text.split('\n'); + const find = (pattern) => { + for (let line = 0; line < lines.length; line++) { + const match = pattern.exec(lines[line]); + if (match) { + return { line, character: match.index + match[0].indexOf('.') + 1 }; + } + } + return null; + }; + return find(/\bself\.[A-Za-z_]/) + || find(/[A-Za-z_][A-Za-z0-9_]*\.[A-Za-z_]/) + || { line: 0, character: 0 }; +} + +async function main() { + const opts = parseArgs(process.argv); + const codebase = requireDir(process.env.LSP_CODEBASE, 'LSP_CODEBASE'); + const annotations = requireDir(process.env.LSP_ANNOTATIONS, 'LSP_ANNOTATIONS'); + const server = process.env.LSP_SERVER || defaultServerPath(); + if (!fs.existsSync(server)) { + throw new Error(`server binary not found: ${server}\nBuild it with: cargo build -p glua_ls --release`); + } + const target = opts.file ? path.resolve(codebase, opts.file) : largestLuaFile(codebase); + if (!fs.existsSync(target)) throw new Error(`target file not found: ${target}`); + + const extraArgs = (process.env.LSP_SERVER_ARGS || '').split(' ').filter(Boolean); + const proc = spawn(server, [ + '--communication', 'stdio', + '--gmod-annotations-path', annotations, + ...extraArgs, + ], { stdio: ['pipe', 'pipe', 'pipe'] }); + + const client = new LspClient(proc); + const report = { + workspace: codebase, + file: path.relative(codebase, target), + server, + runs: opts.runs, + measurements: {}, + checks: {}, + }; + + let workspaceLoadedAt = null; + client.onNotification = (message) => { + if (message.method === 'gluals/serverStatus' + && message.params && message.params.state === 'workspaceLoaded') { + workspaceLoadedAt = Date.now(); + } + }; + + const startedAt = Date.now(); + await client.request('initialize', { + processId: process.pid, + rootUri: fileUri(codebase), + workspaceFolders: [{ uri: fileUri(codebase), name: path.basename(codebase) }], + capabilities: clientCapabilities(), + initializationOptions: {}, + clientInfo: { name: 'Visual Studio Code', version: '1.95.0' }, + }); + client.notify('initialized', {}); + + const loadDeadline = Date.now() + 300000; + while (!workspaceLoadedAt && Date.now() < loadDeadline) await sleep(100); + if (!workspaceLoadedAt) { + proc.kill(); + throw new Error('workspace never finished loading (5 minute timeout)'); + } + report.measurements.workspaceLoad = { runs: 1, min: workspaceLoadedAt - startedAt, median: workspaceLoadedAt - startedAt, max: workspaceLoadedAt - startedAt }; + + const uri = fileUri(target); + const original = fs.readFileSync(target, 'utf8'); + let text = original; + let version = 1; + client.notify('textDocument/didOpen', { + textDocument: { uri, languageId: 'lua', version, text }, + }); + await sleep(1500); + + // `--completion-find` puts the cursor immediately after a given snippet, so + // a specific completion can be reproduced instead of whatever member access + // happens to come first in the file. + const completionPosition = (currentText) => { + if (opts.completionFind === null) return memberAccessPosition(currentText); + const index = currentText.indexOf(opts.completionFind); + if (index < 0) { + throw new Error(`--completion-find text not present in document: ${JSON.stringify(opts.completionFind)}`); + } + const lines = currentText.slice(0, index + opts.completionFind.length).split('\n'); + return { line: lines.length - 1, character: lines[lines.length - 1].length }; + }; + + const editOffset = text.indexOf('\n') + 1; + const settledCompletion = []; + const typingCompletion = []; + const settledDiagnostic = []; + const typingHover = []; + const typingCompletionConcurrent = []; + const highlightAfterEdit = []; + const highlightRetries = []; + const editToFresh = []; + const cancelledPulls = []; + const completionDrift = []; + let previousResultId; + + const labelsOf = (items) => + (Array.isArray(items) ? items : []).map((item) => item.label); + + // Recomputed per call: every edit inserts a line, so a position captured + // once would drift and silently start measuring an empty completion. + const completionAt = async () => client.request('textDocument/completion', { + textDocument: { uri }, + position: completionPosition(text), + context: { triggerKind: 2, triggerCharacter: '.' }, + }); + + let editSerial = 0; + const editDocument = () => { + version += 1; + editSerial += 1; + + // `--edit-find`/`--edit-replace` reproduce one specific edit, swapping + // back and forth so every keystroke is a real change to real code. Use + // it to measure the edit a user actually complained about. + if (opts.editFind !== null) { + const [from, to] = editSerial % 2 === 1 + ? [opts.editFind, opts.editReplace] + : [opts.editReplace, opts.editFind]; + if (!text.includes(from)) { + throw new Error(`--edit-find text not present in document: ${JSON.stringify(from)}`); + } + text = text.replace(from, to); + client.notify('textDocument/didChange', { + textDocument: { uri, version }, + contentChanges: [{ text }], + }); + return; + } + + // `--edit comment` keeps the edit syntactically inert, which makes runs + // comparable but is also the case an early-cutoff optimisation can make + // fast without helping anyone. `--edit code` (the default) declares a + // global function instead, so the edit really does change what the file + // exports and the whole dependency ripple has to run. + const inserted = opts.edit === 'comment' + ? '-- perf\n' + : `function _PerfProbe${editSerial}(a) return a end\n`; + text = text.slice(0, editOffset) + inserted + text.slice(editOffset); + client.notify('textDocument/didChange', { + textDocument: { uri, version }, + contentChanges: [{ text }], + }); + }; + + // A settled measurement is only meaningful once nothing is pending. An + // uncancelled diagnostic pull returns exactly when the analysis is fresh, + // so it is the cheapest way to wait for quiescence without guessing. + const waitUntilQuiet = async () => { + await client.request('textDocument/diagnostic', { + textDocument: { uri }, previousResultId, + }); + }; + + for (let run = 0; run < opts.runs; run++) { + await waitUntilQuiet(); + + // Settled: no pending edit, so this is the pure compute cost. + const settled = await completionAt(); + settledCompletion.push(settled.ms); + const items = settled.message.result + ? (settled.message.result.items || settled.message.result) + : []; + report.checks.completionItemCount = Array.isArray(items) ? items.length : 0; + + const diagnostic = await client.request('textDocument/diagnostic', { + textDocument: { uri }, previousResultId, + }); + settledDiagnostic.push(diagnostic.ms); + const described = describeReport(diagnostic.message.result); + if (described.resultId) previousResultId = described.resultId; + report.checks.diagnosticCount = described.count ?? report.checks.diagnosticCount; + + // While typing: the request the user actually waits on. Its result is + // compared against the settled one, because the whole risk of answering + // before a reindex finishes is answering *differently* — a thinner or + // wrong list is the failure mode, not a slow one. + editDocument(); + const typing = await completionAt(); + typingCompletion.push(typing.ms); + const typingItems = typing.message.result + ? (typing.message.result.items || typing.message.result) + : []; + const settledLabels = new Set(labelsOf(items)); + const typingLabels = new Set(labelsOf(typingItems)); + const missing = [...settledLabels].filter((l) => !typingLabels.has(l)); + const extra = [...typingLabels].filter((l) => !settledLabels.has(l)); + completionDrift.push({ missing: missing.length, extra: extra.length, + sampleMissing: missing.slice(0, 5) }); + + // Completion is not the only thing gated on freshness. Hover is issued + // from the same keystroke and measured separately, so a fix that makes + // completion answer early but leaves every other position-based feature + // parked shows up here instead of looking like a win. + editDocument(); + const [hovered, completed] = await Promise.all([ + client.request('textDocument/hover', { + textDocument: { uri }, position: completionPosition(text), + }), + completionAt(), + ]); + typingHover.push(hovered.ms); + typingCompletionConcurrent.push(completed.ms); + + // Syntax highlighting. The client retries on ContentModified rather than + // waiting, so the number the user feels is the time until a real token + // set arrives, not the latency of any single request. + editDocument(); + const highlightStarted = Date.now(); + let highlightRetried = 0; + for (;;) { + const tokens = await client.request('textDocument/semanticTokens/full', { + textDocument: { uri }, + }); + const code = tokens.message.error && tokens.message.error.code; + if (code !== CONTENT_MODIFIED) break; + highlightRetried++; + await sleep(50); + } + highlightAfterEdit.push(Date.now() - highlightStarted); + highlightRetries.push(highlightRetried); + + // Keystroke to the first answer any index-reading handler can give. + editDocument(); + const fresh = await client.request('textDocument/diagnostic', { + textDocument: { uri }, previousResultId, + }); + editToFresh.push(fresh.ms); + + // A pull cancelled mid-flight must never come back as an empty full + // report — that is what clears the file's diagnostics in VS Code. + editDocument(); + const doomed = client.request('textDocument/diagnostic', { + textDocument: { uri }, previousResultId, + }); + await sleep(20); + client.cancel(doomed.id); + const cancelled = await doomed; + const shape = describeReport(cancelled.message.result); + cancelledPulls.push({ + emptyFullReport: shape.kind === 'full' && shape.count === 0, + errorCode: cancelled.message.error && cancelled.message.error.code, + }); + } + + report.measurements.completionSettled = summarise(settledCompletion); + report.measurements.completionWhileTyping = summarise(typingCompletion); + report.measurements.diagnosticSettled = summarise(settledDiagnostic); + report.measurements.hoverWhileTyping = summarise(typingHover); + report.measurements.completionWhileTypingConcurrent = summarise(typingCompletionConcurrent); + report.measurements.highlightAfterEdit = summarise(highlightAfterEdit); + report.checks.highlightRetries = highlightRetries.reduce((a, b) => a + b, 0); + report.measurements.editToFreshAnswer = summarise(editToFresh); + report.checks.emptyFullReportsOnCancel = + cancelledPulls.filter((p) => p.emptyFullReport).length; + // A mid-edit completion that differs from the settled one is a correctness + // regression, however fast it came back. + report.checks.completionDriftWhileTyping = { + worstMissing: Math.max(0, ...completionDrift.map((d) => d.missing)), + worstExtra: Math.max(0, ...completionDrift.map((d) => d.extra)), + sampleMissing: (completionDrift.find((d) => d.missing > 0) || {}).sampleMissing || [], + }; + report.checks.serverInitiatedRequests = [...client.serverRequests].sort(); + + proc.kill(); + + if (opts.json) { + console.log(JSON.stringify(report, null, 2)); + return failedChecks(report); + } + + const rows = [ + ['workspace load', report.measurements.workspaceLoad], + ['completion (settled)', report.measurements.completionSettled], + ['completion (while typing)', report.measurements.completionWhileTyping], + ['diagnostic (settled)', report.measurements.diagnosticSettled], + ['hover (while typing)', report.measurements.hoverWhileTyping], + ['completion (concurrent)', report.measurements.completionWhileTypingConcurrent], + ['highlight after edit', report.measurements.highlightAfterEdit], + ['edit -> fresh answer', report.measurements.editToFreshAnswer], + ]; + console.log(`workspace : ${report.workspace}`); + console.log(`file : ${report.file}`); + console.log(`server : ${report.server}`); + console.log(`runs : ${report.runs}\n`); + console.log(' min median max'); + for (const [label, stats] of rows) { + if (!stats) continue; + const fmt = (n) => `${n}ms`.padStart(10); + console.log(`${label.padEnd(28)}${fmt(stats.min)}${fmt(stats.median)}${fmt(stats.max)}`); + } + console.log(`\ncompletion items : ${report.checks.completionItemCount}`); + console.log(`diagnostics : ${report.checks.diagnosticCount}`); + console.log(`empty reports on cancel : ${report.checks.emptyFullReportsOnCancel}` + + (report.checks.emptyFullReportsOnCancel === 0 ? ' (good)' : ' (BAD: clears the file)')); + const drift = report.checks.completionDriftWhileTyping; + const driftOk = drift.worstMissing === 0 && drift.worstExtra === 0; + console.log(`completion drift mid-edit : -${drift.worstMissing} / +${drift.worstExtra}` + + (driftOk ? ' (good)' : ` (differs from settled: ${drift.sampleMissing.join(', ')})`)); + + return failedChecks(report); +} + +// The correctness checks, as opposed to the timings. Timings are reported for +// comparison and never fail; these two are defects whatever the latency was. +function failedChecks(report) { + const failures = []; + if (report.checks.emptyFullReportsOnCancel > 0) { + failures.push(`${report.checks.emptyFullReportsOnCancel} cancelled diagnostic pull(s) ` + + 'came back as an empty full report, which clears the file in the editor'); + } + const drift = report.checks.completionDriftWhileTyping; + if (drift.worstMissing > 0 || drift.worstExtra > 0) { + failures.push(`completion mid-edit differed from settled by -${drift.worstMissing}` + + ` / +${drift.worstExtra} items`); + } + return failures; +} + +main().then((failures) => { + if (!failures || failures.length === 0) return; + console.error('\nFAILED:'); + for (const failure of failures) console.error(` - ${failure}`); + process.exit(1); +}).catch((error) => { + console.error(String(error && error.message ? error.message : error)); + process.exit(1); +});