Skip to content

Latest commit

 

History

History
547 lines (432 loc) · 26.6 KB

File metadata and controls

547 lines (432 loc) · 26.6 KB

Builder API Reference

This page is the authoritative reference for constructing and enriching libcpg graphs: the CpgBuilder trait and its tree-sitter implementation, the CFG and DFG extractors, the opt-in call-resolution post-pass, the on-demand PDG builder and program slicers, and the parser registry and node mapper that back the tree-sitter frontend. Every signature is transcribed from src/builder/.

For the shape of the graphs these APIs produce, see the Graph reference; for analysis over them, the Pattern reference. Term definitions live in the Glossary.

The CPG construction pipeline: parse, AST, CFG, optional call resolution, DFG

Figure — the construction pipeline: tree-sitter parse → AST conversion → config-gated CFG extraction → opt-in evidence-bounded call resolution → config-gated DFG extraction. Source: diagrams/construction-pipeline.puml.


Where the builder types live

Re-exported from the crate root:

use libcpg::{
    CpgBuilder, CpgBuilderConfig, TreeSitterCpgBuilder,
    FunctionAnalysisCache, FunctionAnalysisCacheLimits, FunctionAnalysisCacheStats,
    CfgExtractor, CfgExtractorConfig, BasicBlockIdentifier,
    DfgExtractor, DfgExtractorConfig,
    Definition, DefinitionKind, Use, UseKind, DefUseChain, build_def_use_chains,
    PdgBuilder, backward_slice, forward_slice,
};

Two builder types are not re-exported at the root — reach them by full path:

use libcpg::builder::{NodeMapper, ParserRegistry};

CpgBuilder trait

The construction contract. build is the language-parsing entry point; build_file and supports_language are provided.

pub trait CpgBuilder: Send + Sync {
    fn build(&self, source: &str, language: Language) -> Result<CodePropertyGraph>;

    fn build_file(&self, path: &std::path::Path) -> Result<CodePropertyGraph> { /* provided */ }

    fn supported_languages(&self) -> &[Language];

    fn supports_language(&self, language: Language) -> bool { /* provided */ }
}
Method Signature Notes
build fn build(&self, source: &str, language: Language) -> Result<CodePropertyGraph> Parse source and construct the CPG.
build_file fn build_file(&self, path: &Path) -> Result<CodePropertyGraph> Reads the file, infers the language via Language::from_extension, calls build, and records the source path.
supported_languages fn supported_languages(&self) -> &[Language] The languages this builder could handle.
supports_language fn supports_language(&self, language: Language) -> bool supported_languages().contains(&language).

Honesty — default = [] means build fails for every language. Parsing requires a lang-* grammar feature; with no features enabled, build (and build_file) return Error::UnsupportedLanguage because ParserRegistry has no grammar registered. Only build_from_tree (Mode B) and the feature-free hand-built surface work with default = [].

The Result type is libcpg::Result (alias for std::result::Result<T, libcpg::Error>); errors are libcpg::Error — there is no CpgError.


CpgBuilderConfig

Controls which overlays are built and how the input is bounded.

pub struct CpgBuilderConfig {
    pub retain_source: bool,
    pub build_cfg: bool,
    pub build_dfg: bool,
    pub include_comments: bool,
    pub max_file_size: usize,
    pub resolve_imports: bool,
    pub parallel_analysis: bool,
}
Field Default Meaning
retain_source false Keep the source string in the graph (source_code()).
build_cfg true Run the CfgExtractor after AST construction.
build_dfg true Run the DfgExtractor after CFG.
include_comments false Keep comment nodes in the AST.
max_file_size 10 * 1024 * 1024 (10 MiB) Reject larger inputs in build (see the security guidance).
resolve_imports false Run evidence-bounded call/import resolution after CFG and before DFG extraction.
parallel_analysis false Compute independent per-function CFG and DFG plans on the current Rayon pool, then emit them in exact serial order.

Builder methods (each returns Self): new(), with_source(bool), with_cfg(bool), with_dfg(bool), with_comments(bool), with_max_file_size(usize), with_import_resolution(bool), with_parallel_analysis(bool). Default yields the table above.

// requires: no features (config construction is feature-free)
use libcpg::CpgBuilderConfig;

let config = CpgBuilderConfig::new()
    .with_source(true)      // retain the source string
    .with_cfg(true)
    .with_dfg(true)
    .with_comments(true)
    .with_import_resolution(true)  // resolve unique calls before interprocedural DFG
    .with_parallel_analysis(true); // use the caller's current Rayon pool

The historical field/method name says “imports,” but the implemented post-pass also resolves in-file lexical and unique bare-name calls. A visible import whose target is absent from the current CPG remains an External evidence record and creates no edge. The default remains false, preserving the established graph shape. See Evidence-bounded call resolution.


TreeSitterCpgBuilder

The tree-sitter-backed CpgBuilder. Wraps a CpgBuilderConfig.

pub struct TreeSitterCpgBuilder { /* private: config: CpgBuilderConfig */ }
Method Signature Notes
new fn new() -> Self Default configuration.
with_config fn with_config(config: CpgBuilderConfig) -> Self Custom configuration.
config fn config(&self) -> &CpgBuilderConfig Borrow the configuration.
set_config fn set_config(&mut self, config: CpgBuilderConfig) Replace the configuration.
build (via CpgBuilder) fn build(&self, source: &str, language: Language) -> Result<CodePropertyGraph> Mode A: internal parse.
build_from_tree fn build_from_tree(&self, tree: &tree_sitter::Tree, source: &str, language: Language) -> Result<CodePropertyGraph> Mode B: caller-supplied tree.
build_with_analysis_cache fn build_with_analysis_cache(&self, source: &str, language: Language, cache: &mut FunctionAnalysisCache) -> Result<CodePropertyGraph> Mode A with bounded exact CFG/DFG-plan reuse.
build_from_tree_with_analysis_cache fn build_from_tree_with_analysis_cache(&self, tree: &tree_sitter::Tree, source: &str, language: Language, cache: &mut FunctionAnalysisCache) -> Result<CodePropertyGraph> Mode B with bounded exact CFG/DFG-plan reuse.
build_file (via CpgBuilder) fn build_file(&self, path: &Path) -> Result<CodePropertyGraph> Infers language from the extension.

Mode A — build (internal parse)

build checks source.len() against config.max_file_size, fetches the grammar from ParserRegistry::global() (erroring with Error::UnsupportedLanguage if the lang-* feature is off), parses with tree-sitter, and then delegates the entire post-parse pipeline to build_from_tree so both paths stay identical.

// requires: features = ["lang-rust"]
use libcpg::{TreeSitterCpgBuilder, CpgBuilder, Language};

let builder = TreeSitterCpgBuilder::new();
let cpg = builder.build("fn main() { let x = 1; let y = x; }", Language::Rust)?;
assert!(cpg.node_count() > 1);
# Ok::<(), libcpg::Error>(())

Mode B — build_from_tree (caller-supplied tree)

Mode B: the caller parses with its own grammar and hands the tree to build_from_tree

Figure — the Mode-B sequence: a caller (e.g. pgmcp) that already links a grammar parses the source itself and hands the tree_sitter::Tree to build_from_tree, which runs the identical AST → CFG → optional call-resolution → DFG pipeline without a lang-* feature. Source: diagrams/mode-b-sequence.puml.

Mode B is for a caller that has already parsed the source with a tree-sitter grammar it owns. It:

  • needs no lang-* feature (the caller supplies the grammar);
  • skips the max_file_size check (the caller already owns the tree);
  • selects the NodeMapper from language; and
  • is the only path for Rholang and MeTTa, whose rholang/metta features are pure cfg toggles that gate node-mapper arms and vendor no grammar.
// requires: no libcpg lang feature; the caller owns the grammar crate
use libcpg::{TreeSitterCpgBuilder, Language};

// `tree` came from the caller's own tree-sitter parser for this language.
let builder = TreeSitterCpgBuilder::new();
let cpg = builder.build_from_tree(&tree, source, Language::Rholang)?;
# Ok::<(), libcpg::Error>(())

Equivalence. The inline test test_build_from_tree_matches_build (src/builder/tree_sitter.rs) parses one Rust snippet both ways and asserts the two graphs share node_count(), edge_count(), and language() — the caller-supplied-tree path produces a graph identical in shape to the internal parse.

AST conversion

Both modes convert tree-sitter nodes with an explicit preorder frame machine. For each retained node the builder: maps its kind via NodeMapper::map_kind; records a SourceRange from byte offsets; adds the node; wires an AstChild edge from the parent and sets the child's parent pointer (ancestor-based analyses read the pointer, not the edge); and marks every Function node as a CFG entry. Comments and pure punctuation are filtered per NodeMapper::should_include_node.

Security — max_file_size applies only to build

build rejects source.len() > config.max_file_size (default 10 MiB) with Error::Construction. build_from_tree performs no such check — a Mode-B caller feeding untrusted input must bound the source (and parse time) itself.

FunctionAnalysisCache

FunctionAnalysisCache is a caller-owned, deterministic least-recently-used (LRU) cache of exact function-local CFG, DFG, and PDG plans. It never caches a tree or a whole CPG: both cached builder methods rebuild the AST and allocate a fresh NodeId domain, then expand retained function-local ordinals into that domain. The default limits are 4,096 entries and 64 MiB of estimated resident key, artifact, and metadata bytes.

// requires: features = ["lang-rust"]
use libcpg::{
    CpgBuilderConfig, FunctionAnalysisCache, FunctionAnalysisCacheLimits,
    Language, TreeSitterCpgBuilder,
};

let builder = TreeSitterCpgBuilder::with_config(CpgBuilderConfig::new());
let mut cache = FunctionAnalysisCache::with_limits(
    FunctionAnalysisCacheLimits::new(512, 8 * 1024 * 1024),
);
let source = "fn stable(x: i32) -> i32 { x + 1 }";

let first = builder.build_with_analysis_cache(source, Language::Rust, &mut cache)?;
let second = builder.build_with_analysis_cache(source, Language::Rust, &mut cache)?;
assert_eq!(first.node_count(), second.node_count());
assert!(cache.stats().cfg_hits > 0);
# Ok::<(), libcpg::Error>(())

new, with_limits, limits, stats, reset_stats, and clear are the public cache operations. FunctionAnalysisCacheStats separates CFG/DFG/PDG hits, misses, dependency/key/capacity bypasses, evictions, occupancy, and resident key/artifact/metadata estimates. A zero entry or byte bound disables reuse while preserving the ordinary construction result. Digest equality is only an index lookup: a hit also requires full canonical-key byte equality and a valid local plan. See the component contract, usage guide, and security boundary.


ParserRegistry

Lazily maps Language values to tree-sitter grammars, one per enabled lang-* feature. Full path: libcpg::builder::ParserRegistry.

Method Signature Notes
global fn global() -> &'static ParserRegistry Process-wide singleton (OnceLock), initialized on first use.
get fn get(&self, lang: Language) -> Option<tree_sitter::Language> Grammar for lang, or None if its feature is off.
supports fn supports(&self, lang: Language) -> bool Whether a grammar is registered.
supported_languages fn supported_languages(&self) -> impl Iterator<Item = Language> + '_ The actually registered languages.
language_count fn language_count(&self) -> usize How many grammars are registered.

supported_languages() on the builder is a static list of 16 candidate languages regardless of features. The real availability is ParserRegistry::global().supports(lang) (or .get(lang).is_some()), which reflects the lang-* features you actually compiled. Query the registry, not the builder's static list, when you need ground truth.

The 16 grammars gated by lang-* features: Rust, Python, JavaScript, TypeScript, Go, Java, C, C++, JSON, HTML, CSS, Bash, TOML, YAML, Markdown, Ruby.


NodeMapper

Translates a tree-sitter node-kind string into a CpgNodeKind, per language. Full path: libcpg::builder::NodeMapper.

Method Signature Notes
new fn new(language: Language) -> Self Mapper for one language.
language fn language(&self) -> Language The configured language.
map_kind fn map_kind(&self, ts_kind: &str, node: &tree_sitter::Node, source: &str) -> CpgNodeKind Dispatch on language to the per-language mapper.
should_include fn should_include(&self, ts_kind: &str, include_comments: bool) -> bool String-keyed inclusion test (drops punctuation, comments, transparent wrappers).
should_include_node fn should_include_node(&self, node: &tree_sitter::Node, include_comments: bool) -> bool Node-aware test used by the builder walk; additionally drops anonymous Rholang keyword/operator tokens via is_named().

map_kind dispatches to map_rust, map_python, map_javascript (also TypeScript), map_go, map_java, map_c_cpp, map_ruby, map_json, map_html, map_css, map_bash, map_config (YAML/TOML), map_markdown, and — under the rholang/metta cfg toggles — map_rholang and map_metta (both Mode-B only). Anything else falls through to map_generic (a CpgNodeKind::Unknown). The Rholang/MeTTa mappings are detailed in the node-mapper component guide.


CfgExtractor & CfgExtractorConfig

Adds control-flow edges to a CPG that already has AST edges. The extractor borrows &self and mutates the CPG through &mut; extract returns ().

pub struct CfgExtractorConfig {
    pub include_fallthrough: bool,   // default true
    pub include_exceptions: bool,    // default true
    pub include_call_edges: bool,    // default true
}
Method Signature Notes
new fn new() -> Self Default config.
with_config fn with_config(config: CfgExtractorConfig) -> Self Custom config.
extract fn extract(&self, cpg: &mut CodePropertyGraph) Build CFG for every Function node.
extract_parallel fn extract_parallel(&self, cpg: &mut CodePropertyGraph) Plan every function CFG on the current Rayon pool; indexed collection plus ordered emission is exactly serial-equivalent.
extract_function_cfg fn extract_function_cfg(&self, cpg: &mut CodePropertyGraph, function: NodeId) Build CFG for one function.

extract_function_cfg marks the function as a CFG entry, connects it to its body (the last AST child), and runs a typed heap continuation machine over control constructs — If, While, For, Loop, Match, Return, Break, Continue, Try, Throw, and (when include_call_edges) Call — emitting the 14 CfgEdgeKind variants; a loop/try context stack routes break/continue/throw edges. Its native call depth is independent of AST depth, active-path Enter/Leave state cuts malformed cycles without suppressing shared DAG paths, and every emission uses connect_unique, so it is idempotent.

extract and extract_parallel share the same immutable-plan/ordered-emitter architecture. The latter parallelizes plan discovery only; graph mutation stays serial so edge identifiers, entry/exit order, and serde bytes do not depend on scheduling. See ADR-0059.

// requires: no features (feature-free over a hand-built or Mode-B graph)
use libcpg::CfgExtractor;

CfgExtractor::new().extract(&mut cpg);

BasicBlockIdentifier

Legacy leader-map API retained for compatibility. Its documentation points new callers to the feature-free, typed block_cfg analysis.

Method Signature Notes
new fn new() -> Self Constructor (unit struct).
identify fn identify(&self, cpg: &CodePropertyGraph, function: NodeId) -> FxHashMap<NodeId, Vec<NodeId>> Maps each legacy block leader to its original node ids, including emitted CFG fragments among AST descendants after a terminator.

New callers should prefer block_cfg, which additionally retains deterministic BlockId values, typed block edges, node membership, entry/exit blocks, errors, and post-order/RPO traversal. Leaders are the entry, every non-Sequential target, all successors of a multi-successor node, and every merge point; a block extends only along a unique Sequential relation. Once the CFG exists, cyclomatic_complexity() reads it directly.


DfgExtractor & DfgExtractorConfig

Adds data-flow edges via an AST-ordered reaching-definitions sweep. Like the CFG extractor, extract takes &self + &mut cpg and returns ().

pub struct DfgExtractorConfig {
    pub include_field_access: bool,     // default true: field/index flow + member Reference
    pub include_parameters: bool,       // default true
    pub include_return_values: bool,    // default true
    pub track_aliases: bool,            // default false
    pub max_iterations: usize,          // default 100
}
Method Signature Notes
new fn new() -> Self Default config.
with_config fn with_config(config: DfgExtractorConfig) -> Self Custom config.
extract fn extract(&self, cpg: &mut CodePropertyGraph) Build DFG for every Function.
extract_parallel fn extract_parallel(&self, cpg: &mut CodePropertyGraph) Plan every function DFG on the current Rayon pool and emit in exact serial order.
extract_function_dfg fn extract_function_dfg(&self, cpg: &mut CodePropertyGraph, function: NodeId) Build DFG for one function.

The sweep abstract-interprets the function body in source order, threading each statement's reaching definitions down into nested identifier uses (e.g. buf in decode(buf)). A let/Variable, Parameter, or plain Assignment generates a definition; in straight-line context it performs a strong update (latest write wins), and inside a conditional region a weak update; loop bodies are swept twice for loop-carried dependence. Each currently-reaching definition is linked to a use with a DataFlow(DefUse) edge. Typed heap Enter/Bind/Exit frames preserve that order while keeping AST depth off the native call stack; an active-path set cuts malformed cycles without suppressing legitimate shared-subtree visits. The pass is idempotent. It also emits Parameter, ReturnValue, and field/index access edges per config. (Two earlier CFG-fixpoint-based approaches are retained but compiled out under #[cfg(any())]; this is not SSA.)

Def / Use / chain types

The following are re-exported at the crate root:

pub struct Definition { pub variable: Arc<str>, pub node: NodeId, pub kind: DefinitionKind }
pub enum   DefinitionKind { Declaration, Assignment, Parameter, FieldWrite, IndexWrite }
pub struct Use { pub variable: Arc<str>, pub node: NodeId, pub kind: UseKind }
pub enum   UseKind { Read, FieldRead, IndexRead, Argument }

DefUseChain collects, per variable, its definitions, uses, and both directions of the mapping:

pub struct DefUseChain {
    pub variable: Arc<str>,
    pub definitions: Vec<NodeId>,
    pub uses: Vec<NodeId>,
    pub def_to_uses: FxHashMap<NodeId, Vec<NodeId>>,
    pub use_to_defs: FxHashMap<NodeId, Vec<NodeId>>,
}
DefUseChain method Signature
new fn new(variable: Arc<str>) -> Self
add_definition fn add_definition(&mut self, def: NodeId)
add_use fn add_use(&mut self, use_site: NodeId)
link fn link(&mut self, def: NodeId, use_site: NodeId)
uses_of fn uses_of(&self, def: NodeId) -> &[NodeId]
definitions_of fn definitions_of(&self, use_site: NodeId) -> &[NodeId]

The free function build_def_use_chains builds these from the DFG edges already in a graph:

pub fn build_def_use_chains(
    cpg: &CodePropertyGraph,
    function: NodeId,
) -> FxHashMap<Arc<str>, DefUseChain>;

PdgBuilder, backward_slice, forward_slice

The PDG is built on demand — never during initial construction — and is the substrate for program slicing.

PdgBuilder

pub struct PdgBuilder { /* private */ }
Method Signature Notes
new fn new() -> Self Constructor.
build fn build(&self, cpg: &mut CodePropertyGraph, function: NodeId) Add ControlDependence + DataDependence edges for function. Idempotent.
build_with_analysis_cache fn build_with_analysis_cache(&self, cpg: &mut CodePropertyGraph, function: NodeId, cache: &mut FunctionAnalysisCache) Reuse an exact local PDG plan or recompute on a boundary crossing.
build_all fn build_all(&self, cpg: &mut CodePropertyGraph) Plan and emit every function in stable order with one shared existing-edge snapshot.
build_all_with_analysis_cache fn build_all_with_analysis_cache(&self, cpg: &mut CodePropertyGraph, cache: &mut FunctionAnalysisCache) Cached serial planning with stable ordered emission.
build_all_parallel fn build_all_parallel(&self, cpg: &mut CodePropertyGraph) Plan every function on the current Rayon pool, then emit in exact serial order.
build_all_parallel_with_analysis_cache fn build_all_parallel_with_analysis_cache(&self, cpg: &mut CodePropertyGraph, cache: &mut FunctionAnalysisCache) Consume hits in function order and plan misses on the current Rayon pool.

build requires that function's CFG and DFG already exist. It computes control dependence [1] as the reverse dominance frontier: it forms the intraprocedural CFG, adds a virtual exit as a collision-free dense index linked from every real exit, and consumes the public deterministic post_dominator_tree result. That analysis uses bounded Cooper–Harvey–Kennedy post-dominators and the Cytron et al. [2] frontier. Data dependence edges re-project the DFG's DefUse/ReachingDef edges within the function.

Slicing

Both slicers are free functions at the crate root, feature-free, and return an FxHashSet<NodeId> (rustc-hash's set) — not a Vec:

pub fn backward_slice(cpg: &CodePropertyGraph, criterion: NodeId, max_nodes: usize) -> FxHashSet<NodeId>;
pub fn forward_slice (cpg: &CodePropertyGraph, criterion: NodeId, max_nodes: usize) -> FxHashSet<NodeId>;
Function Meaning
backward_slice Nodes that (transitively) affect criterion — reverse-BFS over incoming PDG edges.
forward_slice Nodes (transitively) affected by criterion — forward-BFS over outgoing PDG edges.

Both include criterion itself and stop once the slice reaches max_nodes nodes (a hard bound: pass a small cap for untrusted input). They traverse only ControlDependence/DataDependence edges (kind.is_pdg()), so call PdgBuilder::build first. Introduced by Weiser [3].

// requires: no features (feature-free)
use libcpg::{CfgExtractor, DfgExtractor, PdgBuilder, backward_slice};

CfgExtractor::new().extract(&mut cpg);
DfgExtractor::new().extract(&mut cpg);
PdgBuilder::new().build(&mut cpg, function);

let slice = backward_slice(&cpg, use_site, 256);
assert!(slice.contains(&use_site)); // the criterion is always included

See also


References

  1. Ferrante, J., Ottenstein, K. J., Warren, J. D. (1987). The Program Dependence Graph and Its Use in Optimization. ACM TOPLAS 9(3). DOI: 10.1145/24039.24041
  2. Cytron, R., Ferrante, J., Rosen, B. K., Wegman, M. N., Zadeck, F. K. (1991). Efficiently Computing Static Single Assignment Form and the Control Dependence Graph. ACM TOPLAS 13(4). DOI: 10.1145/115372.115320
  3. Weiser, M. (1984). Program Slicing. IEEE Transactions on Software Engineering SE-10(4). DOI: 10.1109/TSE.1984.5010248 (originally ICSE '81).