meta-language

A Rust foundation for a universal, self-describing meta language backed by a links network. The initial crate focuses on the common structural substrate: links, references, source spans, parse status metadata, configurable trivia attachment, self-description roots, and verification that a parsed region is clean.

What Is Implemented

• A mutable LinkNetwork where every item is a link.
• Self-referential point links, so a point is represented without introducing a separate primitive.
• Relation links as ordinary links with ordered references to other links.
• Field labels as explicit links instead of side-table metadata.
• Source metadata: link type, named/anonymous flag, byte range, row/column points, and is_error, has_error, is_missing, is_extra flags.
• verify_full_match() for reporting error and missing links in a selected source region.
• parse() as the default lossless parse entry point; the explicit parse_lossless_text() boundary remains available.
• reconstruct_text() for byte-for-byte reconstruction from non-missing token links ordered by source span.
• insert_source_token(), insert_syntax_node(), and render_source() for emitting target-language source from programmatically constructed syntax networks whose token leaves do not come from a prior parse.
• projected_links() for viewing the same lossless network as concrete syntax, abstract syntax, or semantic-only data by stripping lower-level preservation links from the view.
• NetworkSnapshot and MutableNetworkSnapshot for immutable versioned snapshots, editable forks, provenance, and forward commits.
• AccessMode for a read-only or mutable engine per user configuration: freeze() / as_read_only() yield a ReadOnlyNetwork view whose mutators are unreachable at compile time, and parse_engine() returns an EngineNetwork that rejects mutation with a clear diagnostic under AccessMode::ReadOnly; the frozen form reuses snapshot Arc sharing.
• LinkStore and EngineLinkStore for storage-backed create/read/update/delete and search operations: reads take &self, writes take &mut self, the default store is the in-memory LinkNetwork, and read-only access mode rejects writes through the same storage boundary.
• Optional doublets feature support for a file-mapped DoubletsLinkStore using doublets-rs 0.4, with lossless network round trips and a documented doublets-web backend label for browser/WASM exchange of the same binary graph layout.
• ParseConfiguration with containment-link, token-link, or combined trivia attachment policies.
• Mixed-region links for Markdown fenced code and HTML regions, plus HTML script, style, and style-attribute regions, with txt fallback for prose regions that content sniffing cannot classify.
• LinkQuery for structural matching by link type, term, language, named flag, tree-sitter-query-like S-expressions, captures, and host predicates.
• find() / replace() for codemod-style query transforms over captured links while preserving unchanged source bytes.
• SubstitutionRule / apply_substitution() for the link-cli-style match-and-substitute operation.
• apply_edit() for incremental source reparsing with stable outside-edit link ids, snapshot fork sharing for unchanged links, and structural diff sets for changed, added, and removed links.
• Concept-to-language syntax mappings for cross-language reconstruction.
• reconstruct_text_as() for semantic cross-language reconstruction and configurable formalization levels.
• Exact-match concept interning with language-bound expression links, queryable external-id aliases, LiNo concept-set import, and seed_common_concept_ontology() for the default 351-concept semantic lexicon plus structural programming-language concepts.
• Object-identity links, many-valued TruthValue semantics, and fixed-point ProbabilisticTruthValue confidence semantics.
• A testable parity registry and upstream-provenanced PARITY_FIXTURES for executable competitor and ecosystem feature gates.
• Structural LiNo parsing for links-notation doublets, triplets, named links, simple indented definitions, and self-references.
• LANGUAGE_FIXTURES with lossless parse/reconstruction samples for every required markup, programming-language, and natural-language target.
• NATURAL_LANGUAGE_GRAMMAR_FIXTURES with pass/fail grammaticality fixtures for the ten natural-language targets, including provenance for the repo-authored sentences and UD-derived tag vocabulary.
• Coverage targets for full txt, Markdown, and HTML support, mixed grammar embedding, ten programming-language parser targets, and ten natural-language parser targets.
• Self-description roots for link, reference, relation link, language, grammar, type, Type, concept, point, field, trivia, region, and object.
• A lossless text parser boundary that preserves tokens, trivia, recovery markers, and mixed-region metadata behind the same representation.

Usage

use meta_language::{LinkNetwork, ParseConfiguration};

let network = LinkNetwork::parse("alpha beta", "plain-text", ParseConfiguration::default());
let report = network.verify_full_match(None);

assert!(report.is_clean());
assert_eq!(network.reconstruct_text(), "alpha beta");

The default parse path is lossless. Callers that need a narrower view can use a projection without mutating the original network:

use meta_language::{LinkNetwork, NetworkProjection, ParseConfiguration};

let network = LinkNetwork::parse("alpha beta", "plain-text", ParseConfiguration::default());
let abstract_links = network
    .projected_links(NetworkProjection::AbstractSyntax)
    .count();

assert!(abstract_links < network.len());

Construct source directly as a syntax network when code should be generated before validation:

use meta_language::{LinkNetwork, ParseConfiguration};

let mut network = LinkNetwork::new();
let tokens = [
    network.insert_source_token("JavaScript", "const answer = "),
    network.insert_source_token("JavaScript", "42"),
    network.insert_source_token("JavaScript", ";\n"),
];
let declaration = network.insert_syntax_node("JavaScript", "lexical_declaration", tokens);
network.insert_syntax_node("JavaScript", "program", [declaration]);

let source = network.render_source("JavaScript");
assert_eq!(source, "const answer = 42;\n");
assert!(LinkNetwork::parse(&source, "JavaScript", ParseConfiguration::default())
    .verify_full_match(None)
    .is_clean());

Configure the engine read-only when a parsed network must never be mutated. The frozen view exposes every read operation but no mutators (calling one is a compile error), and the EngineNetwork boundary rejects mutation at runtime:

use meta_language::{AccessMode, LinkNetwork, ParseConfiguration};

let configuration = ParseConfiguration::default().with_access_mode(AccessMode::ReadOnly);
let mut engine = LinkNetwork::parse_engine("alpha beta", "plain-text", configuration);

assert!(engine.is_read_only());
assert_eq!(engine.reconstruct_text(), "alpha beta");
assert!(engine.as_mutable().is_err()); // read-only engine rejects mutation

Use the storage trait directly when links need to move between in-memory and binary stores. The optional doublets backend is enabled with --features doublets:

use meta_language::{
    DoubletsLinkStore, LinkNetwork, LinkStore, LinkStoreQuery, ParseConfiguration,
};

let network = LinkNetwork::parse("const answer = 42;\n", "JavaScript", ParseConfiguration::default());
let mut store = DoubletsLinkStore::create_file("network.doublets").expect("create doublets store");
store.replace_with_network(&network).expect("write network");

let restored = DoubletsLinkStore::open_file("network.doublets")
    .expect("open doublets store")
    .to_network()
    .expect("read network");
assert_eq!(restored.to_lino(), network.to_lino());

let links = LinkStore::search(&restored, &LinkStoreQuery::new()).expect("search links");
assert_eq!(links.len(), restored.len());

LinkStoreBackend::DoubletsWeb names the WASM/browser exchange target for this binary graph representation; native code uses DoubletsLinkStore, while a browser host can map the same logical records through doublets-web.

Codemod-style transforms can select links with an S-expression query and replace only captured source ranges:

use meta_language::{LinkNetwork, LinkQuery, ParseConfiguration, ReplacementRule};

let mut network = LinkNetwork::parse(
    "const oldName = call(oldName);\n",
    "JavaScript",
    ParseConfiguration::default(),
);
let query = LinkQuery::from_sexpression(
    r#"
    (identifier) @target
    (#eq? @target "oldName")
    "#,
)
.expect("query parses");
let captures = network.find(&query);

network.replace(
    &captures,
    &ReplacementRule::captured_text("target", "newName"),
);

assert_eq!(network.reconstruct_text(), "const newName = call(newName);\n");

Cross-language reconstruction can naturalize a parsed semantic proposition into another target language, or expose progressively more formal representations:

use meta_language::{FormalizationLevel, LinkNetwork, ParseConfiguration};

let network = LinkNetwork::parse(
    "Hawaii is a state.\n",
    "English",
    ParseConfiguration::default(),
);

assert_eq!(
    network.reconstruct_text_as("Russian", ParseConfiguration::default()),
    "Гавайи это штат.\n"
);
assert_eq!(
    network.reconstruct_text_as(
        "Russian",
        ParseConfiguration::default().with_formalization_level(FormalizationLevel::Concept),
    ),
    "statehood(Q782, Q35657)\n"
);

CLI

cargo run -- describe
cargo run -- verify --language plain-text --text "alpha beta"

describe prints the built-in self-description network as LiNo-style definition lines that round-trip through parse() and reconstruct_text(). verify parses the text with the lossless text boundary and exits successfully when the resulting region has no error or missing links.

Parity Implementation

The crate exposes PARITY_TARGETS, MARKUP_LANGUAGE_TARGETS, PROGRAMMING_LANGUAGE_TARGETS, SECOND_TIER_PROGRAMMING_LANGUAGE_TARGETS, NATURAL_LANGUAGE_TARGETS, DATA_FORMAT_TARGETS, and GRAMMAR_EMBEDDING_TARGETS so comparison scope is part of the tested Rust API. It also exposes PARITY_FIXTURES, with executable, provenance-tracked fixtures covering every advertised target capability, and LANGUAGE_FIXTURES, with a lossless fixture for every requested language target. The current registry tracks tree-sitter, LibCST, Recast, jscodeshift, Rowan, cstree, Roslyn, links-notation, link-cli, lino-objects-codec, relative-meta-logic, formal-ai, and meta-expression. Internal ecosystem fixtures now structurally parse links-notation doublet, triplet, tuple, indented, and self-reference cases; link-cli create/update/delete/swap substitutions; lino object round-trip, shared-reference, and circular-reference cases; relative-meta-logic dependent, many-valued, probabilistic, and liar-paradox cases; formal-ai seed and benchmark .lino corpora; and meta-expression formalize and naturalize examples backed by the verified 351-concept lexicon.

See docs/parity-roadmap.md for the feature matrix, executable fixture gates, and language coverage targets.

Development

cargo fmt --check
cargo clippy --all-targets --all-features
cargo test --all-features
rust-script scripts/check-no-src-tests.rs

This repository uses changelog fragments in changelog.d/; code changes should include a fragment with the intended semantic-version bump.