diff --git a/.github/workflows/pages.yml b/.github/workflows/pages.yml index b2b7ae3..bff464a 100644 --- a/.github/workflows/pages.yml +++ b/.github/workflows/pages.yml @@ -43,8 +43,8 @@ jobs: FMPL_BOOTSTRAP_PHASE=1 cargo build -p fmpl-bootstrap touch fmpl-core/build.rs - - name: Build fmpl-wasm (release) - run: cargo build -p fmpl-wasm --release --target wasm32-unknown-unknown + - name: Build fmpl-wasm (size-tuned wasm-release profile) + run: cargo build -p fmpl-wasm --profile wasm-release --target wasm32-unknown-unknown # Must match the exact wasm-bindgen version pinned in fmpl-wasm/Cargo.toml. - name: Install wasm-bindgen CLI @@ -52,9 +52,19 @@ jobs: - name: Generate JS bindings into the site run: | - wasm-bindgen target/wasm32-unknown-unknown/release/fmpl_wasm.wasm \ + wasm-bindgen target/wasm32-unknown-unknown/wasm-release/fmpl_wasm.wasm \ --out-dir docs/repl --target web --no-typescript + # Pinned binaryen release — apt's is too old to accept the sign-ext + # ops Rust emits by default. 129 matches the locally verified build. + # Rust also emits bulk-memory ops by default; wasm-opt must be told. + - name: Shrink with wasm-opt + run: | + curl -sL https://github.com/WebAssembly/binaryen/releases/download/version_129/binaryen-version_129-x86_64-linux.tar.gz | tar xz + binaryen-version_129/bin/wasm-opt -Oz --enable-bulk-memory --enable-nontrapping-float-to-int \ + --strip-producers docs/repl/fmpl_wasm_bg.wasm -o docs/repl/fmpl_wasm_bg.wasm + ls -l docs/repl/fmpl_wasm_bg.wasm + - name: Upload Pages artifact uses: actions/upload-pages-artifact@v3 with: diff --git a/Cargo.toml b/Cargo.toml index f3c6556..67b9b6b 100644 --- a/Cargo.toml +++ b/Cargo.toml @@ -50,3 +50,14 @@ rkyv = { version = "0.8", features = ["smol_str-0_3"] } # Async streaming tokio-stream = "0.1" async-trait = "0.1" + +# Size-tuned profile for the browser wasm build only (pages.yml). Safe with +# panic="abort": VM errors are Results by design (no catch_unwind in-tree), +# so unwinding machinery is pure weight here. ~2.1MB -> ~1.1MB raw. +[profile.wasm-release] +inherits = "release" +opt-level = "z" +lto = "fat" +codegen-units = 1 +panic = "abort" +strip = true diff --git a/README.md b/README.md index f45d5ec..cb34792 100644 --- a/README.md +++ b/README.md @@ -6,7 +6,9 @@ image is the source of truth; source files are a bootstrapping convenience. New here? Read the **[engineering tour](https://mparrett.github.io/fmpl/fmpl-tour.html)** — architecture, verified capabilities, and the honest gap ledger, in one page — -or try the **[browser REPL](https://mparrett.github.io/fmpl/repl.html)** +learn the language from the **[language guide](https://mparrett.github.io/fmpl/fmpl-guide.html)** +(the web rendering of [`TUTORIAL.md`](TUTORIAL.md)), or try the +**[browser REPL](https://mparrett.github.io/fmpl/repl.html)** (fmpl-core compiled to WebAssembly, no install). > **Status: experimental.** FMPL is a working prototype under active diff --git a/docs/fmpl-guide.html b/docs/fmpl-guide.html new file mode 100644 index 0000000..9f6da89 --- /dev/null +++ b/docs/fmpl-guide.html @@ -0,0 +1,1421 @@ + + +
+ + +Language guide · github.com/mparrett/fmpl
+Norman Nunley’s tutorial for experienced programmers, rendered for the web — expressions, pattern matching, grammars, objects, and the metaprogramming pipeline.
+ + +Everything on this page comes from TUTORIAL.md / DEMO.md, which CI executes against the real interpreter on every push — the -- Returns: comments are asserted, not aspirational. And it all runs here too, starting with this terminal: press ▸ run to step through a session.
The fastest path is this page: the terminal above is live, and every block below steps through the same VM — nothing to install. When you want the full toolchain on your machine:
+git clone https://github.com/mparrett/fmpl.git && cd fmpl
+just build # bootstrap the FMPL-generated parser, then build
+cargo run -p fmpl-cli # the REPL
+cargo run -p fmpl-web # web UI on http://localhost:3000
+cargo run -p fmpl-tui # terminal UI
+
+ There’s also a full browser REPL for blank-slate scratch sessions (opens in a new tab) — but the guided path continues right here: every block below runs in place.
+ +FMPL is expression-oriented: every expression produces a value, and statements don’t exist. Literals evaluate to themselves:
+"Hello, World!" -- String literals evaluate to themselves
+42 -- Numbers evaluate to themselves
+true -- Booleans evaluate to themselves
+
+ One thing to internalize before anything else: FMPL is purely functional with immutable bindings. Once bound, a name cannot be reassigned. Loops are recursion; accumulation is fold. Most surprises for newcomers trace back to this.
-- Numbers
+42
+3.14
+-10
+
+-- Strings
+"Hello, World!"
+"Line 1\nLine 2\tTabbed" -- Escape sequences: \n \t \r \\ \" \' \0
+
+-- Booleans
+true
+false
+
+-- Null
+null
+
+ -- Single-line comments start with double dash
+
+/*
+ Multi-line comments
+ are supported
+*/
+
+ -- Arithmetic operators
+1 + 2 -- 3
+10 - 4 -- 6
+3 * 4 -- 12
+15 / 3 -- 5
+
+-- Comparison operators
+1 == 1 -- true
+1 != 2 -- true
+5 < 10 -- true
+5 <= 5 -- true
+10 > 5 -- true
+10 >= 10 -- true
+
+-- Logical operators
+true && false -- false
+true || false -- true
+!true -- false
+
+ Arithmetic is checked — overflow is a clean error, not a panic. Exponentiation (**) is planned but not yet implemented. String concatenation uses + and requires both operands to be strings: a mixed "n = " + 42 is a type mismatch and evaluates to null.
-- List literals (`;` separates consecutive expressions — a `[` opening a
+-- new line would otherwise be read as indexing the previous expression)
+[1, 2, 3];
+["apple", "banana", "cherry"];
+[1, "mixed", true];
+
+-- Empty list
+[]
+
+-- Indexing, length, and (immutable) append
+let numbers = [1, 2, 3]
+numbers[0] -- => 1
+numbers.len() -- => 3
+numbers.push(4) -- => [1, 2, 3, 4] (a new list; `numbers` is unchanged)
+
+ Lists are immutable. The higher-order methods work — [1, 2, 3].map(\x x * 2) returns [2, 4, 6], and .fold() is available. For element access beyond indexing, use pattern matching with the @ operator or recursive functions.
-- Map literal
+%{name: "Alice", age: 30, city: "NYC"}
+
+-- Empty map
+%{}
+
+-- Map access
+let person = %{name: "Bob", age: 25}
+person.name -- "Bob"
+person.age -- 25
+
+ FMPL is prototype-based, not class-based. Objects are created with object expressions — more in §8:
-- Basic object (must be named)
+object counter {
+ count: 0
+ increment(): self.count + 1
+ value(): self.count
+}
+
+ Objects must be named in the current implementation. The receiver inside methods is self — there is no this. Anonymous object literals and constructors (^name) are planned features.
@The @ operator is FMPL’s swiss-army knife — apply the thing on the right to the value on the left. It covers three jobs with one mechanism: applying grammars to parse text, matching patterns against values, and transforming data via pattern-directed rules.
-- Match strings against regex-style patterns
+"hello" @ {
+ [a-z]+ => "word"
+}
+-- Returns: "word" (matches [a-z]+)
+
+"12345" @ {
+ [0-9]+ => "number"
+}
+-- Returns: "number"
+
+ Map patterns extract values via bindings. Arms are separated with ;, and the OMeta-style binding syntax is _:name:
-- Extract values using bindings, with a wildcard fallback arm
+let response = %{status: 200, body: "ok"}
+
+response @ { %{status: _:code, body: _:msg} => msg; _ => "other" }
+-- Returns: "ok"
+
+-- Nested map patterns work too
+%{outer: %{inner: "value"}} @ { %{outer: %{inner: _:i}} => i }
+-- Returns: "value"
+
+ To match on a specific value, bind it and guard with when (or its alias if):
%{status: 200, body: "ok"} @ {
+ %{status: _:s} when s == 200 => "success";
+ %{status: _:s} => "failed"
+}
+-- Returns: "success"
+
+%{code: 404} @ { %{code: _:c} when c == 404 => "not_found"; _ => "found" }
+-- Returns: "not_found"
+
+ Literal values directly inside map patterns (%{status: 200} => ...) are not yet supported — the compiler rejects them. Bind and guard, as above.
-- Match a list and extract elements
+[1, 2, 3] @ { [ _:x, _:y, _:z ] => [x, y, z] }
+-- Returns: [1, 2, 3]
+
+-- Length must match: this arm does not match a 3-element list
+[1, 2, 3] @ { [ _:x, _:y ] => "two"; _ => "not two" }
+-- Returns: "not two"
+
+-- Empty list pattern
+[] @ { [] => "empty" }
+-- Returns: "empty"
+
+ Rest patterns ([first | rest]) are planned but not yet implemented. In the REPL, write @ { ... } match blocks on a single line with ; between arms — multi-line @ { blocks are routed to the grammar engine. Multi-line works fine with the match keyword form.
match 5 { n if n > 3 => "big"; _ => "small" }
+-- Returns: "big"
+
+ And when you don’t need a pattern at all, plain field access works:
+let response = %{
+ tool: "curl.get",
+ args: %{url: "https://example.com"}
+}
+
+response.tool
+-- Returns: "curl.get"
+
+response.args.url
+-- Returns: "https://example.com"
+FMPL includes an OMeta-style PEG grammar system for parsing and transformation. Grammar rules match input and run semantic actions; capture matched text with a :binding suffix on a pattern element:
-- Define a grammar: capture the digits, return them from the action
+let g = grammar { num = [0-9]+:d => d }
+
+"42" @ g.num
+-- Returns: "42"
+
+-- Actions are arbitrary expressions
+let shout = grammar { word = [a-z]+:w => w + "!" }
+"hello" @ shout.word
+-- Returns: "hello!"
+
+ A full JSON parser written this way ships with the repo — see lib/json.fmpl. The metacircular FMPL parser itself (lib/core/fmpl_parser.fmpl) is the largest grammar in the tree: the language’s canonical parser is written in the language (the tour’s §3 tells that story).
Built-in base grammar rules are available under base::parser:
-- Apply built-in base grammar rules to input
+"12345" @ base::parser.integer -- Returns: "12345"
+"hello" @ base::parser.word -- Returns: "hello"
+
+ Grammar inheritance (<: with <super.rule> overrides) is a designed feature that is deliberately deferred (DESIGN-005). Compose grammars by referencing shared rules for now.
FMPL uses then/else keywords, not braces — and if is an expression:
-- if-then-else
+if 15 > 10 then "big" else "small"
+-- Returns: "big"
+
+-- Nested with expressions
+if 150 > 100 then
+ "huge"
+else if 15 > 10 then
+ "big"
+else
+ "small"
+-- Returns: "huge"
+
+-- With let bindings
+let (value = 42)
+ if value > 10 then "big" else "small"
+-- Returns: "big"
+
+ There are no mutable variables, so there are no loop counters. Iteration is recursion:
+-- Sum numbers recursively (lambda bound at top level)
+let sum_range = \start, end if start > end then 0 else start + sum_range(start + 1, end)
+
+sum_range(1, 10)
+-- Returns: 55
+
+-- Factorial via recursion
+let factorial = \n if n <= 1 then 1 else n * factorial(n - 1)
+
+factorial(5)
+-- Returns: 120
+
+ Recursion works through top-level (statement-style) let bindings — the lambda body resolves the name at call time. Scoped let (f = ...) expression bindings cannot see themselves recursively yet (a known limitation; see docs/known-gaps.md).
The corollary, from the demo’s “known limitations” file: a for body cannot mutate an outer binding — sum = sum + x inside a loop creates a new sum in the inner scope and the outer one is unchanged. The idiomatic shape is map + fold:
let numbers = [1, 2, 3, 4, 5]
+let doubled = numbers.map(\x x * 2)
+doubled.fold(0, \acc, x acc + x) -- => 30
+
+ -- let-in expression (scoped binding)
+let (x = 42) x * 2
+-- Returns: 84
+
+-- Multiple bindings
+let (x = 10, y = 20) x + y
+-- Returns: 30
+
+-- Statement-style let (binds to current scope)
+let x = 42
+let y = x * 2
+y + 10
+-- Returns: 94
+Functions are lambdas bound to names with let:
-- Bind a lambda to a name
+let add = \a, b a + b
+add(1, 2) -- 3
+
+-- The lambda keyword form is equivalent
+let inc = lambda (n) n + 1
+inc(41) -- 42
+
+ The name(args): body definition syntax only exists inside object blocks (as method definitions) — it is not a top-level function form. Functions must be defined before they’re called.
-- Lambda syntax: single param, multi-param (comma-separated), curried
+\x x + 1
+\x, y x + y
+\x \y x + y
+
+-- Apply lambda immediately
+(\x x * 2)(5) -- 10
+
+-- Store and call later
+let doubler = \x x * 2
+doubler(7) -- 14
+
+-- Curried application
+let addc = \x \y x + y
+addc(3)(4) -- 7
+
+ -- Functions can take other functions as arguments
+let apply_twice = \f, x f(f(x))
+let add_one = \x x + 1
+
+apply_twice(add_one, 5);
+-- Returns: 7
+
+-- Built-in higher-order list methods
+[1, 2, 3].map(\x x * 2)
+-- Returns: [2, 4, 6]
+Prototype objects carry properties and methods; methods are called through the object’s name:
+-- Define object
+object counter {
+ count: 0
+ increment(): self.count + 1
+ value(): self.count
+}
+
+-- Access methods via the object name
+counter.value() -- 0
+counter.increment() -- 1
+
+ Inside a method body, a small set of context variables is always available:
+self — the receiver of the method callparent — the parent object, for prototype-chain lookupcaller — the object that called this methoduser — the current user contextargs — the list of all arguments passed to the methodobject greeter {
+ name: "world"
+ show(): "Hello, " + self.name
+}
+
+greeter.show()
+-- Returns: "Hello, world"
+
+ The receiver is self, as in Python or Smalltalk — there is no this, and using this silently breaks the enclosing object definition.
FMPL supports first-class AST and IR values — you can write compilers, DSLs, and code generators entirely in FMPL. This is the language’s thesis feature: the real bootstrap pipeline is built from exactly the pieces below.
+Algebraic data types are written as lists whose first element is a symbol — the single canonical list form (DESIGN-002):
+-- Create tagged values
+[:Int, 42];
+[:Binary, :+, [:Int, 1], [:Int, 2]];
+[:User, "alice", %{active: true}]
+
+-- Pattern match on tagged values (bare identifiers bind in tagged patterns)
+let value = [:Binary, :+, [:Int, 1], [:Int, 2]]
+value @ {
+ [:Binary, :+, a, b] => "addition";
+ [:Binary, :-, a, b] => "subtraction";
+ [:Int, n] => "just a number"
+}
+-- Returns: "addition"
+
+ Operator symbols like :+, :-, :* are ordinary symbols and can appear in tagged values and patterns. The legacy constructor syntax :Int(42) is rejected with a hint: use [:Int, 42] instead.
ast::parse turns source text into a tagged AST; ir::compile turns IR into executable bytecode; code::eval runs it:
let ast = ast::parse("1 + 2")
+-- Returns: [:Binary, :+, [:Int, 1], [:Int, 2]]
+
+let ast2 = ast::parse("if true then 1 else 2")
+-- Returns: [:If, [:Bool, true], [:Int, 1], [:Int, 2]]
+
+let code = ir::compile([:Add, [:LoadInt, 1], [:LoadInt, 2]])
+code::eval(code)
+-- Returns: 3
+
+ Put the three together and a compiler is a pattern match:
+-- Parse source, transform AST to IR, compile, execute
+let ast = ast::parse("1 + 2")
+
+let ir = ast @ {
+ [:Binary, :+, [:Int, a], [:Int, b]] => [:Add, [:LoadInt, a], [:LoadInt, b]];
+ [:Binary, :-, [:Int, a], [:Int, b]] => [:Sub, [:LoadInt, a], [:LoadInt, b]];
+ [:Binary, :*, [:Int, a], [:Int, b]] => [:Mul, [:LoadInt, a], [:LoadInt, b]];
+ [:Binary, :/, [:Int, a], [:Int, b]] => [:Div, [:LoadInt, a], [:LoadInt, b]]
+}
+
+let code = ir::compile(ir)
+code::eval(code)
+-- Returns: 3
+
+ This is the shape of the real thing: lib/core/ast_to_ir.fmpl transforms full ASTs to IR the same way, and ast::parse itself is backed by the FMPL-written parser in lib/core/fmpl_parser.fmpl (DESIGN-001, the metacircular bootstrap). Supported IR nodes: LoadNull, LoadBool, LoadInt, LoadFloat, LoadString, LoadVar, Var, Add, Sub, Mul, Div, Mod, Neg, Not, Eq, NotEq, Lt, Gt, LtEq, GtEq, Let, Seq, If, Return, MakeList, MakeTagged.
Streaming is FMPL’s first-class concern, and the observable surface today is the cursor API: observe a value as a stream, then move a cursor over it. Cursors are immutable — advancing one returns a new cursor:
+let data = [10, 20, 30]
+let cursor = stream::observe(data)
+
+-- Get current position
+cursor::position(cursor) -- => 0 (Int)
+
+-- Advance cursor
+let advanced = cursor::advance(cursor, 1)
+cursor::current(advanced) -- => 20
+
+-- Rewind cursor
+let rewound = cursor::rewind(advanced, 1)
+cursor::current(rewound) -- => 10
+
+ Multiple cursors over the same stream are independent (copy-on-write):
+let data = [10, 20, 30]
+let cursor1 = stream::observe(data)
+let cursor2 = stream::observe(data)
+
+cursor::position(cursor1) -- => 0
+cursor::position(cursor2) -- => 0 (independent cursor)
+
+let advanced = cursor::advance(cursor1, 1)
+cursor::position(advanced) -- => 1
+cursor::position(cursor2) -- => 0 (unchanged)
+
+ This is the substrate the grammar engine parses over — the same cursor discipline backs backtracking and streaming input. The async operators that will surface it in the language (<-, spawn, |>) have syntax in the parser today with the runtime in progress.
-- Parse JSON string
+let json_str = "{\"name\": \"Alice\", \"age\": 30}"
+
+-- Use json::parse builtin
+let parsed = json::parse(json_str)
+-- Returns: %{age: 30, name: "Alice"}
+
+-- Validate structure
+parsed @ {
+ %{name: _:n, age: _:a} when a >= 18 => "Adult: " + n;
+ %{name: _:n, age: _:a} => "Minor: " + n;
+ _ => "Invalid structure"
+}
+-- Returns: "Adult: Alice"
+
+ requires network — not run by CI
+-- Make HTTP GET request using the curl builtin
+let response = curl.get("https://api.example.com/data")
+
+-- Parse JSON response
+let data = json::parse(response)
+
+-- Extract specific fields
+data @ {
+ %{status: _:s, results: _:r} when s == "ok" => r;
+ %{error: _:e} => "Error: " + e;
+ _ => "Unknown response"
+}
+
+ The pattern-matching machinery is exactly what tool-call dispatch wants:
+requires network — not run by CI
+-- Dispatch tool calls by binding the tool name and guarding on it
+let handle_tool_call = \tc tc @ {
+ %{tool: _:t, args: _:a} when t == "curl.get" => curl.get(a.url);
+ %{tool: _:t, args: _:a} when t == "curl.post" => curl.post(a.url, a.body);
+ %{text: _:txt} => txt;
+ _ => "Error: Unrecognized response"
+}
+
+-- Simulate LLM response
+let llm_output = "{\"tool\": \"curl.get\", \"args\": {\"url\": \"https://example.com\"}}"
+let parsed = json::parse(llm_output)
+
+-- Handle the tool call (performs the HTTP request)
+handle_tool_call(parsed)
+FMPL is designed for agentic AI workflows — closing the loop between LLMs, tools, and human oversight. Real LLM clients ship in the standard library: lib/anthropic.fmpl (Claude API, requires ANTHROPIC_API_KEY) and lib/ollama.fmpl (local models), with shared plumbing in lib/llm-common.fmpl.
The single-turn shape is the agent loop from §11; a multi-turn loop is the same thing, recursive (llm_complete and execute_tool stand in for your LLM client and tool registry):
design sketch not yet runnable
+let agent_turn = \input, history
+ llm_complete(%{history: history, input: input}) @ {
+ %{tool: _:t, args: _:a} => agent_turn(execute_tool(t, a), history);
+ %{answer: _:ans} => ans;
+ _ => "Error: Unexpected LLM output"
+ }
+
+let result = agent_turn("Search for latest Rust version", [])
+
+ The project’s north star goes further: express agent control flow as grammars. Grammar rules define behavior declaratively, backtracking gives retry-on-failure for free, and rules are inspectable data rather than opaque code. Not yet a working feature — this sketch shows the intended shape:
+design sketch not yet runnable
+grammar ToolAgent <: base::tree {
+ -- Main loop: process messages
+ turn = message:m => {
+ let ctx = %{history: get_history()}
+ ::llm_complete(m, ctx) |> tool_output
+ }
+
+ -- Handle LLM output stream
+ tool_output =
+ | %{tool: t, args: a} => {
+ let result = ::execute_tool(t, a)
+ turn(result) -- recurse with result
+ }
+ | %{done: r} => r -- terminate
+ | %{text: t} => t -- stream text
+}
+
+ The same goes for durable suspension: the persistence engine (Fjall-backed) exists in-core, and language-level checkpoint / resume_from builtins are designed but not yet exposed — the intended payoff is pause-and-resume workflows, human-in-the-loop approvals, and crash recovery for long-running agents.
The short version of what’s real today. The tour’s §4 carries the full verified capability list and §5 the gap ledger; docs/known-gaps.md in the repo is the canonical inventory, grouped by root cause.
And don’t take the page’s word for any of it: verify re-runs every block above in a fresh VM, right here in your browser, and reports a verdict per block — without disturbing the text.
+ast::parse / ir::compile / code::eval)json::parse, curl.get/post, stream cursors, type predicates<-, spawn, |>) — syntax in, runtime in progress^name) — designed, implementation evolving**, string interpolation, rest patterns, literal map-pattern values, recursive scoped let<:), tuple space, capability policies, durable approvalsTo go deeper: TUTORIAL.md and DEMO.md are this page’s sources of truth; specs/ holds the implementation specs (VM, grammars, objects, persistence), and the engineering tour covers the architecture those specs describe.
+Start with TUTORIAL.md — every snippet in it runs as documented, verified line-by-line against the REPL. DEMO.md is the quick version; specs/ holds the implementation specs (VM, grammars, objects, persistence, tuplespace); docs/known-gaps.md is the honest map of the frontier. If you want to contribute, the metacircular-parser bucket is the critical path: pick a file from the ledger, run it with -- --ignored, and land the feature its tests describe.
Start with the language guide — the web rendering of TUTORIAL.md and DEMO.md, every snippet of which runs as documented, verified line-by-line against the REPL. specs/ holds the implementation specs (VM, grammars, objects, persistence, tuplespace); docs/known-gaps.md is the honest map of the frontier. If you want to contribute, the metacircular-parser bucket is the critical path: pick a file from the ledger, run it with -- --ignored, and land the feature its tests describe.