Skip to content

Fiyy/gridlang

Repository files navigation

GridLang

tests pages License: MIT Python 3.9+ Latest release

The AI-native spreadsheet format — data, formulas, and visualization in a single plain-text file.

🚀 Try it live in your browser → No install needed; the entire toolkit runs client-side via Pyodide.

GridLang is to Excel what Markdown is to Word: a human-readable, version-controllable, AI-friendly format that captures the full power of spreadsheets — 67 formulas, 9 chart types, conditional formatting, JS/Python compute engines, real-time CRDT collaboration — without the binary blob.

Quick Install

cd /data/gridlang
pip install -e .

Or skip the install entirely — the live demo runs the whole stack in your browser via Pyodide. Try the examples, edit cells, import an Excel file, export to .xlsx / .csv — all client-side.

Quick Start

Create hello.grid:

--- meta ---
name: "Q1 Sales"
version: "1.0"

--- data ---
Region,Jan,Feb,Mar,Total
North,120,135,150,=SUM(B2:D2)
South,95,110,125,=SUM(B3:D3)
West,80,90,105,=SUM(B4:D4)

--- present ---
chart: bar
  data: B2:D4
  labels: A2:A4
  title: "Q1 Sales by Region"

Run it:

gridlang run hello.grid

CLI Commands

gridlang run

Execute a .grid file and display results in the terminal.

gridlang run report.grid              # terminal output
gridlang run report.grid --json       # JSON output
gridlang run report.grid --sheet Q2   # run a specific sheet

gridlang render

Render to a standalone HTML file with embedded charts.

gridlang render dashboard.grid -o dashboard.html

gridlang validate

Check file format and formula syntax without executing.

gridlang validate report.grid

gridlang info

Print a structural summary: sheets, cell count, formulas, charts.

gridlang info report.grid

gridlang import

Import from Excel or CSV into .grid format.

gridlang import data.xlsx -o data.grid
gridlang import sales.csv -o sales.grid

gridlang export

Export a .grid file to Excel (with native charts) or CSV.

gridlang export report.grid -o report.xlsx
gridlang export report.grid -o report.csv --format csv
gridlang export remote.grid -o report.xlsx --allow-remote
gridlang export report.grid -o raw.xlsx --raw
gridlang export report.grid -o values.xlsx --values-only  # flatten formulas

export shares the same @source policy and cache controls as run and render: HTTP(S) requires --allow-remote, and --no-cache forces a refetch.

gridlang serve

Launch a live-preview web server with auto-reload on file changes.

gridlang serve dashboard.grid --port 8080
gridlang serve dashboard.grid --port 8080 --edit            # editor UI
gridlang serve dashboard.grid --port 8080 --edit --collab   # multi-peer (v0.8)
GRIDLANG_TOKEN=change-me gridlang serve dashboard.grid --host 0.0.0.0 --collab

With --collab, multiple browser tabs can connect to the same URL and edit cells live; changes converge via a CRDT (LWW per cell with HLC clocks). See Collaborative Editing.

The server handles requests concurrently. Binding outside loopback requires either --token / GRIDLANG_TOKEN, or the explicit unauthenticated --insecure-network override. With token auth enabled, open http://host:8080/?token=change-me once to establish an HttpOnly cookie; API clients can send Authorization: Bearer change-me or X-GridLang-Token. Use --reload-interval 0.5 to tune file-change polling. The built-in server is plain HTTP, so use a TLS reverse proxy when the network is not trusted.

gridlang js-bundle

Bundle a JavaScript-engine .grid file into a self-contained .js that runs anywhere a JS engine does — Node, browser, Web Worker, edge function. The bundle embeds the data, helpers, pipeline runner, and your compute code; no gridlang or Python at runtime.

gridlang js-bundle report.grid -o report.bundle.js   # Node bundle
gridlang js-bundle report.grid --browser -o worker.js  # Worker bundle
node report.bundle.js                                # prints JSON

Excel vs GridLang

Dimension Excel (.xlsx) GridLang (.grid)
AI readability Requires binary parsing libraries Plain text — readable as-is
Version control Binary diffs are opaque git diff works perfectly
Formulas 400+ proprietary functions 67 Excel-compatible built-ins
Charts GUI-only creation Declarative, inline SVG
Multi-sheet Tab-based UI --- data:name --- sections
Conditional formatting Dialog-heavy Inline rules
Remote data Manual import / Power Query @source: <url> directive
Reactive editing VBA macros / OLE {{ cell("B2") }} + bind: form widgets
Live collaboration OLE + Office365 servers serve --collab over CRDT, no cloud needed
Compute language VBA + 400 functions Python or JavaScript (engine: selector)
Bundle for browser Closed-source COM/OLE gridlang js-bundle --browser produces a Web Worker
Testability Nearly impossible Standard unit tests (563 passing)
Interop Locked ecosystem Import/export Excel & CSV

Feature Highlights

Cell Formulas + 67 Compute Functions

Formulas can live directly in data cells. They are evaluated before the compute layer, including dependency chains and cross-sheet references.

Region,Jan,Feb,Mar,Total,Status
North,120,135,150,=SUM(B2:D2),=IF(E2>400,"On target","Below")

--- data:summary ---
Q1 Total,=SUM(sales!B2:B50)

Supported cell syntax includes A1/range references, arithmetic, comparisons, IF/IFERROR, common aggregate, lookup, text, math, and date functions. Circular references and invalid references are reported with the exact cell. The compute sandbox additionally exposes 67 pandas-oriented Excel-compatible functions, including PIVOT, FILTER, and GROUPBY.

9 SVG Chart Types

Declarative charts rendered as clean SVG. Use either the Chart DSL block syntax (preferred, AI-friendly) or call the helpers directly with {{ bar_chart(...) }}.

--- present ---
chart: line
  data: B1:B12
  labels: A1:A12
  title: "Monthly Trend"

chart: heatmap
  data: B2:D10
  title: "Correlation Matrix"

chart: sparkline
  data: C2:C50
  inline: true

Supported types: bar, line, pie, scatter, area, stacked_bar, heatmap, sparkline, color_scale

References inside DSL blocks (Revenue → column, agg.foo → aggregate, B2:D4 → A1 range, Q1,Q2,Q3 → multi-series, sales!Revenue → cross-sheet) all resolve automatically. See spec/SPEC.md §16 for the full grammar.

Multi-Sheet Support

Organize data across named sheets, reference between them.

--- data:revenue ---
Region,Q1,Q2,Q3,Q4
North,120,135,150,160

--- data:summary ---
Total,=SUM(revenue!B2:E2)

Conditional Formatting

--- present ---
format: color_scale
  range: B2:B100
  min_color: "#ffffff"
  max_color: "#e74c3c"

format: data_bar
  range: C2:C50

format: rules
  range: D2:D50
  rule: ">90 -> bold green"
  rule: "<60 -> italic red"

Remote Data Sources

Load data from URLs or local files instead of (or in addition to) inline CSV. Inline rows act as a fallback when the remote source is unavailable or --allow-remote is not given.

--- data ---
@source: https://api.example.com/sales.json
@format: json
@select: data.records
@cache: 1h
@header: Authorization: Bearer xyz

# Fallback used when the remote is denied / fails
Region,Total
North,100
gridlang run report.grid                  # uses inline fallback
gridlang run report.grid --allow-remote   # fetches the URL

Supported schemes: file:// (always allowed), http(s):// (opt-in via --allow-remote). Format auto-detected from the URL extension; csv/tsv/ json/xlsx are supported. JSON @select drills into a sub-path with a.b.c[0] syntax. See spec/SPEC.md §17 for the full directive table.

Reactive Bindings

Make individual cells editable from the rendered preview. Edits go directly back into the .grid source and trigger a re-render — like a tiny spreadsheet, but the source-of-truth stays in plain text.

Inline cell binding with the cell() Jinja helper:

<td>{{ cell("B2") }}</td>            <!-- editable cell -->
<td>{{ cell("B2", fmt=",.2f") }}</td> <!-- formatted -->
<td>{{ cell("B2@sales") }}</td>      <!-- cross-sheet -->

Form-style binding with bind: blocks:

bind: input
  cell: B2
  label: "Unit Price"
  type: number
  step: 0.10

bind: select
  cell: A2
  options: North, South, East, West
gridlang serve dashboard.grid --port 8080 --edit
# Open http://localhost:8080 — click a cell, type, blur to commit.

The server's POST /api/cell-edit endpoint accepts {cell: "B2", value: ...} and rewrites only the target row, preserving comments, blank lines, @directives, and formulas in other cells. Header rows are read-only. See spec/SPEC.md §18 for the full grammar and protocol.

Excel Import with Formula Conversion

Preserves scalar Excel formulas as executable GridLang cell formulas. Imported workbooks retain dependency chains, Excel row coordinates for headerless sheets, and cross-sheet references even when sheet names are normalized for .grid.

gridlang import quarterly_report.xlsx -o report.grid
# Formulas and sheet references are preserved and evaluated on run/render

Excel array formulas retain their anchor and target range under meta.cell_formulas.spills. GridLang evaluates range arithmetic and common dynamic functions (UNIQUE, FILTER, SORT, TRANSPOSE, SEQUENCE, and array IF) into that range, rejects size/overwrite conflicts, and restores the native array formula on XLSX export.

Imported workbook presentation is also retained under meta.excel_styles: column widths, row heights, hidden rows/columns, merged cells, cell fonts, fills, borders, alignment, number formats, and common conditional formats are rendered in the generated sheet preview and restored by aligned XLSX export through either openpyxl or XlsxWriter.

The same metadata preserves worksheet visibility, frozen panes, filter ranges, gridline/zoom settings, data validation, comments, hyperlinks, and native Excel tables. Comments and links remain visible in the browser preview. Workbook and sheet-local defined names live under meta.excel_workbook; common structured table references and defined names are executable in data-cell formulas. Print areas/titles, page setup, margins, headers/footers, and worksheet protection settings are also retained. Disabling visual style extraction does not discard these semantic workbook objects. Cell-level locked/hidden flags, sheet tab colors, and manual row/column page breaks round-trip as well.

Worksheet and table filters retain value/date/custom/dynamic/icon/Top-N rules and sort state. Raster images are embedded as base64 assets with editable one-cell or two-cell anchors and appear in the browser preview. Core/custom workbook properties, calculation settings, window views, and workbook structure protection are stored under meta.excel_workbook. Custom workbook theme XML is retained there as editable text so theme colors and fonts continue to resolve natively after export.

Conditional formatting now retains formula, text, time-period, duplicate, Top-N, icon-set, two/three-color scale, and data-bar rules with their priorities and differential styles, rather than flattening them to three basic presets. Rich-text cells retain editable per-run font, color, emphasis, underline, strikeout, and subscript/superscript metadata and render as styled spans in the browser preview. Excel sparklines retain structured group options, source and target ranges, axis/point settings, and colors. Package-level restoration keeps them native in both OpenPyXL and XlsxWriter exports. Macro-enabled .xlsm and .xltm imports retain VBA project binaries, digital signature relationships, content types, and associated package parts as editable XML/base64 metadata. Both export engines restore a valid macro-enabled package after worksheet data or formulas are changed. External workbook links retain their target URI, external sheet/name catalog, and cached link XML, and are rebuilt as native externalLink parts on export. Workbook data connections retain provider connection strings, commands, refresh flags, and their native package content type/relationship. A parsed connection summary accompanies the authoritative editable XML. Scenario Manager definitions and one/two-variable What-If Data Table formulas are structured per worksheet and restored natively by both export engines. Modern threaded comments retain people, reply parent IDs, timestamps, resolved state, and mention ranges as structured metadata and native Office 2019 parts. Power Query DataMashup packages retain their internal metadata while exposing every Formulas/*.m section as directly editable text. QueryTable bindings, fields, connection IDs, and refresh options are structured and rebound by table name or worksheet after export. Workbook XML Maps expose map names, root/schema identities, import options, DataBinding attributes, and editable XSD text. Single-cell and table-column XML bindings expose their cell references, unique names, Map IDs, XPath expressions, and XML data types, and are restored by both XLSX engines. PivotTables retain editable table locations, row/column/page/data field attributes and styles. Shared PivotCache definitions, worksheet sources, refresh behavior, and compressed cache records are restored without duplicating caches across tables. Slicers, timelines, PivotCharts, chartEx charts, DrawingML shapes, text boxes, groups, connectors, and SmartArt retain editable object metadata and native package relationships. Power Pivot workbooks preserve the model binary exactly while exposing model tables, relationships, XMLA summaries, and DAX measures. Forms and ActiveX controls expose names, text, macros, links, anchors, property bags, and VML state; referenced images and custom-property streams are restored. Embedded and linked OLE objects expose ProgIDs, display modes, relationships, object/VML anchors, and preview-image metadata. Legacy Compound Binary files and nested XLSX/DOCX/PPTX packages are retained as replaceable encoded payloads with their native content types and relationship graphs. Calculation chains, XLM macro/dialog sheets, header/footer images, file-sharing attributes, and workbook package properties also round-trip through both XLSX engines. Legacy shared-workbook history exposes tracking settings, revision authors and timestamps, sheet-ID maps, reviewed IDs, recursive revision records, and shared-user metadata; agents can edit existing history or add new users and revision-log parts while retaining native workbook relationships.

Excel date, datetime, and time cells retain their temporal type and seconds during import/export. Since Excel has no timezone storage, timezone-aware values are normalized to UTC before being written as native Excel date serials.

Excel Export with Native Formulas and Charts

Exports produce real .xlsx files. Data-cell formulas are preserved as native Excel formulas when the output still aligns with the source data; use --values-only to flatten them. Numeric dashboards can also receive native charts. Both openpyxl and XlsxWriter compile declared bar, line, pie, scatter, area, and stacked_bar blocks while retaining their selected data, title, dimensions, and colors; documents without declarations retain automatic Bar/Line/Pie chart generation.

gridlang export dashboard.grid -o dashboard.xlsx

See the Excel capability matrix for the exact round-trip guarantees and the remaining unsupported Excel object families.

Sandboxed Python Execution

The compute layer runs in a restricted sandbox — safe for AI-generated code.

--- compute ---
def transform(df):
    df['Growth'] = df['Revenue'].pct_change() * 100
    return df

Additional installed modules can be explicitly enabled per document:

--- meta ---
name: "Forecast"
engine: python
version: "1.0"
dependencies: [scipy]

dependencies contains Python import names, not pip version constraints. The modules must already be installed. Host-level modules such as os, pathlib, socket, and subprocess remain blocked even when declared.

JavaScript Compute Engine

Set engine: javascript in the meta section to author the compute layer in JS. Useful when you're sharing .grid files with frontend codebases or want the compute layer to be runnable by both Python and Node tooling.

--- meta ---
name: "Q1 Sales"
engine: javascript
version: "1.0"

--- compute ---
function transform(df) {
  df.addColumn('Tax', r => r.Revenue * 0.2);
  return df;
}
function aggregates(df) {
  return { total: df.sum('Revenue'), tax: df.sum('Tax') };
}

The DataFrame is exposed as an array of records with a ~25-method helper API covering aggregations (sum, mean, std, quantile, describe), filtering (where, head, distinct, find), reshaping (pluck, drop, rename, assign), sorting/grouping (sortBy, groupBy, countBy), joins (join, leftJoin, concat), and conversion (toCSV, toRecords).

function aggregates(df) {
  const top3 = df.sortBy('Revenue', { desc: true }).head(3);
  const byRegion = df.groupBy('Region');
  const sums = {};
  for (const k of Object.keys(byRegion)) sums[k] = byRegion[k].sum('Revenue');
  return {
    median:    df.median('Revenue'),
    p90:       df.quantile('Revenue', 0.9),
    distinct:  df.distinct('Region').count(),
    top:       top3.col('Product'),
    by_region: sums,
  };
}

User code runs in a Node vm sandbox — no require, process, or filesystem access. Requires Node 18+; falls back gracefully via JsRuntimeUnavailable when Node isn't on PATH. See spec/SPEC.md §19 and §20 for the full helper table and bundle format.

Self-Contained JS Bundles

gridlang js-bundle packages a .grid file's data + compute layer into a single JS file that runs anywhere a JS engine does — without gridlang, without Python, without npm.

# Node: prints pipeline result as JSON on stdout
gridlang js-bundle report.grid -o bundle.js
node bundle.js

# Browser / Web Worker: drop into a <script> or new Worker(blob)
gridlang js-bundle report.grid --browser -o worker.js

The bundle embeds your data (post-@source resolution), the compute code, the helper API, and the pipeline runner — verbatim, no minification chain, no CDN. Two identical .grid files produce byte-identical bundles, so they diff cleanly and cache by SHA.

Web Preview with Auto-Reload

gridlang serve report.grid --port 8080
# Open http://localhost:8080 — updates live as you edit

Collaborative Editing (v0.8)

Multiple browser tabs (or separate users on the same network) can edit the same .grid file at once. Changes converge via a CRDT — every replica that has seen the same set of operations ends up with the same per-cell value, regardless of arrival order.

gridlang serve dashboard.grid --collab --edit
# Open http://localhost:8080 in two browser tabs.
# Edit a cell in tab A → within ~700ms the value appears (and flashes blue) in tab B.
# Edit the same cell in both tabs at once → the higher-HLC write wins on every replica.

The implementation is three small modules:

Module Role
gridlang.crdt HLC + LWW per-cell + version vectors
gridlang.collab CollabSession — peers, persistence, sync
gridlang.collab_client Self-contained browser JS (~250 lines)

Wire protocol is JSON-over-HTTP — POST /api/collab/{join,op,poll,leave}, GET /api/collab/{snapshot,stats}. The server's on-disk .grid file remains the source of truth; every commit is persisted, so gridlang run or render after a collab session sees the merged values.

See spec/SPEC.md §21 for the full protocol, convergence proof sketch, and programmatic Python API.

Project Structure

gridlang/
├── gridlang/
│   ├── parser.py          # .grid file parser
│   ├── schema.py          # data layer validation
│   ├── runtime.py         # compute engine (sandboxed)
│   ├── js_runtime.py      # alternative JavaScript compute engine (v0.6)
│   ├── js_bundle.py       # Node + Web Worker bundle generator (v0.7)
│   ├── js/                # JS source files (df_helpers, pipeline, bridge)
│   ├── crdt.py            # HLC + LWW per-cell CRDT (v0.8)
│   ├── collab.py          # CollabSession — peers, persistence, sync (v0.8)
│   ├── collab_client.py   # browser-side collab JS (v0.8)
│   ├── renderer.py        # HTML/SVG rendering
│   ├── chart_dsl.py       # chart:/format: DSL preprocessor
│   ├── data_sources.py    # @source remote data loader + cache
│   ├── bindings.py        # reactive cell + form bindings (v0.5)
│   ├── formulas.py        # 67 built-in functions
│   ├── cell_formulas.py   # safe A1/range formula parser + workbook evaluator
│   ├── charts.py          # 9 SVG chart types
│   ├── excel_import.py    # .xlsx → .grid conversion
│   ├── excel_export.py    # .grid → .xlsx with native formulas/charts
│   ├── csv_io.py          # CSV import/export
│   ├── server.py          # live preview server
│   └── cli.py             # CLI entry point
├── spec/SPEC.md           # format specification
├── examples/              # 13 example .grid files + sample.xlsx
└── tests/                 # 563 passing tests

License

MIT — see also CHANGELOG.md for the version history.

About

AI-native spreadsheet format — data + compute + presentation in one .grid file. Plain-text, version-controllable, with Python/JS engines and CRDT-based real-time collaboration.

Topics

Resources

License

Stars

0 stars

Watchers

0 watching

Forks

Packages

 
 
 

Contributors