v5.6.5 — Rust rewrite, queryable cold storage

PFC-JSONL v5.6.5

Complete rewrite in Rust. Same CLI interface, same file format compatibility — significantly faster, smaller, and now available on all major platforms.

What is new in v5.6.5

• 4 MB native binary — no Python runtime, no pip, no virtualenv (was 40 MB Python bundle)
• 2x faster compression, 2x faster decompression vs v3.4.x
• macOS Intel x64 and Windows x64 — new platforms (v3.4 was Linux + macOS ARM64 only)
• Global dictionary — consistent token mapping across all blocks, better compression on large archives
• Newline-aligned blocks — no JSON line ever split across a block boundary
• Parallel seek-blocks — multiple blocks decompressed simultaneously for faster DuckDB queries
• S3 direct access — s3-info and s3-fetch via HTTP Range requests, presigned URLs supported
• Reads all v3.4 archives — fully backward compatible

Install

Linux x86_64:
curl -L https://github.com/ImpossibleForge/pfc-jsonl/releases/latest/download/pfc_jsonl-linux-x64 -o /usr/local/bin/pfc_jsonl && chmod +x /usr/local/bin/pfc_jsonl

macOS Apple Silicon:
curl -L https://github.com/ImpossibleForge/pfc-jsonl/releases/latest/download/pfc_jsonl-macos-arm64 -o /usr/local/bin/pfc_jsonl && chmod +x /usr/local/bin/pfc_jsonl

See README for macOS Intel and Windows install instructions.

DuckDB Extension

Unchanged — INSTALL pfc FROM community still works. Upgrade the binary, the extension picks it up automatically.

Benchmarks

Across 10 enterprise log types at 1 GB scale — PFC-JSONL v5.6.5 is 25-57% smaller than gzip-9 and 33-57% smaller than zstd-3, with 1-hour time-range queries in 2.8-5.3 seconds via the DuckDB Community Extension.

Full benchmark report: https://impossibleforge.com

v3.4.4 — Remove Community Mode, fix block-boundary Unicode

Changes

• Remove Community Mode: No more 5 GB/day limit. Free for personal and open-source use. Commercial use requires a license.
• Fix block-boundary Unicode bug: now filters incomplete lines at block boundaries (). Previously caused DuckDB queries to fail with UTF-8 decode errors.
• DuckDB extension: Added defensive filter (belt-and-suspenders layer).

Install

v3.4.3

Fix: version string now consistently v3.4 across all commands

PFC-JSONL v3.4.2 — Community Mode for all operations

Changes

• Community Mode now covers all operations — compress, decompress, query, seek-block, seek-blocks all free up to 5 GB/day
• No license key required for small users
• For production use (>5 GB/day): contact impossibleforge@gmail.com

Install

curl -L https://github.com/ImpossibleForge/pfc-jsonl/releases/latest/download/pfc_jsonl-linux-x64 \n     -o /usr/local/bin/pfc_jsonl && chmod +x /usr/local/bin/pfc_jsonl

DuckDB Extension

INSTALL pfc FROM community;
LOAD pfc;
SELECT * FROM read_pfc_jsonl(/path/to/events.pfc);

PFC-JSONL v3.4.1 — seek-blocks + DuckDB Extension support

Changes

• seek-blocks subcommand (DuckDB Extension primitive) — decompress multiple specific blocks in one call
• Binary index format .pfc.bidx — 32 bytes/block, directly readable by DuckDB C++ extension
• Community Mode — no license key required for ≤5 GB decompressed/day, tracked locally in ~/.pfc/usage.json
• No network calls, no phone-home

Install

curl -L https://github.com/ImpossibleForge/pfc-jsonl/releases/latest/download/pfc_jsonl-linux-x64 
     -o /usr/local/bin/pfc_jsonl && chmod +x /usr/local/bin/pfc_jsonl

DuckDB Extension

This release is required for the pfc DuckDB community extension.

INSTALL pfc FROM community;
LOAD pfc;
SELECT * FROM read_pfc_jsonl(/path/to/events.pfc);

PFC-JSONL v3.4.0

DuckDB extension support: seek-blocks + .pfc.bidx + Community Mode