Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 4 additions & 1 deletion CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,14 +8,17 @@ All notable user-facing changes should be documented here. LAC is still experime
- Added CI coverage for Debug tests, Release builds, and ASan/UBSan smoke tests on GitHub Actions.
- Added self-contained generated WAV fixtures for clean-checkout CI test runs.
- Updated E2E tests to read back temporary `.lac` files before decode.
- Added encoder thread limiting through `LAC_THREADS` and `lac_cli encode --threads=N`.
- Added codec worker limiting through `LAC_THREADS` and `lac_cli encode|decode --threads=N`.
- Clarified the CLI-first PCM WAV roundtrip contract and canonical restored-WAV behavior.
- Hardened WAV parsing against inconsistent RIFF sizes, malformed chunk boundaries, non-canonical PCM metadata, empty payloads, and unchecked data-chunk allocation.
- Hardened `.lac` decoding against non-canonical reserved fields, stereo flags, residual metadata, padding, trailing payload bytes, out-of-range restored samples, oversized decoded allocations, and malformed Rice values.
- Added generated `lac_cli` subprocess roundtrips, full supported WAV-domain generated fixtures, and malformed-input regression coverage.
- Rejected encode or decode commands whose output path refers to the input file.
- Fixed canonical RIFF padding for odd-sized restored PCM payloads and tightened close-time write error handling.
- Bounded tiny-block decoder work, made extreme zigzag decoding portable, and corrected the LPC reconstruction specification.
- Staged CLI output beside the requested path and published it only after successful close, preserving existing files and linked targets on failed writes.
- Added format version 3 compressed block boundaries and bounded parallel decode while retaining serial decode compatibility for canonical version 2 streams.
- Switched encode planning to bounded `16384`-sample windows, added sampled automatic stereo probes for ambiguous blocks, and reduced residual estimation and Rice bit-output work.
- Removed tracked editor cache files and generated compile database symlink from source control.
- Expanded repository roadmap tracking for correctness, fuzzing, security hardening, and release readiness.

Expand Down
1 change: 1 addition & 0 deletions CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,7 @@ add_library(lac STATIC
src/codec/frame/frame_header.hpp
src/codec/lac/decoder.hpp
src/codec/lac/encoder.hpp
src/codec/lac/thread_limit.hpp
src/codec/lac/thread_collector.hpp
src/codec/lpc/lpc.hpp
src/codec/rice/rice.hpp
Expand Down
11 changes: 6 additions & 5 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@
[![CodeQL](https://github.com/audexdev/Lossless-Audio-Codec/actions/workflows/codeql.yml/badge.svg)](https://github.com/audexdev/Lossless-Audio-Codec/actions/workflows/codeql.yml)
[![License](https://img.shields.io/badge/license-Apache--2.0-blue.svg)](LICENSE)

LAC is an experimental CLI-first C++20 lossless audio codec for PCM WAV audio. It is a compact implementation of a custom `.lac` container and bitstream with LPC prediction, adaptive Rice coding, mid/side stereo, zero-run residual coding, residual partitioning, Apple Silicon NEON acceleration, and multithreaded block encoding.
LAC is an experimental CLI-first C++20 lossless audio codec for PCM WAV audio. It is a compact implementation of a custom `.lac` container and bitstream with LPC prediction, adaptive Rice coding, mid/side stereo, zero-run residual coding, residual partitioning, Apple Silicon NEON acceleration, and multithreaded block encoding and decoding.

The current product contract is `lac_cli encode` followed by `lac_cli decode` for the documented PCM WAV domain. The project is intended for codec experimentation, implementation study, and reproducible work on lossless audio compression internals. The file format is still experimental and should not yet be treated as a long-term archival format.

Expand Down Expand Up @@ -68,12 +68,13 @@ Choose a stereo mode explicitly:
./build/lac_cli encode input.wav output.lac --stereo-mode=ms
```

Stereo encoding defaults to automatic per-block LR or mid/side selection. Mono input always uses LR metadata. Restored WAV files preserve PCM samples, channel count, sample rate, and bit depth, but ancillary WAV chunks are not copied. Input and output paths must refer to different files.
Stereo encoding defaults to automatic per-block LR or mid/side selection. Mono input always uses LR metadata. Restored WAV files preserve PCM samples, channel count, sample rate, and bit depth, but ancillary WAV chunks are not copied. Input and output paths must refer to different files. The CLI stages output beside the requested path and publishes it only after the completed file closes successfully.

Limit encoder worker threads:
Limit codec worker threads:

```sh
./build/lac_cli encode input.wav output.lac --threads=12
./build/lac_cli decode output.lac restored.wav --threads=12
LAC_THREADS=12 ./build/lac_cli encode input.wav output.lac
```

Expand All @@ -96,15 +97,15 @@ LAC_THREADS=4 ctest --test-dir build-tests --output-on-failure

The default CTest configuration uses lightweight generated WAV fixtures and exercises both internal codec paths and `lac_cli` subprocess roundtrips. To opt into larger local E2E fixtures, configure with `-DLAC_TEST_ASSETS_DIR="$PWD/assets"`. The generated fixtures keep clean checkouts and routine development self-contained.

Set `LAC_THREADS=N` to cap encoder worker threads during tests. The heavier `test_all.sh` asset roundtrip script defaults to `LAC_THREADS=12` unless the environment already sets a different value.
Set `LAC_THREADS=N` to cap encode and decode worker threads during tests. The heavier `test_all.sh` asset roundtrip script defaults to `LAC_THREADS=12` unless the environment already sets a different value.

## Contributing

Contribution setup, review expectations, and local development commands are documented in [CONTRIBUTING.md](CONTRIBUTING.md).

## Format

The current `.lac` bitstream is documented in [docs/format.md](docs/format.md). The format is versioned internally as frame version `2`, but it is not yet frozen for external compatibility.
The current `.lac` bitstream is documented in [docs/format.md](docs/format.md). The encoder emits frame version `3`, while the decoder retains serial compatibility for canonical version `2` streams. The format is not yet frozen for external compatibility.
Supported WAV/PCM input and output constraints are documented in [docs/supported-formats.md](docs/supported-formats.md).

## Security
Expand Down
36 changes: 27 additions & 9 deletions docs/format.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,28 +18,42 @@ This document describes the current experimental `.lac` format implemented by th
```text
FrameHeader
u32 block_count
u32 block_size[block_count]
repeat block_count:
u32 block_size
u32 compressed_size_bytes
BlockPayload[block_count]
```

The block-size table gives the number of samples per channel in each block. Every channel payload in the same block uses the same `block_size`.
The version `3` block table gives the number of samples per channel and the complete encoded byte length for each block. `compressed_size_bytes` covers the optional per-block stereo flag and every byte-padded channel block. Every channel payload in the same block uses the same `block_size`.

Current top-level limits:

- `block_count` must be non-zero.
- `block_count` must not exceed `1048576`, and the complete block-size table must be present before allocation.
- `block_count` must not exceed `1048576`, and the complete block table must be present before allocation.
- Each `block_size` must be non-zero and no larger than `16384` samples per channel.
- Each `compressed_size_bytes` must be non-zero, and their sum must exactly match the remaining frame payload bytes.
- Every non-final block must contain at least `256` samples per channel. The final block may be shorter.
- The total declared sample count must fit within the implementation's 1 GiB decoded-PCM allocation limit and the classic RIFF/WAV output size limit.

Version `2` streams use the legacy table layout:

```text
FrameHeader(version = 2)
u32 block_count
u32 block_size[block_count]
BlockPayload[block_count]
```

Version `2` remains decode-compatible, but it does not carry encoded block boundaries and is decoded serially.

## Frame Header

The frame header is 80 bits, currently 10 bytes:

| Field | Bits | Meaning |
| --- | ---: | --- |
| sync | 16 | `0x4C41` (`LA`) |
| version | 8 | current format version, `2` |
| version | 8 | current encoder format version, `3`; legacy decode also accepts `2` |
| channels | 8 | `1` mono or `2` stereo |
| stereo_mode | 8 | `0` LR, `1` mid/side, `2` per-block stereo |
| sample_rate_low | 16 | low 16 bits of sample rate |
Expand Down Expand Up @@ -180,7 +194,7 @@ Current limits:
- minimum partition size: 32 samples
- maximum partition order: 8

The partition flag must be set when `partition_order > 0` and unset when `partition_order == 0`. The default residual mode must be `0`, `1`, or `2`, and it must match the first partition metadata entry. Partitioned blocks use stateless Rice adaptation inside each partition. Unpartitioned blocks use stateful Rice adaptation.
The partition flag must be set when `partition_order > 0` and unset when `partition_order == 0`. The default residual mode must be `0`, `1`, `2`, or `3`, and it must match the first partition metadata entry. Modes `0`, `1`, and `2` use Rice adaptation: partitioned blocks use stateless adaptation inside each partition, while unpartitioned blocks use stateful adaptation. Mode `3` uses a fixed Rice parameter for the whole partition.

Partition sizes are computed as:

Expand Down Expand Up @@ -233,7 +247,7 @@ write one zero bit
write r as k bits when k > 0
```

The initial `k` for each residual segment comes from that partition's `u5 initial_k` metadata. `k` is updated after each logical residual sample, including samples represented by zero-run tokens.
For adaptive residual modes, the initial `k` for each residual segment comes from that partition's `u5 initial_k` metadata. `k` is updated after each logical residual sample, including samples represented by zero-run tokens.

#### Stateless Adaptation

Expand Down Expand Up @@ -367,16 +381,20 @@ The fallback path uses the same adaptive `k` model.

Every bin token represents exactly one residual sample. Tags `00`, `01`, and `10` update the adaptive model with unsigned values `0`, `2 or 1`, and `4 or 3` respectively after sign reconstruction. Tag `11` decodes one Rice-coded residual using the current `k`, then updates the same adaptive model.

### Mode 3: Static Rice

Static Rice mode uses the same zigzag mapping and Rice bit coding as mode `0`, but `k` is fixed to the partition's `u5 initial_k` value for every residual in the partition. No adaptive state is updated and `k` never changes.

## Padding

Each channel block is flushed to the next byte boundary after residual encoding. Padding bits must be zero. Non-zero padding is rejected as non-canonical.

## Integrity

The current format does not include a checksum, frame CRC, block CRC, or authenticated length field. Decoders validate structural fields strictly and reject trailing garbage, impossible block sizes, invalid residual tags, Rice values or predictor reconstruction outside the signed 32-bit domain, non-zero reserved fields or padding, and non-canonical metadata. Without an integrity field, a modified payload can still decode successfully if it remains structurally valid and produces in-range PCM samples.
The current format does not include a checksum, frame CRC, block CRC, or authenticated length field. Version `3` compressed block lengths are structural boundaries, not integrity protection. Decoders validate structural fields strictly and reject trailing garbage, impossible block sizes, mismatched version `3` payload lengths, invalid residual tags, Rice values or predictor reconstruction outside the signed 32-bit domain, non-zero reserved fields or padding, and non-canonical metadata. Without an integrity field, a modified payload can still decode successfully if it remains structurally valid and produces in-range PCM samples.

## Compatibility

The format version is currently `2`, but the format is still experimental. Future work may add stronger validation, checksums, fuzzed compatibility tests, streaming decode constraints, or a frozen public specification.
The encoder currently emits format version `3`, but the format is still experimental. Version `3` adds compressed block byte lengths so blocks can be validated and decoded independently. Future work may add stronger validation, checksums, fuzzed compatibility tests, streaming decode constraints, or a frozen public specification.

The canonical version `2` byte sequences emitted by the encoder are unchanged by decoder hardening. Hardened decoders may reject version `2` byte sequences that older permissive decoders accepted when those sequences contain non-canonical reserved fields, metadata, padding, stereo flags, block tables, or trailing payload bytes.
The decoder retains serial compatibility for canonical version `2` streams. Hardened decoders may reject version `2` byte sequences that older permissive decoders accepted when those sequences contain non-canonical reserved fields, metadata, padding, stereo flags, block tables, or trailing payload bytes.
15 changes: 8 additions & 7 deletions docs/supported-formats.md
Original file line number Diff line number Diff line change
Expand Up @@ -44,18 +44,18 @@ supported WAV -> lac_cli encode -> .lac -> lac_cli decode -> restored WAV

For supported input, the restored WAV has the same PCM samples, channel count, sample rate, and bit depth. The restored file is a canonical PCM WAV with a 16-byte `fmt ` chunk followed by one `data` chunk. When the PCM payload size is odd, the writer appends the required zero RIFF padding byte after the `data` payload. Ancillary chunks and their metadata are intentionally not copied. Encode defaults to per-block stereo selection for stereo input; `--stereo-mode=lr` and `--stereo-mode=ms` force a mode. Mono input always uses LR mode metadata.

Encode and decode reject input and output paths that refer to the same file. This prevents an ordinary in-place command from clobbering its source. Output publication is not hardened against concurrent path replacement in a directory modified by another process.
Encode and decode reject input and output paths that refer to the same file, including a second check immediately before output publication. The CLI writes a complete output into a temporary sibling directory and replaces the final path only after the writer closes successfully. A failed write or publication leaves an existing final output untouched, and replacing an existing symlink or hardlink output does not stream bytes into its prior target. Publication is not `fsync`-backed crash durability, and the CLI is not a filesystem access-control boundary for directories concurrently modified by untrusted processes.

The alpha CLI surface is:

| Command or option | Behavior |
| --- | --- |
| `lac_cli encode input.wav output.lac` | encode a supported WAV; stereo input defaults to per-block stereo selection |
| `lac_cli decode input.lac output.wav` | decode one canonical `.lac` stream to canonical PCM WAV |
| `lac_cli decode input.lac output.wav` | decode one supported canonical version `2` or `3` `.lac` stream to canonical PCM WAV |
| `--stereo-mode=lr` | force LR stereo payloads during encode |
| `--stereo-mode=ms` | force mid/side stereo payloads during encode |
| `--threads=N` | cap encoder workers to positive integer `N`; overrides `LAC_THREADS` |
| `LAC_THREADS=N` | cap encoder workers when `--threads=N` is absent |
| `--threads=N` | cap encode or decode workers to positive integer `N`; overrides `LAC_THREADS` |
| `LAC_THREADS=N` | cap encode or decode workers when `--threads=N` is absent |
| `--no-partitioning` | disable residual partitioning during encode |

The CLI may overwrite an existing output file when it is distinct from the input file. Flags beginning with `--debug-` are diagnostic implementation aids and are not part of the stable alpha contract.
Expand All @@ -66,23 +66,24 @@ The CLI may overwrite an existing output file when it is distinct from the input

The current `.lac` container supports:

- format version `2`
- canonical encode format version `3`; legacy version `2` remains decode-compatible
- mono or stereo streams
- LR, mid/side, or per-block stereo mode
- block sizes up to `16384` samples per channel
- fixed, FIR, and LPC predictors
- adaptive Rice, zero-run, and small-residual bin residual modes
- adaptive Rice, zero-run, small-residual bin, and static Rice residual modes

See `docs/format.md` for bitstream details.

## Current Limits

- The format has no checksum, frame CRC, block CRC, or authenticated payload length.
- The format has no checksum, frame CRC, block CRC, or authenticated payload length. Version `3` block lengths are structural boundaries only.
- The WAV reader and LAC decoder reject decoded PCM output above 1 GiB.
- `lac_cli decode` rejects compressed `.lac` input files above 1 GiB before loading them into memory.
- The decoder rejects non-canonical structural metadata, trailing payload bytes, out-of-range restored samples, and output that cannot fit classic RIFF/WAV 32-bit size fields.
- The decoder rejects more than `1048576` blocks and non-final blocks shorter than `256` samples to bound block-table metadata and tiny-block decode work.
- Large-file handling is still bounded by classic RIFF/WAV 32-bit size fields on output.
- On Windows, CLI paths currently pass through narrow argument and stream APIs. Non-ASCII paths outside the active code page are not guaranteed.
- Streaming encode/decode is not currently exposed as a public API.
- Structurally valid payload corruption can still produce different in-range PCM because the format has no integrity field.
- Malformed-input validation continues to be improved through security hardening issues and fuzzing work.
Loading
Loading