From 90dd11f2cc7936087f8422de2566cf3792196596 Mon Sep 17 00:00:00 2001 From: i18n Date: Fri, 4 Sep 2026 12:39:53 +0800 Subject: [PATCH] docs: add ALP/fastalp floating-point compression algorithm introduction --- docs/reference/concept_design/compress.md | 18 ++++++++++++++++++ .../reference/concept_design/compress.md | 18 ++++++++++++++++++ 2 files changed, 36 insertions(+) diff --git a/docs/reference/concept_design/compress.md b/docs/reference/concept_design/compress.md index 01d9a5f76..4bceebe2f 100644 --- a/docs/reference/concept_design/compress.md +++ b/docs/reference/concept_design/compress.md @@ -46,6 +46,24 @@ gorilla的原理与差分类似，区别在于差分是两个数据的差，gori 在不指定压缩算法的情况下，我们默认为浮点型指定这种压缩算法。 +### ALP / fastalp + +主要用于单精度（f32）与双精度（f64）浮点型时序数据。 + +#### 原理 + +ALP（Adaptive Lossless floating-Point Compression）是一种面向十进制时序浮点数的高性能无损压缩算法。现实世界中的绝大多数浮点时序数据（如物联网传感器采样、工业测控、系统监控指标、金融高频报价、GPS 经纬度等）本质上是由有限十进制小数生成的。传统基于 IEEE 754 二进制表示的异或算法（如 Gorilla）因二进制尾数位的随机性导致压缩率有限；而通用熵编码（如 Pco/tANS）虽具备高压缩率但编解码吞吐开销较高。 + +ALP 的核心思想是通过自适应采样自动推导最佳十进制缩放因子（10^e × factor），将浮点数无损精确转换为整型数值，随后结合基准框架编码（Frame-of-Reference）或前向一阶差分（Delta-ALP），并通过向量化 SIMD 位打包（Bitpacking）进行极致高吞吐压缩。对于无法精确十进制转换的浮点数值（如无限循环小数或极端离群值），ALP 使用双精度补码与异常处理表进行精确存储，保证 100% 位精确无损还原。 + +[fastalp](https://github.com/webc-site/wedb_embed/tree/main/fastalp) 是 ALP 算法的高性能纯 Rust 开源实现（发布于 [crates.io](https://crates.io/crates/fastalp)），针对现代 CPU 向量化与内存访问深度优化，具备自动向量化位解包与零堆分配特性。 + +#### 适用情况 + +ALP / fastalp 适用于绝大多数具有固定小数位数或物理量度含义的时序传感器、指标监控、金融行情与地理轨迹等浮点序列。 + +相较于 Gorilla，fastalp 在十进制浮点数据集上通常能大幅提升压缩比（平均提升 1.5x 至 3x 以上）；相较于 Pco，fastalp 提供了极高的编解码吞吐性能（单核编码吞吐可达 4.69 至 6.68 GB/s，单核解码吞吐超 30 GB/s）。在需要兼顾高压缩率与极高写入/查询扫描吞吐的时序列存场景下，ALP / fastalp 提供了出色的平衡。 + ### Pco 主要用于时间戳，整型，无符号整型以及浮点数。 diff --git a/i18n/en/docusaurus-plugin-content-docs/current/reference/concept_design/compress.md b/i18n/en/docusaurus-plugin-content-docs/current/reference/concept_design/compress.md index 1c10fc0ac..36ad7ad7c 100644 --- a/i18n/en/docusaurus-plugin-content-docs/current/reference/concept_design/compress.md +++ b/i18n/en/docusaurus-plugin-content-docs/current/reference/concept_design/compress.md @@ -46,6 +46,24 @@ Compared with the delta type, it is also suitable for time series data scenarios In the absence of specifying compression algorithms, we specify this compression algorithm for floating point type by default. +### ALP / fastalp + +Mainly used for single-precision (f32) and double-precision (f64) floating-point time-series data. + +#### Principle + +ALP (Adaptive Lossless floating-Point Compression) is a high-performance lossless compression algorithm designed for decimal floating-point data. In real-world time-series workloads (such as IoT sensors, industrial telemetry, system metrics, financial quotes, and GPS coordinates), most floating-point numbers originate from decimal representations. Traditional IEEE 754 XOR-based schemes (such as Gorilla) often suffer from mantissa bit randomness, yielding moderate compression ratios. On the other hand, general entropy codecs (such as Pco/tANS) offer high compression ratios but incur noticeable encoding and decoding throughput overhead. + +ALP adaptively samples blocks of floats to find the optimal decimal exponent and factor (10^e × factor), losslessly transforming floating-point values into integers. These integers are then compressed using Frame-of-Reference (FOR) or Delta-ALP, followed by vectorized SIMD bitpacking. For floating-point values that cannot be represented as exact decimals (such as infinite repeating decimals or non-decimal outliers), ALP stores them in an exception table using double-precision complements, ensuring 100% bit-exact lossless roundtrips. + +[fastalp](https://github.com/webc-site/wedb_embed/tree/main/fastalp) is an optimized pure Rust implementation of ALP available on [crates.io](https://crates.io/crates/fastalp), featuring auto-vectorized bit unpacking and zero heap allocations during hot encoding/decoding loops. + +#### Applicability + +ALP / fastalp is well suited for decimal-origin floating-point data across industrial IoT, system monitoring, telemetry, and financial time-series. + +Compared to Gorilla, fastalp typically achieves significantly higher compression ratios (often 1.5x to 3x or more on decimal datasets). Compared to Pco, fastalp delivers dramatically higher compression and decompression throughput (reaching 4.69 to 6.68 GB/s encoding throughput and over 30 GB/s decompression throughput on modern hardware). It provides an attractive balance for columnar time-series storage engines requiring both high compression ratios and high scan speeds. + ### Pco Mainly used for timestamps, integers, unsigned integers and floating points.