Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
33 changes: 33 additions & 0 deletions .clang-format
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
---
BasedOnStyle: Google
IndentWidth: 4
---
Language: Cpp
AlignAfterOpenBracket: BlockIndent
AlignEscapedNewlines: Left
AlignTrailingComments: true
AllowAllArgumentsOnNextLine: true
AllowShortBlocksOnASingleLine: Always
AllowShortCaseLabelsOnASingleLine: false
AllowShortEnumsOnASingleLine: true
AllowShortFunctionsOnASingleLine: All
AllowShortIfStatementsOnASingleLine: Never
AllowShortLambdasOnASingleLine: All
AllowShortLoopsOnASingleLine: false
BinPackArguments: false
BinPackParameters: false
BreakConstructorInitializers: BeforeComma
ColumnLimit: 92
CompactNamespaces: false
DerivePointerAlignment: false
FixNamespaceComments: true
IncludeBlocks: Regroup
IndentCaseLabels: true
KeepEmptyLinesAtTheStartOfBlocks: false
PointerAlignment: Left
ReferenceAlignment: Left
ReflowComments: true
SortIncludes: CaseSensitive
Standard: c++17
UseTab: Never
---
30 changes: 30 additions & 0 deletions .clang-tidy
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
Checks: >
-*,
clang-analyzer-*,
bugprone-branch-clone,
bugprone-infinite-loop,
bugprone-macro-parentheses,
bugprone-macro-repeated-side-effects,
bugprone-narrowing-conversions,
bugprone-sizeof-expression,
bugprone-suspicious-*,
bugprone-use-after-move,
misc-redundant-expression,
misc-static-assert,
misc-throw-by-value-catch-by-reference,
modernize-use-noexcept,
modernize-use-nullptr,
modernize-use-override,
modernize-use-using,
performance-*,
portability-*,
readability-duplicate-include,
readability-inconsistent-declaration-parameter-name,
readability-misleading-indentation,
readability-redundant-control-flow,
-portability-simd-intrinsics,
-clang-analyzer-security.insecureAPI.DeprecatedOrUnsafeBufferHandling

# Keep the selected first-party baseline regression-free. Repository-wide
# naming checks remain deferred because they would change public APIs.
WarningsAsErrors: "*"
27 changes: 27 additions & 0 deletions .editorconfig
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
root = true

[*]
charset = utf-8
end_of_line = lf
insert_final_newline = true
trim_trailing_whitespace = true

[*.{c,cc,cpp,cxx,h,hpp,cu,cuh,cmake,py,pyi}]
indent_style = space
indent_size = 4

[CMakeLists.txt]
indent_style = space
indent_size = 4

[*.{json,yaml,yml}]
indent_style = space
indent_size = 2

[*.md]
indent_style = space
indent_size = 2
trim_trailing_whitespace = false

[Makefile]
indent_style = tab
15 changes: 15 additions & 0 deletions .github/workflows/python.yml
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,21 @@ concurrency:
cancel-in-progress: true

jobs:
python-quality:
name: Python formatting and linting
runs-on: ubuntu-latest
timeout-minutes: 10
steps:
- uses: actions/checkout@v4
- uses: actions/setup-python@v5
with:
python-version: "3.11"
cache: pip
- name: Install Ruff
run: python -m pip install "ruff==0.16.1"
- name: Check Python
run: ./scripts/check-python.sh

python-build:
name: Python ${{ matrix.python-version }}
runs-on: ubuntu-latest
Expand Down
52 changes: 52 additions & 0 deletions .github/workflows/test.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,58 @@ concurrency:
group: ${{ github.head_ref || github.sha }}-${{ github.workflow }}
cancel-in-progress: true
jobs:
format:
name: C++ formatting
runs-on: ubuntu-latest
timeout-minutes: 10
steps:
- uses: actions/checkout@v4
- name: Install clang-format
run: |
sudo apt-get update -qq
sudo apt-get install -y clang-format-15
- name: Check formatting
run: ./scripts/check-format.sh

clang-tidy:
name: C++ static analysis
runs-on: ubuntu-latest
timeout-minutes: 30
steps:
- uses: actions/checkout@v4
- uses: actions/setup-python@v5
with:
python-version: "3.11"
- name: Install dependencies
run: |
sudo apt-get update -qq
sudo apt-get install -y clang-tidy-15 libomp-15-dev cmake ninja-build
python -m pip install "numpy>=1.23" "pybind11>=2.12"
- name: Locate pybind11
run: echo "pybind11_DIR=$(python -m pybind11 --cmakedir)" >> "$GITHUB_ENV"
- name: Configure compilation database
run: >-
cmake -S . -B build-tidy -G Ninja
-DRABITQ_BUILD_TESTS=ON
-DRABITQ_BUILD_PYTHON_BINDINGS=ON
-DCMAKE_BUILD_TYPE=Release
-Dpybind11_DIR="$pybind11_DIR"
- name: Run clang-tidy
run: ./scripts/check-tidy.sh build-tidy

shellcheck:
name: Shell scripts
runs-on: ubuntu-latest
timeout-minutes: 10
steps:
- uses: actions/checkout@v4
- name: Install ShellCheck
run: |
sudo apt-get update -qq
sudo apt-get install -y shellcheck
- name: Check scripts
run: shellcheck scripts/*.sh

ubuntu:
name: Ubuntu
runs-on: ubuntu-latest
Expand Down
2 changes: 2 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -42,6 +42,8 @@ results/
*.pyc
__pycache__/
.clang-*
!.clang-format
!.clang-tidy
.clangd*

# macOS garbages
Expand Down
9 changes: 9 additions & 0 deletions .pre-commit-config.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
repos:
- repo: local
hooks:
- id: clang-format
name: clang-format 15
entry: scripts/apply-format.sh
language: system
files: '\.(c|cc|cpp|cxx|h|hpp|cu|cuh)$'
exclude: '^(include/rabitqlib/third/|include/rabitqlib/utils/fht_avx\.hpp$)'
121 changes: 121 additions & 0 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,121 @@
# Contributing to RaBitQ

Thank you for contributing to RaBitQ. Before submitting a pull request, build
the library, run the relevant tests, and check the C++ formatting.

## C++ formatting

RaBitQ uses the repository's `.clang-format` configuration and clang-format
15. Install that version on Ubuntu or Debian with:

```bash
sudo apt-get install clang-format-15
```

Format all project-maintained C and C++ files:

```bash
./scripts/apply-format.sh
```

Verify formatting without changing files:

```bash
./scripts/check-format.sh
```

The scripts intentionally exclude vendored Eigen code and the imported FFHT
implementation. To use a nonstandard executable name, set `CLANG_FORMAT`; it
must still identify itself as clang-format 15.

clangd embeds its own formatter, so use clangd 15 in editors such as VS Code
if format-on-save must exactly match CI. If another clangd version is required,
disable format-on-save and run the repository scripts before submitting.

To format only lines changed in the staged files, use:

```bash
./scripts/format-changed.sh --staged
```

The complete-file formatter remains useful before the initial formatting pass
or after changing `.clang-format`; CI always checks complete files.

## Optional pre-commit hook

Install [pre-commit](https://pre-commit.com/) and enable the repository hook:

```bash
python -m pip install pre-commit
pre-commit install
```

The hook formats only staged C and C++ files. CI runs the read-only formatting
check over the complete project-maintained source tree.

## Static analysis

clang-tidy performs semantic checks and is kept separate from clang-format.
The required baseline contains focused correctness, portability, and
performance checks. Install the pinned analyzer and the dependencies needed to
configure every first-party target:

```bash
sudo apt-get install clang-tidy-15 libomp-15-dev cmake ninja-build
python -m pip install "numpy>=1.23" "pybind11>=2.12"
```

Then configure the same tests and Python bindings analyzed by CI and run the
check with the same compiler used by CMake:

```bash
export pybind11_DIR="$(python -m pybind11 --cmakedir)"
CXX=c++ cmake -S . -B build-tidy -G Ninja \
-DRABITQ_BUILD_TESTS=ON \
-DRABITQ_BUILD_PYTHON_BINDINGS=ON \
-DCMAKE_BUILD_TYPE=Release \
-Dpybind11_DIR="$pybind11_DIR"
CXX=c++ ./scripts/check-tidy.sh build-tidy
```

The wrapper supplies clang-tidy with that compiler's standard-library include
paths and reports diagnostics only for first-party files. Vendored Eigen,
hnswlib, and the imported FFHT implementation are excluded. New checks should
be added incrementally after their existing first-party findings are fixed.

## Python formatting and linting

Python sources, examples, and tests use Ruff 0.16.1:

```bash
python -m pip install "ruff==0.16.1"
./scripts/check-python.sh
```

To apply Python formatting and safe automatic lint fixes before running the
check:

```bash
ruff check --fix python python_bindings sample/python tests/python
ruff format python python_bindings sample/python tests/python
```

## Shell scripts

Run ShellCheck after changing a contributor or automation script:

```bash
sudo apt-get install shellcheck
shellcheck scripts/*.sh
```

## Performance and compatibility

- Use fixed-width integer types for serialized values and persisted index data.
- Preserve existing public headers, aliases, and index formats unless a change
is explicitly documented as breaking.
- Add backend-independent tests when introducing or changing SIMD kernels.
- Keep scalar, AVX2, and AVX-512 implementations behaviorally equivalent.
- Benchmark allocations or algorithm changes in search and quantization hot
paths, and include the commands and results in the pull request.
- Avoid unrelated refactoring or formatting in performance-sensitive changes.
5 changes: 5 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -127,6 +127,11 @@ GoogleTest is downloaded during test configuration. For a full benchmark on
the GIST dataset, see [`example.sh`](example.sh). More detailed API and
algorithm guidance is available in the [documentation](docs/docs/index.md).

## Contributing

Contributions are welcome. See the [contributing guide](CONTRIBUTING.md) for
the build, formatting, pre-commit, and static-analysis workflows.

## Why RaBitQ?

- **High accuracy with tiny codes.** RaBitQ provides state-of-the-art
Expand Down
2 changes: 1 addition & 1 deletion include/rabitqlib/defines.hpp
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@

#include "rabitqlib/third/Eigen/Dense"

#define BIT_ID(x) (__builtin_popcount((x) - 1))
#define BIT_ID(x) (__builtin_popcount((x)-1))
#define LOWBIT(x) ((x) & (-(x)))

namespace rabitqlib {
Expand Down
20 changes: 17 additions & 3 deletions include/rabitqlib/fastscan/fastscan.hpp
Original file line number Diff line number Diff line change
Expand Up @@ -37,9 +37,23 @@ constexpr static std::array<int, 16> kPos = {
}; // all possible combination for a 4 bit string

constexpr static std::array<int, 16> kPerm0 = {
0, 8, 1, 9, 2, 10, 3, 11, 4, 12, 5, 13, 6, 14, 7, 15
}; // data order of packed quantization code, please refer to code and the link offered
// above for detailed information
0,
8,
1,
9,
2,
10,
3,
11,
4,
12,
5,
13,
6,
14,
7,
15}; // data order of packed quantization code, please refer to code and the link
// offered above for detailed information

template <typename T, class TA>
static inline void get_column(
Expand Down
Loading
Loading