Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
26 changes: 26 additions & 0 deletions .github/workflows/turboquant-ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -62,6 +62,32 @@ jobs:
- name: Backend ops (setup + quantization paths)
run: ./build/bin/test-backend-ops -o FLASH_ATTN_EXT

# MSVC rejects things gcc and clang accept silently. A duplicate declaration of
# the TurboQuant vec_dot — once with GGML_API, once without — was a hard error
# here (C2375, different linkage) and invisible everywhere else. Cheap on a
# public repo, and it covers a compiler neither other job does.
windows:
name: Windows MSVC compile check
runs-on: windows-latest
steps:
- uses: actions/checkout@v4

- name: Configure
run: |
cmake -B build `
-DGGML_NATIVE=OFF `
-DLLAMA_BUILD_TESTS=ON `
-DLLAMA_BUILD_EXAMPLES=OFF `
-DLLAMA_BUILD_SERVER=OFF

# ggml-cpu is the target the linkage conflict broke; test-turboquant pulls
# in the quantization code itself and can actually run here.
- name: Build
run: cmake --build build --config Release --target ggml-cpu test-turboquant

- name: TurboQuant CPU reference tests
run: ./build/bin/Release/test-turboquant.exe

cuda:
name: CUDA compile check
runs-on: ubuntu-latest
Expand Down
1 change: 1 addition & 0 deletions ggml/src/ggml-cpu/ggml-cpu.c
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@
#include "traits.h"
#include "ggml-cpu-impl.h"
#include "ggml-impl.h"
#include "ggml-quants.h" // TurboQuant vec_dot lives in ggml-base
#include "quants.h"
#include "ggml-threading.h"
#include "unary-ops.h"
Expand Down
8 changes: 4 additions & 4 deletions ggml/src/ggml-cpu/quants.h
Original file line number Diff line number Diff line change
Expand Up @@ -42,10 +42,10 @@ void ggml_vec_dot_q5_0_q8_0(int n, float * GGML_RESTRICT s, size_t bs, const voi
void ggml_vec_dot_q5_1_q8_1(int n, float * GGML_RESTRICT s, size_t bs, const void * GGML_RESTRICT vx, size_t bx, const void * GGML_RESTRICT vy, size_t by, int nrc);
void ggml_vec_dot_q8_0_q8_0(int n, float * GGML_RESTRICT s, size_t bs, const void * GGML_RESTRICT vx, size_t bx, const void * GGML_RESTRICT vy, size_t by, int nrc);

// TurboQuant. Defined in ggml-quants.c (not arch-specific): dequantizes the
// left-hand side internally, so the right-hand side is plain f32.
void ggml_vec_dot_turbo3_0(int n, float * GGML_RESTRICT s, size_t bs, const void * GGML_RESTRICT vx, size_t bx, const void * GGML_RESTRICT vy, size_t by, int nrc);
void ggml_vec_dot_turbo4_0(int n, float * GGML_RESTRICT s, size_t bs, const void * GGML_RESTRICT vx, size_t bx, const void * GGML_RESTRICT vy, size_t by, int nrc);
// TurboQuant's vec_dot is declared in ggml-quants.h instead of here. It is the
// only one defined in ggml-quants.c (ggml-base) rather than in this library, so
// it needs GGML_API to cross the library boundary. Declaring it here as well —
// without that attribute — is a linkage conflict MSVC rejects outright.

void ggml_vec_dot_mxfp4_q8_0(int n, float * GGML_RESTRICT s, size_t bs, const void * GGML_RESTRICT vx, size_t bx, const void * GGML_RESTRICT vy, size_t by, int nrc);
void ggml_vec_dot_nvfp4_q8_0(int n, float * GGML_RESTRICT s, size_t bs, const void * GGML_RESTRICT vx, size_t bx, const void * GGML_RESTRICT vy, size_t by, int nrc);
Expand Down
Loading