fix(linalg): register missing Int16/Uint16/Bool rows in CPU Outer dispatch (fixes segfault) by yingjerkao · Pull Request #1099 · Cytnx-dev/Cytnx

yingjerkao · 2026-07-20T04:38:44Z

Problem

linalg::Outer segfaults for Int16, Uint16, or Bool inputs on the CPU.

Outer.cpp promotes both operands to a common dtype and dispatches through
Outer_ii[out_dtype][out_dtype]. The CPU Outer_ii dispatch table in
linalg_internal_interface.cpp was never populated with the Int16, Uint16,
or Bool source rows — every other source dtype has a full 11-entry row, but
these three had none. So when the promoted dtype is Int16/Uint16/Bool,
Outer_ii[out_dtype][out_dtype] is a null function pointer, and the call jumps
to address 0:

AddressSanitizer: SEGV on unknown address 0x000000000000 (pc 0x000000000000 ...)
    #1 ... TestBody tests/linalg_test/Outer_test.cpp:32   // the linalg::Outer(a, b) call

Every sibling dispatch table in the same constructor — MM_ii, Sum_ii,
Diag_ii, Matmul_ii, Matvec_ii, and even the GPU cuOuter_ii — already
registers these dtypes, so the omission is an oversight, not intent. (The GPU
path works; this is CPU-only.)

Reproduces on master (86cb96c). Minimal repro:

import cytnx
a = cytnx.zeros([5], cytnx.Type.Int16); b = cytnx.zeros([7], cytnx.Type.Int16)
cytnx.linalg.Outer(a, b)   # SIGSEGV

Fix

Register the 33 missing rows (Int16, Uint16, Bool sources × 11 target
dtypes). The Outer_internal_{i16,u16,b}t* implementations already exist and
are declared — this is registration only, no kernel or algorithm change.

Testing

New tests/linalg_test/Outer_test.cpp (OuterDtypeCoverage: Int16,
Uint16, Bool, Int16xBool) checks independently hand-computed values
(dtype and value, with negative and boolean cases).
Regression discipline: on the pre-fix binary these crash with the ASan
SEGV above at the linalg::Outer call site; post-fix all four pass.
Existing DtypePromotion.OuterComplexfloatDouble and LinalgKronTest tests
still pass. clang-format (v14) clean.

…patch linalg::Outer segfaulted for Int16, Uint16, or Bool CPU inputs. The CPU Outer_ii dispatch table in linalg_internal_interface.cpp was never populated with the Int16/Uint16/Bool *source* rows (every other source dtype has a full 11-entry row; these three had none). Outer.cpp casts both operands to the promoted dtype and dispatches through Outer_ii[out_dtype][out_dtype]; when out_dtype is Int16/Uint16/Bool that entry was a null function pointer, so the call jumped to address 0 and crashed. Every sibling table (MM_ii, Sum_ii, Diag_ii, Matmul_ii, Matvec_ii, and GPU cuOuter_ii) already registers these dtypes, so the omission was an oversight, not intent. Fix: register the 33 missing rows. The Outer_internal_{i16,u16,b}t* implementations already exist and are declared -- this is registration only, no kernel or algorithm change. Testing: new tests/linalg_test/Outer_test.cpp (OuterDtypeCoverage: Int16, Uint16, Bool, Int16xBool) checks independently hand-computed values. On the pre-fix binary these crash with an ASan SEGV at address 0x0 (null function-pointer call) at the linalg::Outer call site; post-fix all four pass. Existing DtypePromotion.OuterComplexfloatDouble and LinalgKronTest tests still pass. clang-format (v14) clean. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

gemini-code-assist

Code Review

This pull request registers the missing Int16, Uint16, and Bool source rows in the CPU Outer_ii dispatch table to prevent null function pointer dereferences and crashes when using these types as the left operand in linalg::Outer. It also adds comprehensive regression tests in tests/linalg_test/Outer_test.cpp to verify correctness across these data types. The review feedback points out several instances in the new test file where legacy cytnx_uint64 typedefs are used as loop counters, which violates the repository style guide's preference for standard C++ types like size_t or uint64_t in new code.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

gemini-code-assist · 2026-07-20T04:39:57Z

+        for (cytnx_uint64 i = 0; i < 3; i++)
+          for (cytnx_uint64 j = 0; j < 2; j++)


According to the repository style guide, legacy cytnx_XXXX typedefs (such as cytnx_uint64) should not be propagated as ordinary loop counters in new or touched code. Standard C++ types like size_t or uint64_t should be preferred instead.

Suggested change

for (cytnx_uint64 i = 0; i < 3; i++)

for (cytnx_uint64 j = 0; j < 2; j++)

for (size_t i = 0; i < 3; i++)

for (size_t j = 0; j < 2; j++)

References

For new or touched code, avoid propagating legacy cytnx_XXXX typedefs as ordinary loop counters, sizes, flags, or local arithmetic types. Prefer standard C++ types unless the value is specifically a dtype-backed tensor/storage scalar. ^(link)

gemini-code-assist · 2026-07-20T04:39:58Z

+        for (cytnx_uint64 i = 0; i < 2; i++)
+          for (cytnx_uint64 j = 0; j < 3; j++)


According to the repository style guide, legacy cytnx_XXXX typedefs (such as cytnx_uint64) should not be propagated as ordinary loop counters in new or touched code. Standard C++ types like size_t or uint64_t should be preferred instead.

Suggested change

for (cytnx_uint64 i = 0; i < 2; i++)

for (cytnx_uint64 j = 0; j < 3; j++)

for (size_t i = 0; i < 2; i++)

for (size_t j = 0; j < 3; j++)

References

For new or touched code, avoid propagating legacy cytnx_XXXX typedefs as ordinary loop counters, sizes, flags, or local arithmetic types. Prefer standard C++ types unless the value is specifically a dtype-backed tensor/storage scalar. ^(link)

gemini-code-assist · 2026-07-20T04:39:58Z

+        for (cytnx_uint64 i = 0; i < 2; i++)
+          for (cytnx_uint64 j = 0; j < 3; j++)


According to the repository style guide, legacy cytnx_XXXX typedefs (such as cytnx_uint64) should not be propagated as ordinary loop counters in new or touched code. Standard C++ types like size_t or uint64_t should be preferred instead.

Suggested change

for (cytnx_uint64 i = 0; i < 2; i++)

for (cytnx_uint64 j = 0; j < 3; j++)

for (size_t i = 0; i < 2; i++)

for (size_t j = 0; j < 3; j++)

References

For new or touched code, avoid propagating legacy cytnx_XXXX typedefs as ordinary loop counters, sizes, flags, or local arithmetic types. Prefer standard C++ types unless the value is specifically a dtype-backed tensor/storage scalar. ^(link)

gemini-code-assist · 2026-07-20T04:39:58Z

+        for (cytnx_uint64 i = 0; i < 2; i++)
+          for (cytnx_uint64 j = 0; j < 2; j++)


According to the repository style guide, legacy cytnx_XXXX typedefs (such as cytnx_uint64) should not be propagated as ordinary loop counters in new or touched code. Standard C++ types like size_t or uint64_t should be preferred instead.

Suggested change

for (cytnx_uint64 i = 0; i < 2; i++)

for (cytnx_uint64 j = 0; j < 2; j++)

for (size_t i = 0; i < 2; i++)

for (size_t j = 0; j < 2; j++)

References

For new or touched code, avoid propagating legacy cytnx_XXXX typedefs as ordinary loop counters, sizes, flags, or local arithmetic types. Prefer standard C++ types unless the value is specifically a dtype-backed tensor/storage scalar. ^(link)

codecov · 2026-07-20T04:55:37Z

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 72.77%. Comparing base (86cb96c) to head (3a71c72).
✅ All tests successful. No failed tests found.

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #1099      +/-   ##
==========================================
+ Coverage   72.71%   72.77%   +0.05%     
==========================================
  Files         226      226              
  Lines       28151    28184      +33     
  Branches       71       71              
==========================================
+ Hits        20471    20510      +39     
+ Misses       7659     7653       -6     
  Partials       21       21

Flag	Coverage Δ
cpp	`72.89% <100.00%> (+0.05%)`	⬆️
python	`64.13% <ø> (ø)`

Flags with carried forward coverage won't be shown. Click here to find out more.

Components	Coverage Δ
C++ backend	`72.00% <100.00%> (+0.06%)`	⬆️
Python bindings	`77.70% <ø> (ø)`
Python package	`64.13% <ø> (ø)`

Files with missing lines	Coverage Δ
src/backend/linalg_internal_interface.cpp	`100.00% <100.00%> (ø)`

... and 1 file with indirect coverage changes

Continue to review full report in Codecov by Harness.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 86cb96c...3a71c72. Read the comment docs.

🚀 New features to boost your workflow:

📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

IvanaGyro

The comments from bots are worth to be resolved. Except that LGTM.

ianmccul · 2026-07-20T08:24:20Z

Outer.cpp casts both operands to out_dtype, then calls Outer_ii[out_dtype][out_dtype]. Only the three missing diagonal entries are reachable; the other 30 registrations are harmless but dead. Likewise, Int16TimesBool does not exercise Outer_ii[Int16][Bool].

I'm wondering why this PR doesn't replace the whole mechanism by a much simpler implementation. src/linalg/Outer.cpp:24 rejects either operand unless shape().size() == 1. There is only a Tensor overload. So:

Tensor Outer(const Tensor &lhs, const Tensor &rhs) {
  cytnx_error_msg(lhs.is_void(), "[ERROR] lhs must be initialized.%s", "\n");
  cytnx_error_msg(rhs.is_void(), "[ERROR] rhs must be initialized.%s", "\n");
  cytnx_error_msg(lhs.device() != rhs.device(),
                  "[ERROR] the two tensors must be on the same device.%s", "\n");
  cytnx_error_msg(lhs.rank() != 1, "[ERROR] tensor #1 should have rank-1.%s", "\n");
  cytnx_error_msg(rhs.rank() != 1, "[ERROR] tensor #2 should have rank-1.%s", "\n");

  return Kron(lhs, rhs).reshape(std::vector<cytnx_uint64>{lhs.shape()[0], rhs.shape()[0]});
}

ianmccul

see top-level comment above

yingjerkao · 2026-07-20T12:11:05Z

Addressing @ianmccul's review comment: rather than register the missing Int16/Uint16/Bool rows, #1105 reimplements Outer as Kron(a, b).reshape({m, n}) and retires the per-dtype Outer_ii/cuOuter_ii dispatch entirely. Kron's variant-based dispatch already covers those dtype pairs on CPU and GPU, so the missing-row segfault this PR fixed can no longer occur. This PR is superseded by #1105.

ianmccul

hmm, not sure how to 'unsubmit' approval but it doesn't matter, I assume this PR is no longer relevant

yingjerkao · 2026-07-22T01:55:08Z

Closing as superseded by #1105.

This PR registered the missing Outer_ii[Int16/Uint16/Bool][…] rows in the CPU Outer dispatch table to fix a segfault on those source dtypes. Since it was approved, #1105 landed and reworked Outer as Kron(a, b).reshape(...), deleting the Outer_ii/Outerfunc_oii/cuOuter_ii dispatch tables entirely.

As a result, on current master:

The segfault is already fixed — Outer now routes through Kron, whose variant-based type_promote_from_pointer_t dispatch covers every dtype pair, so there's no null dispatch entry to crash on. Verified: Outer(Int16/Uint16/Bool, Double) and Outer(Bool, Bool) all return correct results.
The dispatch registration this PR adds has nowhere to go — the table it patches no longer exists.
The dtype coverage is already tested — tests/linalg_test/Outer_test.cpp on master (added by refactor(linalg): implement Outer as Kron+reshape, retire Outer dispatch (#1003) #1105) exercises the Int16/Uint16/Bool cases.

Nothing here is left to salvage, so closing rather than rebasing. Thanks!

gemini-code-assist Bot reviewed Jul 20, 2026

View reviewed changes

yingjerkao mentioned this pull request Jul 20, 2026

test(linalg): GPU correctness coverage for Kron and Outer (#1003, confirms #999) #1100

Closed

IvanaGyro approved these changes Jul 20, 2026

View reviewed changes

yingjerkao mentioned this pull request Jul 20, 2026

refactor(linalg): typed GPU dispatch for Outer + retire Kron GPU promotion trait (#1003) #1101

Closed

yingjerkao requested review from IvanaGyro and ianmccul July 20, 2026 08:02

IvanaGyro approved these changes Jul 20, 2026

View reviewed changes

ianmccul requested changes Jul 20, 2026

View reviewed changes

yingjerkao mentioned this pull request Jul 20, 2026

refactor(linalg): implement Outer as Kron+reshape, retire Outer dispatch (#1003) #1105

Merged

ianmccul approved these changes Jul 21, 2026

View reviewed changes

ianmccul reviewed Jul 21, 2026

View reviewed changes

ianmccul self-requested a review July 21, 2026 09:50

yingjerkao closed this Jul 22, 2026

yingjerkao mentioned this pull request Jul 22, 2026

test(linalg): strengthen GPU Kron/Outer coverage (salvaged from #1100) #1108

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

fix(linalg): register missing Int16/Uint16/Bool rows in CPU Outer dispatch (fixes segfault)#1099

fix(linalg): register missing Int16/Uint16/Bool rows in CPU Outer dispatch (fixes segfault)#1099
yingjerkao wants to merge 1 commit into
masterfrom
fix/outer-16bit-bool-dispatch

yingjerkao commented Jul 20, 2026

Uh oh!

gemini-code-assist Bot left a comment

Uh oh!

gemini-code-assist Bot Jul 20, 2026

Uh oh!

gemini-code-assist Bot Jul 20, 2026

Uh oh!

gemini-code-assist Bot Jul 20, 2026

Uh oh!

gemini-code-assist Bot Jul 20, 2026

Uh oh!

codecov Bot commented Jul 20, 2026 •

edited

Loading

Uh oh!

IvanaGyro left a comment

Uh oh!

ianmccul commented Jul 20, 2026 •

edited

Loading

Uh oh!

ianmccul left a comment

Uh oh!

yingjerkao commented Jul 20, 2026

Uh oh!

ianmccul left a comment

Uh oh!

yingjerkao commented Jul 22, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

		for (cytnx_uint64 i = 0; i < 3; i++)
		for (cytnx_uint64 j = 0; j < 2; j++)

		for (cytnx_uint64 i = 0; i < 2; i++)
		for (cytnx_uint64 j = 0; j < 3; j++)

Uh oh!

Conversation

yingjerkao commented Jul 20, 2026

Problem

Fix

Testing

Uh oh!

gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

Code Review

Uh oh!

gemini-code-assist Bot Jul 20, 2026

Choose a reason for hiding this comment

Uh oh!

gemini-code-assist Bot Jul 20, 2026

Choose a reason for hiding this comment

Uh oh!

gemini-code-assist Bot Jul 20, 2026

Choose a reason for hiding this comment

Uh oh!

gemini-code-assist Bot Jul 20, 2026

Choose a reason for hiding this comment

Uh oh!

codecov Bot commented Jul 20, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

IvanaGyro left a comment

Choose a reason for hiding this comment

Uh oh!

ianmccul commented Jul 20, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ianmccul left a comment

Choose a reason for hiding this comment

Uh oh!

yingjerkao commented Jul 20, 2026

Uh oh!

ianmccul left a comment

Choose a reason for hiding this comment

Uh oh!

yingjerkao commented Jul 22, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

codecov Bot commented Jul 20, 2026 •

edited

Loading

ianmccul commented Jul 20, 2026 •

edited

Loading