Skip to content

fix(Storage): remove static-init-order dependency from Storage construction#1073

Open
IvanaGyro wants to merge 4 commits into
masterfrom
codex/fix-673-storage-init
Open

fix(Storage): remove static-init-order dependency from Storage construction#1073
IvanaGyro wants to merge 4 commits into
masterfrom
codex/fix-673-storage-init

Conversation

@IvanaGyro

@IvanaGyro IvanaGyro commented Jul 16, 2026

Copy link
Copy Markdown
Member

Problem

Closes #673.

Storage::Init (inline, in the header) dispatched dtype construction through Storage_init_interface::USIInit[dtype], a static function-pointer table populated by the constructor of the global Storage_init_interface __SII object defined in src/backend/Storage.cpp. Because Storage::Init only touches the static table member — not __SII itself — a translation unit that instantiates Storage(size) without also referencing anything else from Storage.cpp (e.g. operator<<) has no guarantee that __SII's constructor has run before USIInit[dtype] is read: a static-initialization-order fiasco. When it hasn't run, USIInit[dtype] is still zero-initialized and Cytnx calls a null function pointer, segfaulting.

Fix

  • Remove Storage_init_interface, __SII, and the USIInit table entirely — there is no more implicit global-initializer side effect to depend on.
  • Replace it with init_storage(dtype), a plain function that folds over Type_list (skipping void) to find and construct the StorageImplementation<T> matching the requested dtype, mirroring the existing as_storage_variant_impl pattern already used in Storage.cpp. This keeps the supported dtype set single-sourced from Type_list instead of a hand-populated table.
  • Update Storage::Init, Storage::_Loadbinary/the file-load path, and the GPU move/copy paths in cuMovemem_gpu.cu to call init_storage(dtype).
  • Templatize Storage::_from_vector so callers that already know the element type T construct StorageImplementation<T> directly, instead of also routing through the (removed) runtime dtype table.
  • Remove now-unused declarations and commented-out scaffolding tied to the old interface.

Testing

  • ctest on debug-openblas-cpu and debug-mkl-cpu: all Storage suites pass (288/288); one pre-existing, unrelated ASan failure in BlockUniTensorTest.CombineBondForceDoesNotRewriteUserHeldBond reproduces identically on the pre-change base.
  • pytest (Storage / dtype / mixed-dtype-arithmetic / rank-zero suites) passes against the rebuilt Python bindings.
  • debug-openblas-cuda: compiles and links (cuMovemem_gpu.cu change) — compile-check only, no GPU available in this environment.
  • pre-commit run clang-format (pinned v14) clean.

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request refactors the Storage initialization mechanism by replacing the legacy Storage_init_interface class and its global instance with a static array named storage_init_fns. This change simplifies the initialization logic and removes redundant code across multiple files, including updates to Tensor_impl and various backend implementations. The reviewer recommends using the inline keyword for the storage_init_fns array in the header file to ensure a single instance across translation units and prevent potential code bloat.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

Comment thread include/backend/Storage.hpp Outdated
Comment on lines +502 to +504
static StorageInitFn storage_init_fns[N_Type] = {
nullptr, SIInit_cd, SIInit_cf, SIInit_d, SIInit_f, SIInit_i64,
SIInit_u64, SIInit_i32, SIInit_u32, SIInit_i16, SIInit_u16, SIInit_b};

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Using static for storage_init_fns at namespace scope in a header file will create a separate copy of the array in each translation unit that includes this header. While this may work, it can lead to unnecessary code bloat.

Since the project targets C++20, you can use inline to ensure there is only one instance of this array across the entire program. This is the modern and correct way to define a global variable in a header file since C++17.

Suggested change
static StorageInitFn storage_init_fns[N_Type] = {
nullptr, SIInit_cd, SIInit_cf, SIInit_d, SIInit_f, SIInit_i64,
SIInit_u64, SIInit_i32, SIInit_u32, SIInit_i16, SIInit_u16, SIInit_b};
inline StorageInitFn storage_init_fns[N_Type] = {
nullptr, SIInit_cd, SIInit_cf, SIInit_d, SIInit_f, SIInit_i64,
SIInit_u64, SIInit_i32, SIInit_u32, SIInit_i16, SIInit_u16, SIInit_b};

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The solution was already posted at #673 (comment)

Comment thread include/backend/Storage.hpp Outdated
@codecov

codecov Bot commented Jul 16, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 94.73684% with 1 line in your changes missing coverage. Please review.
✅ Project coverage is 72.61%. Comparing base (f64b2f5) to head (d202f5f).
⚠️ Report is 45 commits behind head on master.
✅ All tests successful. No failed tests found.

Files with missing lines Patch % Lines
include/backend/Storage.hpp 94.11% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master    #1073      +/-   ##
==========================================
+ Coverage   72.11%   72.61%   +0.49%     
==========================================
  Files         225      226       +1     
  Lines       28207    28103     -104     
  Branches       71       71              
==========================================
+ Hits        20341    20406      +65     
+ Misses       7845     7676     -169     
  Partials       21       21              
Flag Coverage Δ
cpp 72.72% <94.73%> (+0.50%) ⬆️
python 64.13% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

Components Coverage Δ
C++ backend 71.80% <94.73%> (+0.23%) ⬆️
Python bindings 77.75% <ø> (+1.95%) ⬆️
Python package 64.13% <ø> (ø)
Files with missing lines Coverage Δ
include/backend/Tensor_impl.hpp 97.12% <ø> (ø)
src/backend/Storage.cpp 69.08% <100.00%> (ø)
src/backend/Storage_base.cpp 27.84% <ø> (ø)
src/backend/Tensor_impl.cpp 86.95% <ø> (ø)
include/backend/Storage.hpp 97.24% <94.11%> (-1.66%) ⬇️

... and 19 files with indirect coverage changes


Continue to review full report in Codecov by Harness.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update f64b2f5...d202f5f. Read the comment docs.

🚀 New features to boost your workflow:
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@ianmccul

Copy link
Copy Markdown
Collaborator

This is an improvement over Storage_init_interface: it removes the dynamic
initialization bug from #673 and exposes the mechanism for what it is. But now
that #1015 has added Type_list, StorageImplementation<T>, and the typed
storage machinery, I think the proper endpoint is to remove the initializer
table entirely, rather than replace it with another hand-written enumeration
of the supported dtypes.

There should be two construction paths.

For code where T is already known, construct the typed implementation
directly:

template <CytnxType T>
boost::intrusive_ptr<StorageImplementation<T>>
make_storage(cytnx_uint64 size, int device, bool init_zero = true) {
  auto out = boost::intrusive_ptr<StorageImplementation<T>>(
    new StorageImplementation<T>());
  out->Init(size, device, init_zero);
  return out;
}

Eventually StorageImplementation<T> should have a suitable constructor and
the separate Init() call can disappear. This typed path should be used by
_from_vector<T>, typed kernels, and GPU helpers such as
cuMovemem_gpu<T>. Those callers already know T, so routing them back
through an integer dtype and a function-pointer table throws away the type
information unnecessarily.

A runtime-dtype factory is still needed for the public
Storage(size, dtype, device) API, Python, and deserialization. Generate that
factory from Type_list, which should be the single source of truth:

namespace internal {

template <std::size_t... Is>
boost::intrusive_ptr<Storage_base>
make_storage_impl(unsigned int dtype, cytnx_uint64 size, int device,
                  bool init_zero, std::index_sequence<Is...>) {
  boost::intrusive_ptr<Storage_base> out;

  const bool matched = (... || [&] {
    using T = std::variant_alternative_t<Is + 1, Type_list>;  // skip void
    if (dtype != Type_class::cy_typeid_v<T>) return false;

    out = make_storage<T>(size, device, init_zero);
    return true;
  }());

  cytnx_error_msg(!matched, "[Storage] invalid dtype %u.%s", dtype, "\n");
  return out;
}

}  // namespace internal

inline boost::intrusive_ptr<Storage_base>
make_storage(unsigned int dtype, cytnx_uint64 size, int device,
             bool init_zero = true) {
  constexpr std::size_t count = std::variant_size_v<Type_list> - 1;
  return internal::make_storage_impl(
    dtype, size, device, init_zero, std::make_index_sequence<count>{});
}

Then Storage::Init becomes simply:

this->_impl = make_storage(dtype, size, device, init_zero);

This removes:

  • Storage_init_interface;
  • __SII;
  • SIInit_cd, SIInit_cf, etc.;
  • storage_init_fns;
  • the additional manually maintained enumeration of every supported dtype.

The fold above is one possible implementation; a compile-time-generated
function table would also work. The important points are that Type_list is
the only enumeration of supported types, and that typed callers do not go
through runtime dtype dispatch.

Until StorageImplementation<void> becomes the canonical empty state, the
runtime factory should reject Type.Void explicitly rather than leave a null
entry that can be called.

Comment thread include/backend/Storage.hpp Outdated
@@ -976,57 +935,57 @@ namespace cytnx {
}

void _from_vector(const std::vector<cytnx_complex128> &vin, const int device = -1) {

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

templatize these function, just let

p = new StorageImeplementation<T>(); // and Init
this->_impl = boost::intrusive_ptr(p)

@IvanaGyro

Copy link
Copy Markdown
Member Author

I prefer to make this PR simple. There is no many function use this init function. I will preserve function table for the code that need large scope refactoring to use typed constructor directly. The places that are easy to templatize will be refactored.

IvanaGyro and others added 2 commits July 17, 2026 18:52
Collapse the eleven per-dtype _from_vector overloads (and the unused
generic fallback that only raised a runtime error) into a single function
template that constructs StorageImplementation<T> directly instead of
routing through the runtime storage_init_fns dispatch table. Callers of
_from_vector already know the element type T, so the integer-dtype lookup
threw the type information away only to recover it.

The template is constrained with a static_assert on CytnxType<T>, turning
an unsupported element type into a compile-time error at the call site
rather than the previous runtime "[FATAL] ERROR unsupport type". The
bit-packed std::vector<bool> case keeps its element-wise _cpy_bool copy
via an if constexpr branch; every other dtype uses a contiguous memcpy
from vin.data().

Co-Authored-By: Claude <noreply@anthropic.com>
Unify the eleven SIInit_cd/SIInit_cf/... free functions and the
storage_init_fns[N_Type] pointer table into a single init_storage(dtype)
factory. The dispatch is generated by folding over Type_list (skipping
void at index 0), mirroring as_storage_variant_impl, so the supported
dtype set has one source of truth instead of a hand-maintained enumeration
that must be kept in lockstep with Type_list. This also drops the
StorageInitFn typedef.

Runtime-dtype callers (Storage::Init, Storage::_Loadbinary and the load
path in Storage.cpp, and cuMovemem_gpu) now call init_storage(dtype).
Unlike the old table, which stored a nullptr at the Type.Void slot that
would be called on an out-of-range access, the fold raises
cytnx_error_msg on any dtype without a StorageImplementation, Void
included.

Add <utility> and <variant> includes for std::index_sequence and the
std::variant_alternative_t / std::variant_size_v the fold relies on.

Co-Authored-By: Claude <noreply@anthropic.com>
@IvanaGyro IvanaGyro changed the title Replace Storage initialization interface with static function table fix(Storage): remove static-init-order dependency from Storage construction Jul 17, 2026
Guard the behavior of init_storage(dtype), the replacement for the
Storage_init_interface/__SII table that #673 reported segfaulting on.
Construct a Storage of every supported dtype (the exact repro from the
issue for the default Double case, plus a round-tripped representative
value -- including negatives, fractional values, complex, and both Bool
states -- for every other dtype) and check size()/dtype()/value against
independently known expectations rather than one Cytnx path against
another.

Verified this test also passes unmodified against the pre-#1073 code
(Storage_init_interface/__SII, checked out at f64b2f5): the segfault was a
cross-translation-unit static-initialization-order dependency that only
manifests in a fresh C++ process, and pytest already runs inside a fully
loaded cytnx extension module where every global object's constructor has
necessarily already run. So this test cannot reproduce the original
crash and only guards the dispatch behavior of the new code, which the
module docstring states explicitly. A C++ test that can reproduce the
original ordering-dependent failure is tracked in a follow-up issue,
deferred until #1080 (test-suite reorganization) merges.

Co-Authored-By: Claude <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Segmentation fault when initializing Storage in C++

2 participants