fix(cuda): avoid NVCC/cudafe++ crashes on Windows when processing MSVC STL multi-variant visitation.

[IvanaGyro]

Base and downstream topology

This PR is a single commit directly on master.

The later Windows PRs still need the unmerged #1109 and #1110 changes. Their integration history is preserved by codex/windows-msvc-cuda-stack, which points at the former combined #1116 tip and contains the same #1116 patch:

#1109 ? #1110 ? codex/windows-msvc-cuda-stack ? #1117 ? #1118 ? #1119 ? #1113

#1117 is based on that integration anchor. The master-based patch and the original stacked patch have the same stable Git patch ID (025bfb1307f96ff9f4977e7137cde1a8f7358e97).

Summary

• map host std::complex types to their CUDA-native cuda::std::complex storage type ids;
• replace the 11?11 two-variant std::visit expansion with dtype-indexed template dispatch;
• preserve the existing promotion, comparison, scalar, and non-contiguous kernel paths while avoiding CUDA 13 cudafe++ crashes with MSVC.

This is a Windows CUDA compiler-portability fix. It is not a blocker for Linux CUDA or Windows CPU builds. It is the only split PR with mixed-dtype implementation changes and requires focused human review.

Why Linux succeeds

A successful openblas-cuda or debug-openblas-cuda build on Linux is expected. Linux runs nvcc with a GCC/Clang host toolchain; the failing path here is CUDA 13.3's Windows front end processing MSVC's large std::visit instantiation. Linux builds are important regression coverage, but they do not exercise the MSVC-specific failure.

Windows A/B reproduction

The parent and fixed revisions were tested with:

• Visual Studio 2022 MSVC host compiler 19.44.35228;
• PyPI CUDA/nvcc 13.3.73, provisioned through the Pixi cuda environment;
• CMake 3.31.8 with Ninja and C++/CUDA 20;
• USE_CUDA=ON, USE_CUTENSOR=ON, USE_CUQUANTUM=OFF, USE_MKL=ON;
• BUILD_PYTHON=OFF, USE_HPTT=OFF, and IPO disabled for the focused compile;
• CMAKE_CUDA_ARCHITECTURES=75 for the A/B reproduction;
• target CMakeFiles/cytnx.dir/src/backend/linalg_internal_gpu/cuAdd_internal.cu.obj.

Results:

1. On fix(cuda): avoid NVCC/cudafe++ crashes on Windows when processing MSVC STL multi-variant visitation. #1116's original stacked parent (1fc237fa), the object fails because host std::complex<float/double> is not present in Type_list_gpu, then the CUDA front end terminates:

   include/Type.hpp(152): error: static assertion failed with
   "variant_index<T, Variant>: T is not in Variant"
   nvcc error: 'cudafe++' died with status 0xC0000005 (ACCESS_VIOLATION)
   

2. Applying only the host-complex ? CUDA-complex mapping removes both static assertions, but the unchanged two-variant dispatcher still ends with the same cudafe++ access violation.

3. With the complete fix(cuda): avoid NVCC/cudafe++ crashes on Windows when processing MSVC STL multi-variant visitation. #1116 change, incremental MSVC/CUDA 13 compilation succeeds for cuAdd_internal.cu.obj, cuiArithmetic_dispatch.cu.obj, and cuCpr_dispatch.cu.obj using the project's full CUDA architecture list.

NVCC template-instantiation trace

The parent compile was also rerun with CUDA 13.3's --ftemplate-backtrace-limit=0. For both host complex types, NVCC emits the complete lookup chain:

variant_index<cytnx_complex128, variant<>>
-> variant_index<cytnx_complex128, variant<each Type_list_gpu alternative...>>
-> variant_index_v<cytnx_complex128, Type_list_gpu>
-> Type_struct_t<cytnx_complex128>
-> make_type_array_helper<Type_list>(index_sequence<0..11>)
-> make_type_array<Type_list>()

The same chain is emitted for cytnx_complex64. After applying only the complex-type mapping, those diagnostics disappear completely, but cudafe++ still terminates with 0xC0000005; the crashing front end emits no instantiation backtrace for the separate std::visit failure.
This isolates the type mapping and dispatch rewrite as separate necessary parts of the Windows CUDA compiler workaround.

Linux and macOS

macOS does not compile the UNI_GPU path. Linux CUDA compile/tests are the non-Windows regression gate; successful Linux openblas-cuda and debug-openblas-cuda builds have been reported, and fresh PR checks will run from the new master-based head.

Validation

• parent-vs-partial-vs-full A/B reproduction documented above;
• incremental MSVC/CUDA 13 compilation passes for all three affected objects;
• the replayed master-based patch matches the original patch ID;
• repository-pinned pre-commit hooks and clang-format 14 pass.

Part of #1114.

[gemini-code-assist]

Caution

The consumer version of Gemini Code Assist on GitHub has been sunset. All code review activity has officially ceased.

[gemini-code-assist]

Caution

The consumer version of Gemini Code Assist on GitHub has been sunset. All code review activity has officially ceased.

[IvanaGyro]

Why this fails on Windows/MSVC but succeeds on Linux/GCC

I investigated the failure path and the relevant standard-library implementations. The most accurate conclusion is:

This is not because MSVC lacks std::visit or a required C++20 language feature. There are two separate issues: a real Cytnx GPU-complex type-mapping bug, followed by an NVIDIA cudafe++ crash triggered by the shape of Microsoft STL's multi-variant std::visit implementation.

The distinction matters because the first part is a correctness fix, while the dispatch rewrite is a CUDA-compiler portability workaround.

1. The original GPU complex type-id lookup is objectively wrong

Cytnx's logical dtype list contains host complex types:

using Type_list = std::variant<
  void,
  std::complex<double>,
  std::complex<float>,
  ...>;

whereas Type_list_gpu contains CUDA-native complex types:

using Type_list_gpu = std::variant<
  void,
  cuda::std::complex<double>,
  cuda::std::complex<float>,
  ...>;

The previous definition nevertheless performed:

variant_index_v<T, Type_list_gpu>

for an ordinary Cytnx dtype T. Therefore, for T = std::complex<double>, it effectively asks for:

variant_index_v<
  std::complex<double>,
  std::variant<void, cuda::std::complex<double>, ...>>

Those are different C++ types, so the lookup must eventually reach the empty-variant base case and fire:

variant_index<T, Variant>: T is not in Variant

The new gpu_element_type_t<T> mapping is therefore not merely a workaround for MSVC. It is the semantically correct mapping:

std::complex<float>  -> cuda::std::complex<float>
std::complex<double> -> cuda::std::complex<double>
all other types      -> unchanged

The PR's A/B experiment is especially useful here:

1. the parent produces the variant_index assertion and then cudafe++ crashes;
2. applying only the host-complex → CUDA-complex mapping removes the assertion, but cudafe++ still crashes;
3. applying the full dispatch rewrite makes the affected CUDA objects compile.

That isolates the type mapping and the std::visit crash as two independently necessary fixes.

2. Linux success does not prove the old lookup was valid

A class-template specialization does not necessarily instantiate the definitions/initializers of all of its static data members merely because the enclosing class is instantiated. The C++ implicit-instantiation rules explicitly distinguish instantiating member declarations from instantiating their definitions:

• https://eel.is/c++draft/temp.inst

Consequently, an invalid dependent initializer can remain latent until some compilation path actually requires it. The fact that the Linux CUDA build did not diagnose this member does not make the lookup correct; it means that the Linux NVCC/GCC/libstdc++ compilation path did not require the same instantiation at that point.

I would avoid attributing this solely to g++ versus cl.exe, because .cu compilation is mediated by NVCC's proprietary CUDA front end before the generated host code is passed to the host compiler. The observable fact is that the Windows CUDA compilation path instantiated the bad member, while the Linux path did not.

3. The second failure is not a normal MSVC diagnostic: cudafe++ itself crashes

After the type mapping is corrected, the remaining failure is:

nvcc error: 'cudafe++' died with status 0xC0000005 (ACCESS_VIOLATION)

0xC0000005 means the compiler process accessed invalid memory. That is a compiler crash, not a standards-conformance diagnostic. Regardless of whether a source construct is difficult or even ill-formed, the compiler should report a diagnostic rather than terminate with an access violation.

NVCC's documented compilation trajectory separates CUDA device code from host code, processes the CUDA source through NVIDIA compilation stages, and later invokes the platform host compiler (cl.exe on Windows, GCC on Linux) for host compilation:

• https://docs.nvidia.com/cuda/cuda-compiler-driver-nvcc/index.html
• https://docs.nvidia.com/cuda/cuda-programming-guide/02-basics/nvcc.html

CUDA 13.3 officially supports Visual Studio 2022/MSVC 193x and C++20, so this is not an unsupported language-mode/toolchain combination:

• https://docs.nvidia.com/cuda/cuda-installation-guide-microsoft-windows/index.html

Therefore the narrow diagnosis is:

CUDA 13.3's Windows cudafe++ front end crashes while processing this large Microsoft-STL-generated visitation instantiation.

It is not evidence that MSVC lacks std::variant, variadic std::visit, generic lambdas, if constexpr, or another required C++20 feature.

4. Why Microsoft STL produces a particularly difficult template graph here

Each StorageVariant has 11 alternatives. A two-variant visitor must be valid for all:

11 × 11 = 121

ordinary type pairs.

For each pair, the visitor reaches a templated CUDA launch path containing up to five kernel forms:

• scalar/scalar;
• scalar/tensor;
• tensor/scalar;
• contiguous tensor/tensor;
• non-contiguous tensor/tensor.

That gives up to:

121 × 5 = 605

kernel specializations for one operation-code instantiation, before counting the standard-library dispatch machinery itself.

Microsoft STL's current std::visit implementation computes the total dispatch-state count as:

(size_t{1} * ... * (variant_size_v<Variants> + 1))

The +1 reserves a state for valueless_by_exception. For two 11-alternative variants, the dispatch machinery therefore sees:

(11 + 1) × (11 + 1) = 144 states

Because 144 is greater than 64 and no greater than 256, Microsoft STL selects _Visit_strategy<4>, whose implementation stamps out a switch capable of handling 256 states. It also constructs a compile-time Cartesian product of index lists and checks the visitor's result across all ordinary alternative combinations.

Relevant Microsoft STL implementation points:

• _Variant_total_states
• _Meta_cartesian_product
• _Visit_strategy<4>
• _Variant_all_visit_results_same
• visit

Source:

• https://github.com/microsoft/STL/blob/main/stl/inc/variant

Thus cudafe++ is not merely seeing the 121 Cytnx type pairs. It is parsing a much larger AST containing Microsoft STL's Cartesian-product metaprogramming, result checks, dispatch wrappers, and a stamped switch, with each valid leaf instantiating a CUDA kernel-launch family.

5. Why GCC/libstdc++ does not trigger the same crash

libstdc++ implements the general multi-variant case differently. It recursively constructs a multidimensional function-pointer table (_Multi_array / __gen_vtable_impl) whose dimensions are the actual variant sizes, and dispatches with:

auto function = vtable._M_access(variants.index()...);
return (*function)(visitor, variants...);

It checks valueless_by_exception() before entering the ordinary visitation table. For this case, the table shape is therefore conceptually 11 × 11 rather than Microsoft STL's 12 × 12 state encoding followed by a 256-state switch strategy.

Relevant libstdc++ implementation points:

• _Multi_array
• __gen_vtable_impl
• __gen_vtable
• __do_visit

Source:

• https://github.com/gcc-mirror/gcc/blob/master/libstdc%2B%2B-v3/include/std/variant

Both implementations are permitted by the C++ standard. std::visit specifies observable behavior, not a required dispatch implementation. The difference is the compile-time structure presented to cudafe++:

Standard library	General two-variant dispatch structure
Microsoft STL	Cartesian-product index metadata + valueless states + stamped switch strategy
libstdc++	Recursive multidimensional function-pointer table

The Linux build succeeding therefore does not imply better language-feature support in GCC. It indicates that libstdc++ generates a different template/AST shape that does not trigger this Windows cudafe++ failure.

6. Why the dtype-indexed recursive dispatch succeeds

The replacement dispatch still instantiates the complete 11 × 11 type matrix:

dispatch_lhs<LIndex>()
  -> dispatch_rhs<TL, RIndex>()
       -> launch<TO, TL, TR>()

Therefore it preserves:

• exhaustive compile-time type coverage;
• the existing promotion rules;
• the same CUDA-native to_cuda_t boundary;
• scalar, contiguous, and non-contiguous launch paths;
• checked erased-to-typed storage recovery via storage_cast.

What it removes is the standard-library visitation machinery surrounding those leaf instantiations:

• no Microsoft STL Cartesian-product type list;
• no valueless dispatch states;
• no _Variant_dispatcher wrapper for every state;
• no 256-case stamped switch;
• no multi-variant return-type metaprogramming path.

The new template graph is two shallow, predictable chains of if constexpr recursion. It does not reduce the set of Cytnx type combinations, but it is substantially easier for the CUDA front end to lower.

7. Recommended characterization of this PR

I would describe the two changes separately:

1. Correctness fix: map logical host std::complex<T> dtypes to CUDA-native cuda::std::complex<T> before looking them up in Type_list_gpu.
2. Compiler-portability workaround: replace the two-variant std::visit in CUDA translation units with dtype-indexed template dispatch to avoid a CUDA 13.3 Windows cudafe++ access violation triggered by Microsoft STL's large visitation implementation.

A technically precise summary would be:

The old code contains a latent host-complex/CUDA-complex type-id mismatch. After correcting that mismatch, the otherwise valid two-variant visitation still triggers an NVIDIA cudafe++ compiler crash on Windows because Microsoft STL lowers this 11 × 11 visitation through a large Cartesian-product/switch template structure. libstdc++ uses a different multidimensional jump-table implementation, so Linux does not exercise the same compiler failure. The recursive dtype dispatcher preserves exhaustive typed dispatch while avoiding that problematic standard-library instantiation.

The existing PR title is understandable shorthand, but comments and release notes should ideally say "CUDA/NVCC Windows front-end crash with the MSVC host toolchain/STL" rather than implying that cl.exe or MSVC's C++ feature set is incomplete.

A reduced reproducer could later be filed with NVIDIA, but I do not think that should block this patch: a compiler access violation is sufficient justification for using the simpler, semantically equivalent dispatch structure here.

[IvanaGyro]

Follow-up after the unlimited template-instantiation backtrace

The newly added --ftemplate-backtrace-limit=0 trace resolves the one point that was previously uncertain: what causes the invalid cy_typeid_gpu initializer to be instantiated.

The trace shows this chain for both host complex types:

Type_class::Typeinfos
-> make_type_array<Type_list>()
-> make_type_array_helper<Type_list>(index_sequence<0..11>)
-> Type_struct_t<cytnx_complex128>::construct()
-> Type_struct_t<cytnx_complex128>
-> variant_index_v<cytnx_complex128, Type_list_gpu>
-> ...
-> variant_index<cytnx_complex128, variant<>>

and equivalently for cytnx_complex64.

This means the first diagnostic is not triggered by the two-variant std::visit. It occurs while constructing the compile-time Typeinfos array from the ordinary host Type_list.

More importantly, Type_struct_t<T>::construct() does not read cy_typeid_gpu:

static constexpr Type_struct construct() {
  return {name, enum_name, is_unsigned, is_complex, is_float, is_int, typeSize};
}

The class specialization contains cy_typeid_gpu, but that static data member is not needed to evaluate construct() or Typeinfos.

Under the C++ implicit-instantiation rule, instantiating a class-template specialization instantiates the declarations, but not the definitions, of its static data members. See [temp.inst]/3.1:

The implicit instantiation of a class template specialization causes the implicit instantiation of the declarations, but not of the definitions, of ... static data members ...

https://eel.is/c++draft/temp.inst

Therefore, the complete trace supports a stronger and more precise diagnosis than my earlier wording:

In this Windows CUDA compilation mode, NVIDIA's front end eagerly instantiates the dependent initializer/definition of Type_struct_t<T>::cy_typeid_gpu while instantiating Type_struct_t<T> for Typeinfos, even though that static member is not required by construct(). That is frontend-specific over-instantiation and appears inconsistent with the standard delayed-instantiation rule.

The invalid initializer remains a real latent defect in Cytnx:

variant_index_v<std::complex<double>, Type_list_gpu>

cannot succeed because Type_list_gpu contains cuda::std::complex<double>, not std::complex<double>. The new host-complex → CUDA-complex mapping is consequently still the correct robustness/correctness fix. However, the reason the defect becomes observable specifically here is now known: it is reached through Typeinfos because the Windows CUDA front end instantiates more than construct() actually requires.

The A/B result then keeps the two failures cleanly separated:

1. Parent: eager instantiation exposes the invalid GPU complex ID lookup; cudafe++ subsequently crashes.
2. Mapping-only patch: the variant_index diagnostics disappear, proving that chain is fixed, but cudafe++ still crashes without an instantiation trace.
3. Full patch: replacing the two-variant Microsoft-STL std::visit structure removes the independent access violation.

So the refined attribution is:

• the static assertions come from a latent Cytnx type-mapping bug exposed by NVIDIA frontend over-instantiation through Typeinfos;
• the later 0xC0000005 is a separate NVIDIA frontend crash triggered by the large MSVC-STL multi-variant visitation graph;
• neither failure indicates that MSVC lacks the required C++20 features.

[IvanaGyro]

I am not sure if compiler will optimize this to a function table. Maybe using two-fold visit is a better choice.