Fix npy writer shape buf#168
Merged
Merged
Conversation
PyTorch + C parity demo for a 1D-CNN reconstruction autoencoder on the UCR ECG5000 dataset. Training is filtered to class-1 normals only; at eval, reconstruction MSE acts as an anomaly score against the multi- class test set, with the threshold derived from training-set normals. First example to exercise Conv1dTransposed. Adds: - examples/ecg_anomaly_ae/ — prepare_data.py (download + parse ECG5000 into [N,1,140] .npy), train_pytorch.py (1D-CNN encoder/decoder), train_c.c (11-layer model wired through Conv1d/Conv1dTransposed, JSON log writer, post-training reconstruction writes), compare.py (MSE + AUC parity + plot emission), README, CMakeLists. - examples/_shared/plotting.py — plot_reconstructions and plot_anomaly_score_hist (with degenerate-bin guard). - examples/README.md mark Stage 2 done; examples/CMakeLists.txt wires the ecg_anomaly_ae subdir. The K=2 stride-2 decoder substitution (forced by Conv1dTransposed supporting only paddingType=VALID + outputPadding) slows convergence enough that spec section 4.2's 50 epochs is insufficient; EPOCHS=200 provides a safety margin past the expected test_mse around 0.05. Also folds in the I/O hardening (was PR #166) for both examples' train_c.c mains: 1. Self-bootstrap logs/ and outputs/ via an ensureDir helper that mkdir(2)'s with EEXIST treated as success. Previously those dirs were gitignored and implicitly created only by train_pytorch.py via Path.mkdir; running C-only used to fail with a cryptic fopen ENOENT. 2. Propagate npyWrite* failures to the program's exit code via a status accumulator; previously the error was printed to stderr then return 0 silently swallowed it. Verified end-to-end: ECG with logs/ and outputs/ removed re-creates both dirs and exits 0; both examples' compare.py parity PASS.
writeNpy's shape_buf[256] is filled by snprintf calls whose remaining-size argument `sizeof(shape_buf) - shape_len` unsigned-underflowed when shape_len exceeded 256 (e.g., ndim around 120+, or smaller with multi- digit dims). The underflowed huge size_t bypassed snprintf's bounds check and allowed OOB writes — caught by UBSan at npy_writer.c:22 as "index 257 out of bounds for type 'char[256]'". Fix: extract appendBounded helper that vsnprintf's into a fixed buffer with explicit bounds checking (returns -1 on truncation, error, or already-past-end offset). All four shape_buf append sites route through it; a single rc=6 is returned if any indicates truncation. Regression test builds an ASan+UBSan harness with ndim=200, asserts no sanitizer trip and rc != 0. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.