Skip to content

fix: Stop the iterator reading one limb past the end - #6

Merged
chenpeizhi merged 2 commits into
masterfrom
fix/iterator-past-the-end-read
Aug 15, 2026
Merged

fix: Stop the iterator reading one limb past the end#6
chenpeizhi merged 2 commits into
masterfrom
fix/iterator-past-the-end-read

Conversation

@chenpeizhi

Copy link
Copy Markdown
Contributor

Two defects in the header, both present since long before the C++20 migration.

The iterator reads one limb past the end

Fbitset::const_iterator::get_next advances the limb pointer and then dereferences it, and only afterwards re-tests it against the sentinel:

while (curr_ < last_ && limb_ == 0) {
    ++curr_;
    limb_ = *curr_;      // curr_ may already be last_ here
    base_ += LIMB_BITS;
}

When the final limb is exhausted, ++curr_ makes curr_ == last_ and the read goes one limb past the end of the storage. The value read is discarded immediately, because the loop condition then fails, so iteration results are correct. The read itself is still undefined behaviour, and it is reached by every complete walk over a bit set, so it happens constantly.

Found while running libparenth under the address sanitizer:

ERROR: AddressSanitizer: stack-buffer-overflow
    #0 fbitset::Fbitset<1ul, unsigned long long, fbitset::No_ext>::const_iterator::get_next() fbitset.hpp:531
    #1 ... ::const_iterator::operator++() fbitset.hpp:517
    #2 libparenth::Parenther<...>::form_chunks(...) libparenth.hpp:818

The fix reads the limb only after the bound check.

The header cannot be included from two translation units

is_no_ext is a variable template whose explicit specialization was declared without inline:

template <typename T> constexpr bool is_no_ext = false;
template <> constexpr bool is_no_ext<No_ext> = true;

An explicit specialization does not inherit the internal linkage that constexpr gives the primary template, so it is emitted in every translation unit that includes the header. Any program including fbitset.hpp twice fails to link:

duplicate symbol 'fbitset::internal::is_no_ext<fbitset::No_ext>' in:
    .../nonclassical.cpp.o
    .../matrixchain.cpp.o

For a header-only library that is a hard restriction on how it can be used. Nothing had hit it because every consumer so far included the header from exactly one translation unit. Both declarations are now inline.

Tests

test/multi_tu.cpp is a second translation unit including the header, which covers the linkage. It also walks a bit set to exhaustion in every supported configuration, which is the path that used to read past the end.

The out-of-bounds read is invisible to a plain test run, since the value is discarded. So there is also a new CI job that builds the tests with the address and undefined behaviour sanitizers. Reverting only the get_next change makes that job fail, and the plain jobs still pass, which is exactly why the job is needed.

Effect on results

None. This was checked by fingerprinting the full output of the libparenth search (every intermediate, its parenthesization and its cost) over 500 randomly generated contraction problems in all six mode and inclusivity combinations, before and after. All 2861 configurations that completed give byte-identical results.

`Fbitset::const_iterator::get_next` advanced the limb pointer and then
dereferenced it before checking it against the sentinel.  When the last
limb was exhausted, that read one limb past the end of the storage.  The
value read is discarded right away, so the iteration result is correct,
but the read itself is undefined behaviour, and it faults under the
address sanitizer.  It is reached by any complete walk over a bit set, so
it happens constantly.  Read the limb only after the bound check.

The explicit specialization of `is_no_ext` also needed to be `inline`.
Without it the specialization has external linkage and is emitted in
every translation unit including the header, so a program including this
header from two translation units failed to link with a duplicate symbol.
For a header-only library that is a hard restriction, and it is what the
new second test translation unit ran into first.

Add that second translation unit as a test, which both covers the linkage
and walks every supported configuration to exhaustion, and add a CI job
building the tests with the address and undefined behaviour sanitizers.
The out-of-bounds read is invisible to the plain test run, so without the
sanitizer job nothing would catch it coming back.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Fixes long-standing undefined behavior in Fbitset iteration (one-past-the-end limb read) and resolves a header-only ODR/linkage issue with a variable-template specialization. Adds a multi-translation-unit test and a sanitizer CI job to reliably catch the previously invisible UB.

Changes:

  • Fix Fbitset::const_iterator::get_next() to avoid dereferencing curr_ after it reaches the sentinel.
  • Make internal::is_no_ext (primary + explicit specialization) inline constexpr to prevent duplicate symbols across translation units.
  • Add a second-TU test and a new GitHub Actions job that runs tests under ASan/UBSan.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 1 comment.

File Description
include/fbitset.hpp Fixes iterator UB and makes is_no_ext safe for header-only multi-TU inclusion.
test/multi_tu.cpp Adds coverage for multi-TU header inclusion and iterator exhaustion paths.
test/CMakeLists.txt Builds the new second translation unit into the test executable.
.github/workflows/ci.yml Adds a sanitizer job to detect UB that regular test runs won’t observe.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread test/multi_tu.cpp Outdated
Comment on lines +21 to +27
namespace fbitset {
template <Size N, typename Int, typename Cont>
std::ostream& operator<<(std::ostream& os, const Fbitset<N, Int, Cont>& bs)
{
return os << "{Fbitset}";
}
}

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Right on both counts, and dropped in e7793a0.

This translation unit never checks an Fbitset directly, so it does not need the stream operator at all. test/basic.cpp keeps its copy, which is now the only one.

Two copies of the same template in separate translation units is exactly the hazard this pull request is about, so removing one is better than keeping them in step by hand. If a third unit ever needs it, the definition should move to a shared test header rather than be copied again.

Tests still pass in both the plain and the sanitizer builds.

Addressing review feedback.

`test/basic.cpp` defines a stream operator for `Fbitset` so that Catch2
does not try to treat it as a range.  The new translation unit repeated
that definition, but it never checks an `Fbitset` directly, so it does not
need one.  Two copies of the same template in different translation units
is exactly the kind of hazard this pull request is about, so drop the
copy rather than keep it in step by hand.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@chenpeizhi
chenpeizhi merged commit 0a21596 into master Aug 15, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants