Skip to content

testbench1.cpp: undefined behavior in signed saturation reference code for testcases 610 and 612 #1

Description

@oltolm

The reference implementation for compress_saturated in testbench1.cpp is incorrect for signed destination types in testcases 610 and 612.

It computes the saturated result with:

(RT(1) << (sizeof(RT) * 8 - 1)) - (elements1[i] > 0)

This has undefined behavior for signed RT because it left-shifts into the sign bit, and for positive overflow cases it can overflow again when subtracting 1.

As a result, the testbench reference code can produce incorrect expected results even when compress_saturated itself is correct.

Expected behavior

For signed narrowing saturation, the reference code should clamp to std::numeric_limits<RT>::max() for positive overflow and std::numeric_limits<RT>::min() for negative overflow.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions