Skip to content

fix undefined behavior from signed int overflow#30

Merged
Daivuk merged 1 commit into
Daivuk:masterfrom
c-f-h:int-overflow-bugfix
May 28, 2026
Merged

fix undefined behavior from signed int overflow#30
Daivuk merged 1 commit into
Daivuk:masterfrom
c-f-h:int-overflow-bugfix

Conversation

@c-f-h

@c-f-h c-f-h commented May 28, 2026

Copy link
Copy Markdown
Contributor

The Doom code uses unsigned BAM angles and relies on unsigned wraparound in its angle arithmetic. However, PureDOOM has the line

offsetangle = (angle_t)doom_abs((int)rw_normalangle - (int)rw_angle1);

This casts the angles to signed integers and then does the subtraction. This is problematic because signed integer overflow is undefined behavior in C, and some compilers will emit incorrect code here. The correct code is the pattern already used a few pages further down in the segtextured section:

offsetangle = rw_normalangle - rw_angle1;
if (offsetangle > ANG180)
    offsetangle = -offsetangle;   // offsetangle = abs(rw_normalangle - rw_angle1)

This is not just academic, it produces actual rendering artifacts under some compilers which this PR fixes:

image

@Daivuk Daivuk merged commit 6727d79 into Daivuk:master May 28, 2026
6 of 10 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants