Skip to content

[PR] Stack Instrumentation Fixup#196

Open
elazaro-riverside wants to merge 60 commits into
mainfrom
stack-instrumentation-fixup
Open

[PR] Stack Instrumentation Fixup#196
elazaro-riverside wants to merge 60 commits into
mainfrom
stack-instrumentation-fixup

Conversation

@elazaro-riverside
Copy link
Copy Markdown
Collaborator

@elazaro-riverside elazaro-riverside commented Apr 16, 2026

This PR fixes inconsistencies with our stack instrumentation approach. During the E-BOSS PI meeting, we observed that the stack instrumentation was inconsistent for the challenge problems supplied by TA2.

The implementation before this PR, tried to pad each LLVM stack allocation (alloca) instruction with an additional byte to act as a guard bit between allocations. There was an effort to implement the padding using a struct with an additional i8 field but LLVM's optimizations would change the layout of the transformed alloca causing the instrumentation to be inconsistent. Changing the form of the alloca is dangerous because LLVM apply optimizations based on the shape and size of it. The solution we arrived at takes the original array type alloca and creates another alloca with [N + 1 x T] elements in it by creating a new alloca with a new shape. Getting this right was tricky but now it works. In the non-constant size case, we need to insert an add instruction after the size argument is zero-extended but before the allocation occurs.

We also realized in the challenge problems that Clang lowers VLA's (variable length arrays) into alloca's of a single type and a non-constant argument. This also caused issues for the instrumentation because we had not considered handling VLA's which live on the stack but have dynamic sizing.

An additional consideration, it is possible that an alloca is not executed along a particular control flow path even though it appears in the IR. To address this, Jackson proposed a per function pointer variable (shadow slot) initialized to null. After each instrumented alloca, the instrumentation inserts a store instruction to store the stack address. At the function epilogue, the instrumentation inserts load instruction to load the pointer from memory and pass it as an argument to __resolve_invalidate_stack to invalidate the associated stack region. If the variable remains null the call becomes a no-op, ensuring that no invalid stack deallocation occurs for alloca's that were never executed along the taken control flow path.

@elazaro-riverside elazaro-riverside self-assigned this Apr 16, 2026
@elazaro-riverside elazaro-riverside added the CVEAssert Related to CVEAssert component label Apr 16, 2026
@elazaro-riverside elazaro-riverside changed the title [DRAFT] Stack Instrumentation Fixup [PR] Stack Instrumentation Fixup Apr 16, 2026
@elazaro-riverside elazaro-riverside force-pushed the stack-instrumentation-fixup branch from 271bc1b to bf74fa1 Compare April 28, 2026 12:41
@elazaro-riverside elazaro-riverside marked this pull request as ready for review May 14, 2026 12:45
@elazaro-riverside elazaro-riverside marked this pull request as draft May 14, 2026 17:27
@elazaro-riverside elazaro-riverside marked this pull request as ready for review May 19, 2026 16:57
Ethan Lazaro added 2 commits May 19, 2026 13:41
…lude glaze_INSTALL command reflecting the latest change in the subproject.
…ed for shadow slots to skip instrumentation.
@elazaro-riverside elazaro-riverside added the enhancement New feature or request label May 21, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

CVEAssert Related to CVEAssert component enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

FIX: CVEAssert does not properly remediate an expected library; CVEAssert-041026-025335

1 participant