Skip to content

release/v1.1.2#5

Merged
jhjones-riverside merged 4 commits into
microhammerfrom
fix/tok-len-truncation
Mar 31, 2026
Merged

release/v1.1.2#5
jhjones-riverside merged 4 commits into
microhammerfrom
fix/tok-len-truncation

Conversation

@jhjones-riverside
Copy link
Copy Markdown
Collaborator

  • Fix token length truncation bug from Issue#3
  • Document change to prevent confusion over magic number
  • Add test to check that truncation bug is not possible

Copy link
Copy Markdown
Collaborator

@melbasiouny-riverside melbasiouny-riverside left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changes look good to me

@Lurk390 Lurk390 requested a review from Copilot March 31, 2026 19:53
@Lurk390 Lurk390 assigned Lurk390 and jhjones-riverside and unassigned Lurk390 Mar 31, 2026
@Lurk390 Lurk390 added the bug Something isn't working label Mar 31, 2026
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR bumps the project to v1.1.2 and addresses Issue#3 by preventing silent truncation of h_token() lengths, documenting the constraint, and adding a regression test around the failure mode.

Changes:

  • Bump VERSION and Debian changelog to 1.1.2.
  • Add an assertion/cast in h_token__m() intended to prevent token-length truncation.
  • Add a new test intended to ensure len > UINT8_MAX cannot silently truncate.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 3 comments.

File Description
VERSION Bumps library version to 1.1.2.
src/parsers/token.c Adds length guard + cast when building HToken.
tests/parsers/test_token.c Adds regression test for oversized token length behavior.
debian/changelog Adds 1.1.2-1 packaging entry documenting the fix/test.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread src/parsers/token.c Outdated
Comment on lines 79 to 85
// Length has to be <= 255 (uint8) as definied by HToken struct
assert(len <= UINT8_MAX);
HToken *t = h_new(HToken, 1);
uint8_t *str_cpy = h_new(uint8_t, len);
memcpy(str_cpy, str, len);
t->str = str_cpy, t->len = len;
t->str = str_cpy, t->len = (uint8_t)len;
return h_new_parser(mm__, &token_vt, t);
Copy link

Copilot AI Mar 31, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

h_token__m currently relies on assert(len <= UINT8_MAX) and then unconditionally casts len to uint8_t. If the library is compiled with NDEBUG (asserts compiled out), len > 255 will silently truncate again (e.g., 256 -> 0), reintroducing the original bug in release builds. Consider fixing this by either (a) storing the length as size_t in HToken and iterating with size_t, or (b) performing a non-assert runtime check and returning NULL/error when len > UINT8_MAX so truncation is impossible regardless of build flags.

Copilot uses AI. Check for mistakes.
Comment thread src/parsers/token.c Outdated
Comment on lines +79 to +80
// Length has to be <= 255 (uint8) as definied by HToken struct
assert(len <= UINT8_MAX);
Copy link

Copilot AI Mar 31, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Typo in the new comment: "definied" -> "defined". Also consider referencing UINT8_MAX in the comment to avoid introducing a new magic number ("255").

Copilot uses AI. Check for mistakes.
Comment on lines +41 to +57
// Test token.c: len not > UINT8_MAX assert
static void test_token_len_assert(gconstpointer backend) {
(void)backend;

if (g_test_subprocess()) {
uint8_t expected[256];
memset(expected, 0x41, sizeof(expected));

HParser *parser = h_token(expected, sizeof(expected));

// Shouldn't get here
(void)parser;
return;
}

g_test_trap_subprocess(NULL, 0, 0);
g_test_trap_assert_failed();
Copy link

Copilot AI Mar 31, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This test currently asserts that h_token aborts via assert for len == 256. That makes the test (and the bug fix) dependent on assertions being enabled; with NDEBUG, the subprocess may exit successfully and the test will fail while the truncation bug can still occur. Prefer testing a build-flag-independent behavior (e.g., h_token returns NULL for oversized lengths, or h_token supports lengths > 255 without truncation) and assert on that outcome instead of expecting an assertion failure.

Suggested change
// Test token.c: len not > UINT8_MAX assert
static void test_token_len_assert(gconstpointer backend) {
(void)backend;
if (g_test_subprocess()) {
uint8_t expected[256];
memset(expected, 0x41, sizeof(expected));
HParser *parser = h_token(expected, sizeof(expected));
// Shouldn't get here
(void)parser;
return;
}
g_test_trap_subprocess(NULL, 0, 0);
g_test_trap_assert_failed();
// Test token.c: ensure len == 256 is handled without truncation
static void test_token_len_assert(gconstpointer backend) {
HParserBackend be = (HParserBackend)GPOINTER_TO_INT(backend);
uint8_t expected[256];
memset(expected, 0x41, sizeof(expected));
HParser *parser = h_token(expected, sizeof(expected));
g_check_cmp_ptr(parser, !=, NULL);
h_compile(parser, be, NULL);
HParseResult *res = h_parse(parser, expected, sizeof(expected));
g_check_cmp_ptr(res, !=, NULL);
if (res) {
g_check_cmp_ptr(res->ast, !=, NULL);
if (res->ast && res->ast->token_type == TT_BYTES) {
g_check_cmp_uint(res->ast->bytes.len, ==, sizeof(expected));
}
h_parse_result_free(res);
}

Copilot uses AI. Check for mistakes.
@jhjones-riverside jhjones-riverside merged commit a1b7f49 into microhammer Mar 31, 2026
3 checks passed
@melbasiouny-riverside melbasiouny-riverside deleted the fix/tok-len-truncation branch April 17, 2026 18:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants