test: lexer advanced unit tests and full integration test suite#50
Merged
Conversation
…es, and edge cases Covers scenarios not in test_lexer.py: - Multiline lineno/offset accuracy (blank lines, leading whitespace, tabs) - Block-comment state machine (/* ... */ skipping, multiline, unclosed behavior) - String-literal state machine (quoted tokens, empty string, offset) - user_state threading across tokens and per-lex() factory invocation - Edge cases: empty input, single-char position, LexingError with correct position Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…eline Defines a small expression language (integer literals, variables, arithmetic, boolean operators, comparisons, let binding, if/then/else) and exercises the complete Plare pipeline in 52 tests: - Multi-level operator precedence (or < and < not < cmp < +/- < */÷ < **) - Right-associative exponentiation: 2**3**2 == 512 - Unary minus via prec_token (UMINUS prec=6 < POW prec=7, so -2**2 == -4) - Let binding with shadowing and body extension - If/else with condition expressions and else-body extension - LexingError and ParsingError with position and expected-token metadata Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
Pytest ReportTest Results
Comment by ✨sambyeol/publish-pytest-action |
Pytest ReportTest Results
Comment by ✨sambyeol/publish-pytest-action |
Super-linter summary
All files and directories linted successfully For more information, see the GitHub Actions workflow run Powered by Super-linter |
Super-linter summary
All files and directories linted successfully For more information, see the GitHub Actions workflow run Powered by Super-linter |
There was a problem hiding this comment.
Pull request overview
This PR expands the project’s test suite by adding advanced unit tests for the lexer and an end-to-end integration suite that exercises both the lexer and LALR parser together via a realistic “Expr” DSL, with particular emphasis on position tracking and precedence/associativity behavior.
Changes:
- Add
tests/test_lexer_advanced.pyto cover multilinelineno/offsettracking, lexer state-machine transitions (block comments / string literals),user_statethreading, and LexingError position accuracy. - Add
tests/test_integration.pyto validate full lexer+parser pipelines (precedence, associativity,%prec-style override viaprec_token, boolean operators, let/if constructs) and error reporting.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated no comments.
| File | Description |
|---|---|
| tests/test_lexer_advanced.py | Adds targeted lexer unit tests for multiline position tracking, state-machine flows, user state threading, and error position reporting. |
| tests/test_integration.py | Adds an end-to-end Expr DSL integration suite validating lexer+parser behavior across precedence/associativity, control-flow constructs, and error cases. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
tests/test_lexer_advanced.py(22 tests): unit tests for lexerscenarios not covered by the existing suite — multiline lineno/offset
tracking, block-comment and string-literal state machines,
user_statethreading, and error-position accuracy
tests/test_integration.py(52 tests): end-to-end tests for arealistic "Expr" DSL built with both the lexer and the parser, covering
multi-level operator precedence, right-associative exponentiation, unary
negation via
prec_token, boolean operators, comparisons, let bindings,conditional expressions, and error reporting
Test plan
pytest tests/ -q)pyright tests/test_lexer_advanced.py tests/test_integration.py)blackreports files unchanged)