fix : apricot test_generated comment bash error - #22
Conversation
|
Automated Code Review Comment: Bugs
Improvements
Suggestions
Code looks good ✅ Suggested Testsdef test_empty_diff():
assert True |
|
added bash for tests to be written in |
haddybhaiya
left a comment
There was a problem hiding this comment.
waiting for approvals , please check carefully!
There was a problem hiding this comment.
Pull request overview
This PR aims to make LLM-generated tests cleaner before they are written/commented by the agent, mainly by normalizing fenced output and adding focused tests for that normalization path.
Changes:
- Added
_normalize_generated_testsand routedgenerate_tests()through it to strip code fences and leadingpytestcommand lines. - Added targeted tests for normalization behavior in a new test module.
- Changed the PR comment formatting for generated tests and replaced the checked-in
tests/test_generated.pycontents.
Reviewed changes
Copilot reviewed 4 out of 4 changed files in this pull request and generated 2 comments.
| File | Description |
|---|---|
tests/test_test_generator_formatting.py |
Adds unit tests for normalization of generated test output. |
tests/test_generated.py |
Replaces prior contents with a trivial placeholder test. |
agent/main.py |
Changes how suggested tests are rendered in the final PR comment. |
agent/llm/test_generator.py |
Introduces output normalization and updates the LLM prompt rules. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
|
|
||
| # 9. Combine output | ||
| final_output = f"{review}\n\n---\n\n### Suggested Tests\n{tests}" | ||
| final_output = f"{review}\n\n---\n\n### Suggested Tests\n```bash\n{tests}\n```" |
| def test_empty_diff(): | ||
| assert True No newline at end of file |
pleasingsunlight
left a comment
There was a problem hiding this comment.
This PR aims at improving formatting for tests via introducing new functions to normalize tests and confirm the same via test_test_generated_formatting.py .
Upon proper review of the changes made, the intention is fulfilled as seen from the comment by 'github-actions-bot' ;
Tho there is still room for improvement here, especially when it comes to test coverage, preventing over-stripping of valid lines.
It is safe to merge the fixes to main.
- A few more tries would serve to better assess the improved formatting of the agent.
Hence for now :
We ball ! 🏐 😄 ~~~
|
Everthing seems fine to me go ahead and merge this pr |
deoxyforge
left a comment
There was a problem hiding this comment.
Go ahead everything looks good
This pull request improves the formatting and reliability of generated test code by introducing a normalization function to clean up LLM outputs, updating prompt instructions, and adding dedicated tests for the new normalization logic.
Test Output Normalization:
_normalize_generated_testsintest_generator.pyto strip markdown code fences, language specifiers, and shell command lines (likepytest ...) from LLM-generated test code. This ensures only valid Python test code is returned.generate_testsfunction to use_normalize_generated_testsinstead of manual string manipulation.Prompt and Output Formatting:
generate_teststo explicitly instruct the LLM to avoid markdown code fences and shell commands in its output.main.py, changed the final output format to wrap suggested tests in a bash code block, improving readability.Testing:
test_test_generator_formatting.pyto verify that_normalize_generated_testscorrectly removes code fences and bash headers, and preserves valid Python code.