Extended testing fixes, simplified np.array reading, citation file#30
Open
hgp297 wants to merge 20 commits into
Open
Extended testing fixes, simplified np.array reading, citation file#30hgp297 wants to merge 20 commits into
hgp297 wants to merge 20 commits into
Conversation
- explicitly checking comp_population headers for 'story' and 'dir' rather than excluding by index
- High level metrics in compare_runs.evaluate_tolerances now considers path if either relative difference < 4% OR absolute difference < 2 days - test_integration.py ungates ASSERT_POINTWISE - renamed metrics writeup to README.md so that it displays in github in the test folder - Added a section in README.md to describe what is considered passing in Mat-Py testing
- added utility file with functions to batch run folder in testing directory
These values correspond to those set at the Jan 22 2026 standup meeting - MAE lowered to 1% - P95 lowered to 2%
Zaidi et al. (2026) is the citation for the package, but the preferred citation is Cook et al. (2022).
- driver.py now has recursive np conversion to traverse nested arrays without pre-specifying levels. - outputting new damage_consequences as consequences.json (contains red tag info)
switched nested functionality array-to-list conversion to use the recursive method
This reverts commit a51a500. System-level point-wise thresholds reverted to P95 <=4% and MAE <= 2%.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Post Jun 5th meeting updates
RCSW_1storyandhaseltonRCMF_12storyResults
Passed 17/18 integration tests. Failed test had Metric 1 = 2.1 > 2.0 and Metric 2 < 4.0.