feat: add extractor-based, non-strict parsing by aschierl-xitaso · Pull Request #693 · csaf-rs/csaf

aschierl-xitaso · 2026-06-22T12:57:28Z

CSAF validators should be able to work on partially invalid documents (according to the schema) and allow performing tests as long as the invalid parts do not affect the test logic. For this, we intend to implement non-strict parsing and validation.

This pull request proposes two alternatives to achieve this goal:

One approach that splits tests into two parts, one that extracts all relevant data for the test from the JSON document in a stream-based fashion while iterating the JSON just once, and a second part that performs the validation on the extracted data. This approach allows to parse streaming data (and syntactically invalid JSON up to the first error) and can avoid deserializing the whole document into memory, but causes more changes to the structure of tests and cannot use the existing abstractions of the traits.
One approach that makes the existing traits non-strict (i.e. every field is an Option that can be None if the JSON does not contain a valid value for that), and implements the traits upon reference wrappers around serde_json::Value. This leaves the test software structure similar to the current state and just requires the tests to with the newly added None options.

For more details, see the ADR document in the pull request.

For the rough idea, see #571, although some details have been implemented differently)

github-actions

CSAF Validation Benchmark

Details

Benchmark suite	Current: `21130ed`	Previous: `f530826`	Ratio
`csaf_2_0_tests/6.1.1`	`118152` ns/iter (`± 532`)	`132928` ns/iter (`± 2476`)	`0.89`
`csaf_2_0_tests/6.1.2`	`47289` ns/iter (`± 241`)	`54492` ns/iter (`± 150`)	`0.87`
`csaf_2_0_tests/6.1.3`	`12392` ns/iter (`± 101`)	`12985` ns/iter (`± 15`)	`0.95`
`csaf_2_0_tests/6.1.4`	`25349` ns/iter (`± 112`)	`28047` ns/iter (`± 200`)	`0.90`
`csaf_2_0_tests/6.1.5`	`13195` ns/iter (`± 64`)	`14239` ns/iter (`± 65`)	`0.93`
`csaf_2_0_tests/6.1.6`	`33184` ns/iter (`± 152`)	`37642` ns/iter (`± 82`)	`0.88`
`csaf_2_0_tests/6.1.7`	`29996` ns/iter (`± 80`)	`33372` ns/iter (`± 230`)	`0.90`
`csaf_2_0_tests/6.1.8`	`81239` ns/iter (`± 779`)	`90703` ns/iter (`± 843`)	`0.90`
`csaf_2_0_tests/6.1.9`	`62123` ns/iter (`± 415`)	`68166` ns/iter (`± 124`)	`0.91`
`csaf_2_0_tests/6.1.10`	`125885` ns/iter (`± 447`)	`134637` ns/iter (`± 1225`)	`0.93`
`csaf_2_0_tests/6.1.11`	`13463` ns/iter (`± 50`)	`14406` ns/iter (`± 27`)	`0.93`
`csaf_2_0_tests/6.1.12`	`31747` ns/iter (`± 145`)	`29897` ns/iter (`± 46`)	`1.06`
`csaf_2_0_tests/6.1.13`	`30887` ns/iter (`± 1876`)	`33765` ns/iter (`± 157`)	`0.91`
`csaf_2_0_tests/6.1.15`	`12088` ns/iter (`± 48`)	`12549` ns/iter (`± 60`)	`0.96`
`csaf_2_0_tests/6.1.16`	`41816` ns/iter (`± 137`)	`47514` ns/iter (`± 276`)	`0.88`
`csaf_2_0_tests/6.1.17`	`13337` ns/iter (`± 85`)	`13647` ns/iter (`± 43`)	`0.98`
`csaf_2_0_tests/6.1.18`	`15790` ns/iter (`± 59`)	`15892` ns/iter (`± 38`)	`0.99`
`csaf_2_0_tests/6.1.19`	`15415` ns/iter (`± 114`)	`15548` ns/iter (`± 31`)	`0.99`
`csaf_2_0_tests/6.1.20`	`13258` ns/iter (`± 55`)	`13464` ns/iter (`± 18`)	`0.98`
`csaf_2_0_tests/6.1.21`	`40812` ns/iter (`± 156`)	`44368` ns/iter (`± 186`)	`0.92`
`csaf_2_0_tests/6.1.22`	`52750` ns/iter (`± 193`)	`58027` ns/iter (`± 621`)	`0.91`
`csaf_2_0_tests/6.1.23`	`14831` ns/iter (`± 119`)	`16192` ns/iter (`± 24`)	`0.92`
`csaf_2_0_tests/6.1.24`	`14952` ns/iter (`± 83`)	`16461` ns/iter (`± 301`)	`0.91`
`csaf_2_0_tests/6.1.25`	`34036` ns/iter (`± 170`)	`36859` ns/iter (`± 52`)	`0.92`
`csaf_2_0_tests/6.1.26`	`18786` ns/iter (`± 55`)	`20309` ns/iter (`± 36`)	`0.93`
`csaf_2_0_tests/6.1.27.1`	`12669` ns/iter (`± 71`)	`13770` ns/iter (`± 49`)	`0.92`
`csaf_2_0_tests/6.1.27.2`	`12724` ns/iter (`± 73`)	`13808` ns/iter (`± 38`)	`0.92`
`csaf_2_0_tests/6.1.27.3`	`12346` ns/iter (`± 64`)	`13508` ns/iter (`± 11`)	`0.91`
`csaf_2_0_tests/6.1.27.4`	`12906` ns/iter (`± 76`)	`13892` ns/iter (`± 18`)	`0.93`
`csaf_2_0_tests/6.1.27.5`	`13478` ns/iter (`± 201`)	`14370` ns/iter (`± 46`)	`0.94`
`csaf_2_0_tests/6.1.27.6`	`12838` ns/iter (`± 28`)	`13804` ns/iter (`± 21`)	`0.93`
`csaf_2_0_tests/6.1.27.7`	`12029` ns/iter (`± 69`)	`12984` ns/iter (`± 49`)	`0.93`
`csaf_2_0_tests/6.1.27.8`	`12208` ns/iter (`± 52`)	`13044` ns/iter (`± 25`)	`0.94`
`csaf_2_0_tests/6.1.27.9`	`23927` ns/iter (`± 127`)	`26289` ns/iter (`± 329`)	`0.91`
`csaf_2_0_tests/6.1.27.10`	`12545` ns/iter (`± 83`)	`14230` ns/iter (`± 24`)	`0.88`
`csaf_2_0_tests/6.1.27.11`	`12682` ns/iter (`± 146`)	`14020` ns/iter (`± 168`)	`0.90`
`csaf_2_0_tests/6.1.28`	`23820` ns/iter (`± 89`)	`24033` ns/iter (`± 34`)	`0.99`
`csaf_2_0_tests/6.1.29`	`12279` ns/iter (`± 37`)	`13024` ns/iter (`± 32`)	`0.94`
`csaf_2_0_tests/6.1.30`	`42310` ns/iter (`± 239`)	`47770` ns/iter (`± 343`)	`0.89`
`csaf_2_0_tests/6.1.31`	`192198` ns/iter (`± 1054`)	`36489` ns/iter (`± 186`)	`5.27`
`csaf_2_0_tests/6.1.32`	`12268` ns/iter (`± 115`)	`13137` ns/iter (`± 51`)	`0.93`
`csaf_2_0_tests/6.1.33`	`18152` ns/iter (`± 72`)	`20098` ns/iter (`± 56`)	`0.90`
`csaf_2_0_tests/6.2.1`	`144778` ns/iter (`± 975`)	`163958` ns/iter (`± 412`)	`0.88`
`csaf_2_0_tests/6.2.2`	`20939` ns/iter (`± 80`)	`22023` ns/iter (`± 81`)	`0.95`
`csaf_2_0_tests/6.2.3`	`20307` ns/iter (`± 60`)	`21321` ns/iter (`± 55`)	`0.95`
`csaf_2_0_tests/6.2.4`	`20093` ns/iter (`± 86`)	`20257` ns/iter (`± 25`)	`0.99`
`csaf_2_0_tests/6.2.5`	`51574` ns/iter (`± 249`)	`59442` ns/iter (`± 187`)	`0.87`
`csaf_2_0_tests/6.2.6`	`51700` ns/iter (`± 199`)	`60373` ns/iter (`± 121`)	`0.86`
`csaf_2_0_tests/6.2.7`	`17495` ns/iter (`± 70`)	`17860` ns/iter (`± 422`)	`0.98`
`csaf_2_0_tests/6.2.8`	`37716` ns/iter (`± 173`)	`40130` ns/iter (`± 541`)	`0.94`
`csaf_2_0_tests/6.2.9`	`36953` ns/iter (`± 180`)	`39965` ns/iter (`± 512`)	`0.92`
`csaf_2_0_tests/6.2.10`	`27500` ns/iter (`± 283`)	`29201` ns/iter (`± 148`)	`0.94`
`csaf_2_0_tests/6.2.11`	`121444` ns/iter (`± 877`)	`155038` ns/iter (`± 695`)	`0.78`
`csaf_2_0_tests/6.2.12`	`35927` ns/iter (`± 218`)	`36810` ns/iter (`± 71`)	`0.98`
`csaf_2_0_tests/6.2.13`	`725984` ns/iter (`± 7073`)	`834553` ns/iter (`± 5375`)	`0.87`
`csaf_2_0_tests/6.2.14`	`48240` ns/iter (`± 293`)	`45396` ns/iter (`± 113`)	`1.06`
`csaf_2_0_tests/6.2.15`	`37498` ns/iter (`± 216`)	`35516` ns/iter (`± 238`)	`1.06`
`csaf_2_0_tests/6.2.16`	`43678` ns/iter (`± 268`)	`47307` ns/iter (`± 57`)	`0.92`
`csaf_2_0_tests/6.2.17`	`17680` ns/iter (`± 94`)	`18387` ns/iter (`± 9`)	`0.96`
`csaf_2_0_tests/6.2.18`	`27687` ns/iter (`± 158`)	`30200` ns/iter (`± 32`)	`0.92`
`csaf_2_0_tests/6.2.19`	`21500` ns/iter (`± 94`)	`22015` ns/iter (`± 18`)	`0.98`
`csaf_2_0_tests/6.2.20`	`1423099` ns/iter (`± 6866`)	`1632865` ns/iter (`± 5214`)	`0.87`
`csaf_2_0_tests/6.3.1`	`41015` ns/iter (`± 289`)	`44151` ns/iter (`± 273`)	`0.93`
`csaf_2_0_tests/6.3.2`	`28492` ns/iter (`± 116`)	`29863` ns/iter (`± 53`)	`0.95`
`csaf_2_0_tests/6.3.3`	`28491` ns/iter (`± 84`)	`29095` ns/iter (`± 54`)	`0.98`
`csaf_2_0_tests/6.3.4`	`34969` ns/iter (`± 175`)	`36450` ns/iter (`± 101`)	`0.96`
`csaf_2_0_tests/6.3.5`	`43168` ns/iter (`± 212`)	`46547` ns/iter (`± 235`)	`0.93`
`csaf_2_0_tests/6.3.6`	`20992` ns/iter (`± 116`)	`21502` ns/iter (`± 27`)	`0.98`
`csaf_2_0_tests/6.3.7`	`21117` ns/iter (`± 131`)	`21569` ns/iter (`± 43`)	`0.98`
`csaf_2_0_tests/6.3.8`	`21188` ns/iter (`± 143`)	`21569` ns/iter (`± 97`)	`0.98`
`csaf_2_0_tests/6.3.9`	`35939` ns/iter (`± 205`)	`37609` ns/iter (`± 178`)	`0.96`
`csaf_2_0_tests/6.3.10`	`32957` ns/iter (`± 133`)	`34318` ns/iter (`± 59`)	`0.96`
`csaf_2_0_tests/6.3.11`	`33611` ns/iter (`± 139`)	`35439` ns/iter (`± 74`)	`0.95`
`csaf_2_1_tests/6.1.1`	`588573` ns/iter (`± 2189`)	`658631` ns/iter (`± 3887`)	`0.89`
`csaf_2_1_tests/6.1.2`	`229200` ns/iter (`± 569`)	`269924` ns/iter (`± 630`)	`0.85`
`csaf_2_1_tests/6.1.3`	`69521` ns/iter (`± 377`)	`72492` ns/iter (`± 122`)	`0.96`
`csaf_2_1_tests/6.1.4`	`114977` ns/iter (`± 555`)	`134674` ns/iter (`± 389`)	`0.85`
`csaf_2_1_tests/6.1.5`	`50446` ns/iter (`± 243`)	`50655` ns/iter (`± 946`)	`1.00`
`csaf_2_1_tests/6.1.6`	`106676` ns/iter (`± 483`)	`119528` ns/iter (`± 570`)	`0.89`
`csaf_2_1_tests/6.1.7`	`78308` ns/iter (`± 398`)	`77157` ns/iter (`± 3512`)	`1.01`
`csaf_2_1_tests/6.1.8`	`78361` ns/iter (`± 546`)	`78155` ns/iter (`± 214`)	`1.00`
`csaf_2_1_tests/6.1.9`	`466558` ns/iter (`± 21962`)	`511064` ns/iter (`± 1919`)	`0.91`
`csaf_2_1_tests/6.1.10`	`632084` ns/iter (`± 10197`)	`656195` ns/iter (`± 1975`)	`0.96`
`csaf_2_1_tests/6.1.11`	`81576` ns/iter (`± 319`)	`90481` ns/iter (`± 196`)	`0.90`
`csaf_2_1_tests/6.1.12`	`90149` ns/iter (`± 426`)	`84268` ns/iter (`± 206`)	`1.07`
`csaf_2_1_tests/6.1.13`	`188076` ns/iter (`± 2035`)	`207985` ns/iter (`± 459`)	`0.90`
`csaf_2_1_tests/6.1.15`	`45300` ns/iter (`± 144`)	`44747` ns/iter (`± 205`)	`1.01`
`csaf_2_1_tests/6.1.16`	`130492` ns/iter (`± 496`)	`168556` ns/iter (`± 381`)	`0.77`
`csaf_2_1_tests/6.1.17`	`50706` ns/iter (`± 249`)	`50202` ns/iter (`± 154`)	`1.01`
`csaf_2_1_tests/6.1.18`	`55906` ns/iter (`± 252`)	`60997` ns/iter (`± 407`)	`0.92`
`csaf_2_1_tests/6.1.19`	`56132` ns/iter (`± 291`)	`57178` ns/iter (`± 229`)	`0.98`
`csaf_2_1_tests/6.1.20`	`51475` ns/iter (`± 187`)	`49519` ns/iter (`± 94`)	`1.04`
`csaf_2_1_tests/6.1.21`	`128667` ns/iter (`± 1349`)	`162515` ns/iter (`± 328`)	`0.79`
`csaf_2_1_tests/6.1.22`	`159872` ns/iter (`± 502`)	`175397` ns/iter (`± 399`)	`0.91`
`csaf_2_1_tests/6.1.23`	`54928` ns/iter (`± 300`)	`56601` ns/iter (`± 61`)	`0.97`
`csaf_2_1_tests/6.1.24`	`52349` ns/iter (`± 307`)	`59793` ns/iter (`± 512`)	`0.88`
`csaf_2_1_tests/6.1.25`	`165429` ns/iter (`± 608`)	`183263` ns/iter (`± 530`)	`0.90`
`csaf_2_1_tests/6.1.26`	`71740` ns/iter (`± 325`)	`74520` ns/iter (`± 87`)	`0.96`
`csaf_2_1_tests/6.1.27.1`	`49164` ns/iter (`± 102`)	`49668` ns/iter (`± 60`)	`0.99`
`csaf_2_1_tests/6.1.27.2`	`49098` ns/iter (`± 145`)	`50209` ns/iter (`± 84`)	`0.98`
`csaf_2_1_tests/6.1.27.3`	`50614` ns/iter (`± 306`)	`50935` ns/iter (`± 55`)	`0.99`
`csaf_2_1_tests/6.1.27.4`	`50635` ns/iter (`± 182`)	`51060` ns/iter (`± 168`)	`0.99`
`csaf_2_1_tests/6.1.27.5`	`52267` ns/iter (`± 106`)	`52351` ns/iter (`± 110`)	`1.00`
`csaf_2_1_tests/6.1.27.6`	`51487` ns/iter (`± 157`)	`51320` ns/iter (`± 89`)	`1.00`
`csaf_2_1_tests/6.1.27.7`	`48218` ns/iter (`± 194`)	`47260` ns/iter (`± 60`)	`1.02`
`csaf_2_1_tests/6.1.27.8`	`47783` ns/iter (`± 291`)	`47556` ns/iter (`± 107`)	`1.00`
`csaf_2_1_tests/6.1.27.9`	`66783` ns/iter (`± 303`)	`68752` ns/iter (`± 181`)	`0.97`
`csaf_2_1_tests/6.1.27.10`	`45725` ns/iter (`± 1399`)	`48723` ns/iter (`± 187`)	`0.94`
`csaf_2_1_tests/6.1.27.11`	`48204` ns/iter (`± 218`)	`50784` ns/iter (`± 40`)	`0.95`
`csaf_2_1_tests/6.1.27.12`	`49842` ns/iter (`± 372`)	`50577` ns/iter (`± 185`)	`0.99`
`csaf_2_1_tests/6.1.27.13`	`76301` ns/iter (`± 323`)	`74051` ns/iter (`± 85`)	`1.03`
`csaf_2_1_tests/6.1.27.14`	`50123` ns/iter (`± 136`)	`50672` ns/iter (`± 52`)	`0.99`
`csaf_2_1_tests/6.1.27.15`	`49183` ns/iter (`± 93`)	`50914` ns/iter (`± 131`)	`0.97`
`csaf_2_1_tests/6.1.27.16`	`49666` ns/iter (`± 256`)	`50829` ns/iter (`± 62`)	`0.98`
`csaf_2_1_tests/6.1.27.17`	`52071` ns/iter (`± 163`)	`54535` ns/iter (`± 51`)	`0.95`
`csaf_2_1_tests/6.1.27.18`	`56144` ns/iter (`± 173`)	`56803` ns/iter (`± 65`)	`0.99`
`csaf_2_1_tests/6.1.27.19`	`54729` ns/iter (`± 302`)	`55314` ns/iter (`± 80`)	`0.99`
`csaf_2_1_tests/6.1.28`	`73749` ns/iter (`± 299`)	`71607` ns/iter (`± 161`)	`1.03`
`csaf_2_1_tests/6.1.29`	`48974` ns/iter (`± 189`)	`49037` ns/iter (`± 3706`)	`1.00`
`csaf_2_1_tests/6.1.30`	`135884` ns/iter (`± 844`)	`173793` ns/iter (`± 682`)	`0.78`
`csaf_2_1_tests/6.1.31`	`1062986` ns/iter (`± 12420`)	`180975` ns/iter (`± 865`)	`5.87`
`csaf_2_1_tests/6.1.32`	`49309` ns/iter (`± 211`)	`49737` ns/iter (`± 67`)	`0.99`
`csaf_2_1_tests/6.1.33`	`85261` ns/iter (`± 491`)	`89248` ns/iter (`± 132`)	`0.96`
`csaf_2_1_tests/6.1.34`	`57233` ns/iter (`± 302`)	`60384` ns/iter (`± 150`)	`0.95`
`csaf_2_1_tests/6.1.35`	`75409` ns/iter (`± 333`)	`81836` ns/iter (`± 490`)	`0.92`
`csaf_2_1_tests/6.1.36`	`85477` ns/iter (`± 138`)	`91032` ns/iter (`± 176`)	`0.94`
`csaf_2_1_tests/6.1.37`	`85044` ns/iter (`± 1585`)	`91327` ns/iter (`± 92`)	`0.93`
`csaf_2_1_tests/6.1.38`	`46967` ns/iter (`± 201`)	`49250` ns/iter (`± 59`)	`0.95`
`csaf_2_1_tests/6.1.39`	`47452` ns/iter (`± 226`)	`49733` ns/iter (`± 45`)	`0.95`
`csaf_2_1_tests/6.1.40`	`46829` ns/iter (`± 101`)	`49812` ns/iter (`± 118`)	`0.94`
`csaf_2_1_tests/6.1.41`	`47180` ns/iter (`± 131`)	`50164` ns/iter (`± 28`)	`0.94`
`csaf_2_1_tests/6.1.42`	`200212` ns/iter (`± 1578`)	`219134` ns/iter (`± 995`)	`0.91`
`csaf_2_1_tests/6.1.43`	`165351` ns/iter (`± 986`)	`181363` ns/iter (`± 428`)	`0.91`
`csaf_2_1_tests/6.1.44`	`163559` ns/iter (`± 685`)	`184088` ns/iter (`± 352`)	`0.89`
`csaf_2_1_tests/6.1.45`	`131829` ns/iter (`± 515`)	`167515` ns/iter (`± 1219`)	`0.79`
`csaf_2_1_tests/6.1.46`	`253829` ns/iter (`± 915`)	`275459` ns/iter (`± 609`)	`0.92`
`csaf_2_1_tests/6.1.47`	`389020` ns/iter (`± 2020`)	`435907` ns/iter (`± 1730`)	`0.89`
`csaf_2_1_tests/6.1.48`	`86962` ns/iter (`± 2059`)	`95921` ns/iter (`± 91`)	`0.91`
`csaf_2_1_tests/6.1.49`	`87803` ns/iter (`± 353`)	`96071` ns/iter (`± 345`)	`0.91`
`csaf_2_1_tests/6.1.50`	`87757` ns/iter (`± 497`)	`96759` ns/iter (`± 441`)	`0.91`
`csaf_2_1_tests/6.1.51`	`135812` ns/iter (`± 404`)	`179815` ns/iter (`± 425`)	`0.76`
`csaf_2_1_tests/6.1.52`	`139251` ns/iter (`± 718`)	`179610` ns/iter (`± 898`)	`0.78`
`csaf_2_1_tests/6.1.53`	`55585` ns/iter (`± 1379`)	`60928` ns/iter (`± 384`)	`0.91`
`csaf_2_1_tests/6.1.54`	`61890` ns/iter (`± 636`)	`68360` ns/iter (`± 137`)	`0.91`
`csaf_2_1_tests/6.1.55`	`63008` ns/iter (`± 329`)	`69482` ns/iter (`± 324`)	`0.91`
`csaf_2_1_tests/6.1.56`	`87674` ns/iter (`± 509`)	`105909` ns/iter (`± 294`)	`0.83`
`csaf_2_1_tests/6.1.57`	`123828` ns/iter (`± 262`)	`136642` ns/iter (`± 318`)	`0.91`
`csaf_2_1_tests/6.1.58`	`90730` ns/iter (`± 836`)	`98625` ns/iter (`± 395`)	`0.92`
`csaf_2_1_tests/6.1.59`	`90397` ns/iter (`± 670`)	`107124` ns/iter (`± 574`)	`0.84`
`csaf_2_1_tests/6.1.60.1`	`77704` ns/iter (`± 467`)	`75387` ns/iter (`± 68`)	`1.03`
`csaf_2_1_tests/6.1.60.2`	`78035` ns/iter (`± 173`)	`75450` ns/iter (`± 124`)	`1.03`
`csaf_2_1_tests/6.1.60.3`	`78751` ns/iter (`± 341`)	`75424` ns/iter (`± 176`)	`1.04`
`csaf_2_1_tests/6.1.61`	`166188` ns/iter (`± 1149`)	`188875` ns/iter (`± 362`)	`0.88`
`csaf_2_1_tests/6.2.1`	`668319` ns/iter (`± 3449`)	`730920` ns/iter (`± 2437`)	`0.91`
`csaf_2_1_tests/6.2.2`	`83845` ns/iter (`± 364`)	`85209` ns/iter (`± 1308`)	`0.98`
`csaf_2_1_tests/6.2.3`	`79949` ns/iter (`± 374`)	`80727` ns/iter (`± 241`)	`0.99`
`csaf_2_1_tests/6.2.4`	`73692` ns/iter (`± 348`)	`73308` ns/iter (`± 316`)	`1.01`
`csaf_2_1_tests/6.2.5`	`173356` ns/iter (`± 808`)	`219630` ns/iter (`± 881`)	`0.79`
`csaf_2_1_tests/6.2.6`	`175138` ns/iter (`± 581`)	`220120` ns/iter (`± 533`)	`0.80`
`csaf_2_1_tests/6.2.7`	`66992` ns/iter (`± 224`)	`65021` ns/iter (`± 800`)	`1.03`
`csaf_2_1_tests/6.2.8`	`181527` ns/iter (`± 369`)	`200264` ns/iter (`± 580`)	`0.91`
`csaf_2_1_tests/6.2.9`	`179173` ns/iter (`± 1163`)	`197991` ns/iter (`± 523`)	`0.90`
`csaf_2_1_tests/6.2.11`	`450229` ns/iter (`± 3661`)	`572521` ns/iter (`± 3520`)	`0.79`
`csaf_2_1_tests/6.2.12`	`132225` ns/iter (`± 265`)	`132991` ns/iter (`± 280`)	`0.99`
`csaf_2_1_tests/6.2.13`	`3683997` ns/iter (`± 40341`)	`3798932` ns/iter (`± 29781`)	`0.97`
`csaf_2_1_tests/6.2.14`	`139107` ns/iter (`± 560`)	`134764` ns/iter (`± 1731`)	`1.03`
`csaf_2_1_tests/6.2.15`	`117233` ns/iter (`± 362`)	`115017` ns/iter (`± 376`)	`1.02`
`csaf_2_1_tests/6.2.16`	`215527` ns/iter (`± 900`)	`246368` ns/iter (`± 1275`)	`0.87`
`csaf_2_1_tests/6.2.17`	`73075` ns/iter (`± 412`)	`76991` ns/iter (`± 357`)	`0.95`
`csaf_2_1_tests/6.2.18`	`149223` ns/iter (`± 485`)	`167069` ns/iter (`± 350`)	`0.89`
`csaf_2_1_tests/6.2.19`	`84822` ns/iter (`± 349`)	`88655` ns/iter (`± 84`)	`0.96`
`csaf_2_1_tests/6.2.20`	`8053604` ns/iter (`± 57235`)	`8681465` ns/iter (`± 96232`)	`0.93`
`csaf_2_1_tests/6.2.21`	`321981` ns/iter (`± 1208`)	`374072` ns/iter (`± 3326`)	`0.86`
`csaf_2_1_tests/6.2.22`	`154387` ns/iter (`± 343`)	`177058` ns/iter (`± 280`)	`0.87`
`csaf_2_1_tests/6.2.23`	`85449` ns/iter (`± 268`)	`92246` ns/iter (`± 147`)	`0.93`
`csaf_2_1_tests/6.2.24`	`222410` ns/iter (`± 926`)	`243467` ns/iter (`± 543`)	`0.91`
`csaf_2_1_tests/6.2.25`	`87385` ns/iter (`± 259`)	`90926` ns/iter (`± 739`)	`0.96`
`csaf_2_1_tests/6.2.26`	`86256` ns/iter (`± 372`)	`93788` ns/iter (`± 85`)	`0.92`
`csaf_2_1_tests/6.2.27`	`86066` ns/iter (`± 619`)	`90558` ns/iter (`± 91`)	`0.95`
`csaf_2_1_tests/6.2.28`	`71002` ns/iter (`± 370`)	`76878` ns/iter (`± 146`)	`0.92`
`csaf_2_1_tests/6.2.29`	`70691` ns/iter (`± 201`)	`76591` ns/iter (`± 137`)	`0.92`
`csaf_2_1_tests/6.2.30`	`71277` ns/iter (`± 231`)	`77140` ns/iter (`± 1236`)	`0.92`
`csaf_2_1_tests/6.2.31`	`209361` ns/iter (`± 529`)	`233565` ns/iter (`± 784`)	`0.90`
`csaf_2_1_tests/6.2.32`	`308396` ns/iter (`± 784`)	`342841` ns/iter (`± 514`)	`0.90`
`csaf_2_1_tests/6.2.33`	`208075` ns/iter (`± 969`)	`250526` ns/iter (`± 493`)	`0.83`
`csaf_2_1_tests/6.2.34`	`87113` ns/iter (`± 398`)	`93287` ns/iter (`± 122`)	`0.93`
`csaf_2_1_tests/6.2.35`	`92108` ns/iter (`± 238`)	`93858` ns/iter (`± 317`)	`0.98`
`csaf_2_1_tests/6.2.36`	`86708` ns/iter (`± 365`)	`94027` ns/iter (`± 723`)	`0.92`
`csaf_2_1_tests/6.2.37`	`86963` ns/iter (`± 438`)	`100457` ns/iter (`± 560`)	`0.87`
`csaf_2_1_tests/6.2.38`	`447559` ns/iter (`± 1637`)	`104757` ns/iter (`± 219`)	`4.27`
`csaf_2_1_tests/6.2.39.1`	`79213` ns/iter (`± 422`)	`75412` ns/iter (`± 70`)	`1.05`
`csaf_2_1_tests/6.2.39.2`	`79590` ns/iter (`± 199`)	`75367` ns/iter (`± 107`)	`1.06`
`csaf_2_1_tests/6.2.39.3`	`79683` ns/iter (`± 1320`)	`75370` ns/iter (`± 66`)	`1.06`
`csaf_2_1_tests/6.2.39.4`	`79393` ns/iter (`± 282`)	`75380` ns/iter (`± 83`)	`1.05`
`csaf_2_1_tests/6.2.39.5`	`79246` ns/iter (`± 318`)	`75390` ns/iter (`± 135`)	`1.05`
`csaf_2_1_tests/6.2.40`	`87134` ns/iter (`± 335`)	`95250` ns/iter (`± 150`)	`0.91`
`csaf_2_1_tests/6.2.41`	`160960` ns/iter (`± 558`)	`206790` ns/iter (`± 529`)	`0.78`
`csaf_2_1_tests/6.2.42`	`87613` ns/iter (`± 426`)	`95188` ns/iter (`± 132`)	`0.92`
`csaf_2_1_tests/6.2.43`	`87772` ns/iter (`± 247`)	`97028` ns/iter (`± 272`)	`0.90`
`csaf_2_1_tests/6.2.44`	`88090` ns/iter (`± 479`)	`96775` ns/iter (`± 239`)	`0.91`
`csaf_2_1_tests/6.2.45`	`88150` ns/iter (`± 629`)	`96909` ns/iter (`± 92`)	`0.91`
`csaf_2_1_tests/6.2.46`	`88385` ns/iter (`± 421`)	`98150` ns/iter (`± 244`)	`0.90`
`csaf_2_1_tests/6.2.47`	`92083` ns/iter (`± 515`)	`98256` ns/iter (`± 100`)	`0.94`
`csaf_2_1_tests/6.2.48`	`161453` ns/iter (`± 382`)	`183403` ns/iter (`± 693`)	`0.88`
`csaf_2_1_tests/6.2.49`	`89930` ns/iter (`± 571`)	`104812` ns/iter (`± 103`)	`0.86`
`csaf_2_1_tests/6.2.50.1`	`79849` ns/iter (`± 298`)	`75413` ns/iter (`± 48`)	`1.06`
`csaf_2_1_tests/6.2.50.2`	`79379` ns/iter (`± 365`)	`75418` ns/iter (`± 139`)	`1.05`
`csaf_2_1_tests/6.2.50.3`	`79572` ns/iter (`± 355`)	`75391` ns/iter (`± 48`)	`1.06`
`csaf_2_1_tests/6.2.51`	`89784` ns/iter (`± 408`)	`98984` ns/iter (`± 682`)	`0.91`
`csaf_2_1_tests/6.2.52`	`196099` ns/iter (`± 592`)	`218923` ns/iter (`± 473`)	`0.90`
`csaf_2_1_tests/6.2.53`	`81619` ns/iter (`± 530`)	`89137` ns/iter (`± 285`)	`0.92`
`csaf_2_1_tests/6.2.54.1`	`79486` ns/iter (`± 459`)	`75378` ns/iter (`± 71`)	`1.05`
`csaf_2_1_tests/6.2.54.2`	`79423` ns/iter (`± 1054`)	`75379` ns/iter (`± 58`)	`1.05`
`csaf_2_1_tests/6.2.54.3`	`80068` ns/iter (`± 1283`)	`75443` ns/iter (`± 360`)	`1.06`
`csaf_2_1_tests/6.2.54.4`	`79298` ns/iter (`± 545`)	`75410` ns/iter (`± 328`)	`1.05`
`csaf_2_1_tests/6.3.1`	`78723` ns/iter (`± 341`)	`78072` ns/iter (`± 503`)	`1.01`
`csaf_2_1_tests/6.3.2`	`102923` ns/iter (`± 377`)	`106743` ns/iter (`± 358`)	`0.96`
`csaf_2_1_tests/6.3.3`	`78857` ns/iter (`± 453`)	`77865` ns/iter (`± 104`)	`1.01`
`csaf_2_1_tests/6.3.4`	`79024` ns/iter (`± 636`)	`77919` ns/iter (`± 130`)	`1.01`
`csaf_2_1_tests/6.3.5`	`79775` ns/iter (`± 525`)	`77893` ns/iter (`± 78`)	`1.02`
`csaf_2_1_tests/6.3.6`	`79926` ns/iter (`± 667`)	`78160` ns/iter (`± 105`)	`1.02`
`csaf_2_1_tests/6.3.7`	`78891` ns/iter (`± 495`)	`78203` ns/iter (`± 117`)	`1.01`
`csaf_2_1_tests/6.3.8`	`79843` ns/iter (`± 666`)	`78122` ns/iter (`± 391`)	`1.02`
`csaf_2_1_tests/6.3.9`	`198096` ns/iter (`± 466`)	`198438` ns/iter (`± 614`)	`1.00`
`csaf_2_1_tests/6.3.10`	`89624` ns/iter (`± 266`)	`97205` ns/iter (`± 134`)	`0.92`
`csaf_2_1_tests/6.3.11`	`89527` ns/iter (`± 307`)	`96992` ns/iter (`± 147`)	`0.92`
`csaf_2_1_tests/6.3.12`	`224109` ns/iter (`± 429`)	`263517` ns/iter (`± 403`)	`0.85`
`csaf_2_1_tests/6.3.13`	`89881` ns/iter (`± 343`)	`97586` ns/iter (`± 161`)	`0.92`
`csaf_2_1_tests/6.3.14`	`89671` ns/iter (`± 397`)	`97339` ns/iter (`± 494`)	`0.92`
`csaf_2_1_tests/6.3.15`	`90314` ns/iter (`± 438`)	`98007` ns/iter (`± 172`)	`0.92`
`csaf_2_1_tests/6.3.16`	`90753` ns/iter (`± 409`)	`98563` ns/iter (`± 231`)	`0.92`
`csaf_2_1_tests/6.3.17`	`90540` ns/iter (`± 387`)	`98478` ns/iter (`± 105`)	`0.92`
`csaf_2_1_tests/6.3.18`	`101659` ns/iter (`± 367`)	`109722` ns/iter (`± 131`)	`0.93`
`csaf_2_1_tests/6.3.19.1`	`78957` ns/iter (`± 313`)	`75407` ns/iter (`± 91`)	`1.05`
`csaf_2_1_tests/6.3.19.2`	`79362` ns/iter (`± 577`)	`75384` ns/iter (`± 64`)	`1.05`
`csaf_2_1_tests/6.3.19.3`	`78943` ns/iter (`± 528`)	`75432` ns/iter (`± 66`)	`1.05`
`csaf_2_1_tests/6.3.19.4`	`78458` ns/iter (`± 292`)	`75430` ns/iter (`± 206`)	`1.04`
`csaf_2_1_tests/6.3.19.5`	`79486` ns/iter (`± 375`)	`75403` ns/iter (`± 145`)	`1.05`
`csaf_2_1_tests/6.3.20`	`102721` ns/iter (`± 4094`)	`110576` ns/iter (`± 641`)	`0.93`
`csaf_2_1_tests/6.3.21.1`	`79736` ns/iter (`± 571`)	`75414` ns/iter (`± 90`)	`1.06`
`csaf_2_1_tests/6.3.21.2`	`79094` ns/iter (`± 439`)	`75406` ns/iter (`± 165`)	`1.05`
`csaf_2_1_tests/6.3.21.3`	`79429` ns/iter (`± 358`)	`75393` ns/iter (`± 81`)	`1.05`
`csaf_2_1_tests/6.3.21.4`	`79377` ns/iter (`± 366`)	`75404` ns/iter (`± 92`)	`1.05`
`csaf_2_1_tests/6.3.21.5`	`79029` ns/iter (`± 519`)	`75435` ns/iter (`± 80`)	`1.05`
`csaf_2_1_tests/6.3.21.6`	`80215` ns/iter (`± 308`)	`75481` ns/iter (`± 136`)	`1.06`
`csaf_2_1_tests/6.3.21.7`	`79055` ns/iter (`± 294`)	`75389` ns/iter (`± 245`)	`1.05`
`csaf_2_1_tests/6.3.21.8`	`79655` ns/iter (`± 359`)	`75403` ns/iter (`± 56`)	`1.06`
`csaf_2_1_tests/6.3.21.9`	`79514` ns/iter (`± 328`)	`75390` ns/iter (`± 49`)	`1.05`
`csaf_2_1_tests/6.3.22`	`90129` ns/iter (`± 355`)	`99093` ns/iter (`± 99`)	`0.91`
`full_validation/csaf_2_0_full_preset`	`7140944` ns/iter (`± 54842`)	`7642247` ns/iter (`± 39926`)	`0.93`
`full_validation/csaf_2_1_full_preset`	`53772104` ns/iter (`± 195642`)	`55397491` ns/iter (`± 472006`)	`0.97`
`parse_only/csaf_2_0`	`1110290` ns/iter (`± 10029`)	`1180398` ns/iter (`± 3519`)	`0.94`
`parse_only/csaf_2_1`	`5548876` ns/iter (`± 39466`)	`5737821` ns/iter (`± 21967`)	`0.97`
`typed_parse/csaf_2_0`	`1919700` ns/iter (`± 9112`)	`2041253` ns/iter (`± 4118`)	`0.94`
`typed_parse/csaf_2_1`	`9128177` ns/iter (`± 25884`)	`9665933` ns/iter (`± 12774`)	`0.94`

This comment was automatically generated by workflow using github-action-benchmark.

github-actions

⚠️ Performance Alert ⚠️

Possible performance regression was detected for benchmark 'CSAF Validation Benchmark'.
Benchmark result of this commit is worse than the previous benchmark result exceeding threshold 1.50.

Benchmark suite	Current: `21130ed`	Previous: `f530826`	Ratio
`csaf_2_0_tests/6.1.31`	`192198` ns/iter (`± 1054`)	`36489` ns/iter (`± 186`)	`5.27`
`csaf_2_1_tests/6.1.31`	`1062986` ns/iter (`± 12420`)	`180975` ns/iter (`± 865`)	`5.87`
`csaf_2_1_tests/6.2.38`	`447559` ns/iter (`± 1637`)	`104757` ns/iter (`± 219`)	`4.27`

This comment was automatically generated by workflow using github-action-benchmark.

gronke · 2026-07-21T09:57:51Z

A data point in favor of this direction, from consuming the generated models for document intake:

Because the schema patterns are enforced through regress inside Deserialize, one cosmetically nonconforming field (say a cve failing the CVE pattern) rejects the entire document — there is no way to parse the 99% that is fine and inspect the rest. And since the typed model cannot retain the verbatim JSON, an ingest that needs both ends up parsing twice: from_str to Value, clone, then from_value into the typed document. An extractor-based, non-strict path would serve both: read what parses, report what does not, and leave the raw Value available alongside.

aschierl-xitaso · 2026-08-11T15:04:50Z

To get an indication about the performance possible with the different approaches, I tried parsing a 32MB CSAF document using different approaches:

Stream-based extraction the document category (see test 6.2.38)
Stream-based parsing into serde_json::Value (as benchmark against direct serde_json deserialization)
Parsing the document into serde_json::Value (for use with Value-based traits)
Parsing the document into serde_json::Value, then CommonSecurityAdvisoryFramework as generated by typify
Parsing the document directly into CommonSecurityAdvisoryFramework as generated by typify

Approach	Duration	Memory used
Stream-based extraction	0.07s	7MB
Parsing into CSAF	0.13s	61MB
Parsing into serde_json::Value	0.16s	94MB
Stream-based into serde_json::Value	0.16s	107MB
Parsing into Value, then CSAF	0.16s	113MB

Especially in cases where only selected tests run and not all data is required, stream-based parsing has the potential to improve speed and memory usage (memory grows only with the size of extracted data, not linear with the size of the whole input file).
Using serde_json::Value as an intermediate format increses the memory usage and run time, so directly deserializing into a non-strict object model of CSAF might also be an efficient solution (not considered in this pull request).

feat: add extractor-based, non-strict parsing

2b68aa3

github-actions Bot reviewed Jun 22, 2026

View reviewed changes

aschierl-xitaso force-pushed the feat/non-strict-parsing branch from 96fc1df to 4aa1491 Compare June 22, 2026 13:36

fix: fix clippy

80c9a22

aschierl-xitaso force-pushed the feat/non-strict-parsing branch from 4aa1491 to 80c9a22 Compare June 22, 2026 13:39

feat: include json pointer in extractor

2f89ec8

aschierl-xitaso force-pushed the feat/non-strict-parsing branch from ef355b8 to 2f89ec8 Compare June 30, 2026 17:50

aschierl-xitaso added 2 commits July 10, 2026 19:57

feat: make extractors more widely applicable, add tests

6663678

refactor: pass json pointer to extractors

aa62020

aschierl-xitaso force-pushed the feat/non-strict-parsing branch from 86ce807 to 0d8bc42 Compare July 17, 2026 07:03

feat: introduce tree extractors, use in test 6.1.31

3282587

aschierl-xitaso force-pushed the feat/non-strict-parsing branch from 0d8bc42 to 3282587 Compare July 17, 2026 07:07

gronke mentioned this pull request Jul 22, 2026

feat: typed CVSS accessors on ContentTrait #715

Merged

aschierl-xitaso added 3 commits July 27, 2026 14:57

doc: add ADR about non-strict parsing

4474d43

test: add explanatory tests for extractors

dbb71a7

feat: add non-strict traits prototype as alternative

21130ed

docs: improve ADR reasoning

0bb36ac

Navigation Menu

Uh oh!

feat: add extractor-based, non-strict parsing - #693

feat: add extractor-based, non-strict parsing#693
aschierl-xitaso wants to merge 10 commits into
mainfrom
feat/non-strict-parsing

aschierl-xitaso commented Jun 22, 2026 •

edited

Loading

Uh oh!

github-actions Bot left a comment •

edited

Loading

Uh oh!

github-actions Bot left a comment •

edited

Loading

Uh oh!

gronke commented Jul 21, 2026

Uh oh!

aschierl-xitaso commented Aug 11, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Conversation

aschierl-xitaso commented Jun 22, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

CSAF Validation Benchmark

Uh oh!

github-actions Bot left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

⚠️ Performance Alert ⚠️

Uh oh!

gronke commented Jul 21, 2026

Uh oh!

aschierl-xitaso commented Aug 11, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

aschierl-xitaso commented Jun 22, 2026 •

edited

Loading

github-actions Bot left a comment •

edited

Loading

github-actions Bot left a comment •

edited

Loading