feat: add harness field to cargo metadata output#17088
Conversation
|
Thanks for the pull request, and welcome! The Rust team is excited to review your changes, and you should hear from @epage (or someone else) some time within the next two weeks. Please see the contribution instructions for more information. Namely, in order to ensure the minimum review times lag, PR authors and assigned reviewers should ensure that the review label (
Why was this reviewer chosen?The reviewer was selected based on:
|
90e4d19 to
9b0a85f
Compare
harness field to cargo metadata output (#11846)harness field to cargo metadata output
| /// Whether tests should be run for the target (`test` field in `Cargo.toml`) | ||
| test: bool, | ||
| /// Whether the target uses the libtest harness (`harness` field in `Cargo.toml`). | ||
| harness: bool, |
There was a problem hiding this comment.
Since we wrote cargo metadata, we adopted a policy to omit defaults, see https://doc.crates.io/contrib/implementation/schemas.html#schema-design
However, the question becomes how do we deal with the lack of the field vs the default?
There was a problem hiding this comment.
Sorry for missing that detail from the guidelines. I'll add skip_serializing_ifhere.
However, the question becomes how do we deal with the lack of the field vs the default?
I'm not sure exactly what you mean here. In RustRover we can easily assume that no value means true. Or do you mean something else?
There was a problem hiding this comment.
Unresolved this because of the open question.
If we always provide harness, then we have three states: true, false, and "unknown". The last one is relevant when using an old version of Cargo. If we skip on default (would it be a true? false?) then a caller can't differentiate it from "unknown".
There was a problem hiding this comment.
Thanks, I understand now :). Dealing with "unknown" is unavoidable, at least for a while, because older versions of cargo won't have this field. The problem is that now there is some ambiguity: a value not present might be either true or unknown depending on the version of cargo.
For us it would be slightly better if the default was not skipped. In that way, our logic will be something like: if harness is not there, fallback to try to get it from Cargo.toml. If we skip the default value, our logic will be: if harness is not there an cargo version > x it's true otherwise fallback.
So we can manage both ways. I understand the spirit of the guidelines and the fact that extra data means extra parsing time and this can affect certain tools. But from my perspective the ambiguity introduced is worse. So I would prefer if we don't skip.
| "test": true | ||
| "test": true, | ||
| /* Whether or not the target uses the libtest harness. | ||
| Corresponds to the `harness` field in `Cargo.toml`. |
There was a problem hiding this comment.
Raised a higher level concern at #11846 (comment)
f0b1d3e to
7dde5d8
Compare
Adds the `harness` field to each target in `cargo metadata` JSON output, exposing the Cargo.toml `[[test]]`/`[[bench]]` `harness` setting so that external tools can tell whether a target uses the libtest harness or provides its own `main()`.
7dde5d8 to
c9b0605
Compare
Adds the
harnessfield to each target incargo metadataJSON output, exposing the Cargo.toml[[test]]/[[bench]]harnesssetting so that external tools can tell whether a target uses the libtest harness or provides its ownmain().What does this PR try to resolve?
I work on the RustRover IDE. We add small gutter buttons in the editor to run tests and benches easily. We also want to show the tests results rendered in a nice tree view. For that purpose, we send some extra arguments when running the tests (
-Z unstable-options, --format=json, --show-output). However these arguments only work with the default rustc test harness (libtest). If the test does not use the default harness, but a different one, for example when using criterion and similar crates, then the custom harnesses will likely fail because they don't understand the extra arguments. We also don't want to display a tree view with the results but simply show the terminal output. There are more details about this in the RustRover tracking issueSo, for RustRover is very important to identify if a target uses or not a custom harness. For now our only workaround is to parse
Cargo.tomldirectly, but this creates a number of problems. It would very handy to get this information fromcargo metadata, which we already query.Fixes #11846
How to test and review this PR?
Testing and reviewing should be straight forward.