Constraint testing: score generated by a single contraint

[niekvanderkooy]

Is your feature request related to a problem? Please describe.
I am trying to unit test that a constraint in situation X generates a (better/worse/different) penalty than in situation Y. I would prefer not to hard code expected penalty values, since these may change, just that two situations are different.
However, a SingleConstraintVerification only provides methods to compare with known values: penalizes(long times), penalizedBy(long matchWeightTotal), penalizesByMoreThan(String s, long l), etc.

Describe the solution you'd like
There are two possible solutions I can think of, which would both work for me:

Comparing two solutions through the constraintVerifier
constraintVerifier.verifyThat(someConstraint).givenSolution(Solution_ someSolution).penalizesByMoreThan(Solution_ someOtherSolution)

Being able to obtain a score through the constraint verifier
Although I realise the API might be a little confusing in my example below (getting a score from a ConstraintVerifier ) something like constraintVerifier.verifyThat(someConstraint).givenSolution(Solution_ someSolution).score() would allow me to write the assertion myself:

HardSoftScore score1 = constraintVerifier.verifyThat(someConstraint).givenSolution(Solution_ someSolution).score()
HardSoftScore score2 = constraintVerifier.verifyThat(someConstraint).givenSolution(Solution_ otherSolution).score()

assertTrue(score1.compareTo(score2) > 0)

Describe alternatives you've considered
I am currently working through the SolutionManager with solutionManager.analyse(solution) and comparing those scores. However, this includes all constraints not just the one I'm trying to test, so this feels more fragile.

[PatrickDiallo23]

@triceo do you mind if I take a look at this?
I also observed that it could be related to issue 937

[triceo]

It seems that I completely missed this one. :-( Sorry, @niekvanderkooy, must have gotten lost among the notifications during my PTO at the time.

@PatrickDiallo23 Feel free to take this one. I would only suggest you wait until we release 2.0.0 next month; the solver codebase is currently undergoing heavy changes and you'd be only asking for merge conflicts.

When it comes to how this should be addressed... Looking back, I consider the approach we took here a mistake. As opposed to providing the data and letting users bring their own assertions, we decided to provide the assertions ourselves - and this creates constant calls for more and more custom assertions, which would go away if we just allowed people to write their own.

For that reason, I'm thinking that ConstraintVerifier should provide methods such as getImpact() and getScore(), which would then allow people to do with the information anything that their hearts desire. (This would also address issue 937.)

[niekvanderkooy]

@triceo no problem, it happens! And I agree with the approach of providing the methods to get the score directly.

Looking forward to the patch, @PatrickDiallo23 thanks in advance!

[PatrickDiallo23]

Hi @triceo! Since Timefold 2.0.0 is out, can I take on this issue? I want a confirmation so that I can get started. Thanks!

[triceo]

@PatrickDiallo23 If you want to provide the getters as outlined in #2007 (comment), go ahead. I didn't give it any more thought since then, so it's possible we'll iterate a bit if/when we find some shortcomings of this approach.