Improving dune support

[Alizter]

I would like to improve the dune support for mutaml. Could you give a run down of the current issues affecting it? I can see what we can do on the Dune side to accommodate your use case.

[jmid]

First of all, help on improving dune support would be very welcome! 🙏

Mutations are modeled with a ppxlib-based instrumentation that runs over the targeted files.
The preprocessor creates an individual lib.muts file for each instrumented lib.ml file (to record which mutations it did) and an overview file mutaml-mut-files.txt listing them. These files are subsequently read when invoking mutaml-runner.
See for example

mutaml/test/testproj-1-module.t/run.t

Lines 52 to 58 in 64b95b0

	$ ls _build/default/*.exe _build/default/*.ml _build/default/*.muts _build/default/*.txt
	_build/default/lib.ml
	_build/default/lib.muts
	_build/default/lib.pp.ml
	_build/default/mutaml-mut-files.txt
	_build/default/ounittest.exe
	_build/default/ounittest.ml

Currently the above files are not registered with dune

• on the instrumentation side when they are produced and
• on the mutaml-runner side when they are consumed

As a result, any dune invocation in between tends to delete them.
From the error message in #18 I suspect this is also the reason that expect tests are not working.

1.5 years ago when I was hacking on this, I struggled to get this to play nicely with dune and ppxlib's cookie mechanism, so I ended up with the above setup somewhat in anger. A lot has happened with both dune and ppxlib since, so it may be possible to achieve such functionality without special casing (too much).

[WardBrian]

Would this be resolved by providing an environment variable to change the location, a-la BISECT_FILE for ppx_bisect?

[jmid]

When originally hacking on this, I needed to output temporary file(s) to record which mutations were created.
I tried using ppxlib's cookie mechanism but found it too constrained - and was simultaniously battling dune removing unregistered (target) files.

AFAIR, the instrumenter is invoked once per (instrumented) source file, and so we write one output file for each.
Writing everything to one file at the project's root (or BISECT_FILE) may be problematic, as the output of two subsequent
instrumentation invocations may be appended to the same file if it is not properly cleared.

As this was on a time-limited contract, I settled on "works if you do exactly like this"-steps and prioritized progress on the mutation tester itself instead.

I'd be happy to receive input on ways to store this info, that plays well with both ppxlib and dune, so it can be reworked.
There's probably a nice way to achieve it. In fairness I've also learned a bit more about dune since then,
so I may be able to figure it out with a fresh look and a bit of tinkering.

Finally, please note that mutaml is a private nights-and-weekends project ATM.

[WardBrian]

BISECT_FILE is actually a file prefix, I believe for similar reasons of avoiding overwriting itself.

I may take a pass at doing something like this in my free time and report back if it does improve the dune picture