Setup solid-state electrolytes benchmark

[ThiagoReschutzegger]

Pre-review checklist for PR author

• I've confirmed the contribution guidelines.

Summary

This pull request introduces initial support for benchmarking solid-state electrolytes (SSEs) within the project. It adds documentation, configuration, metrics, calculation scripts, and an app interface for the new benchmark, called SSE-MD.

Implemented the main calculation script for the SSE-MD RDF benchmark, including test generation, MD simulation setup, and trajectory processing for multiple ML models. Added a metrics definition file for SSE-MD, specifying the score metric and its properties.

Linked issue

Resolves #393

Progress

• Calculations
• Analysis
• Application
• Documentation

Testing

I tested the benchmark on MACE-MP-0 and MACE-MP-0B3.

New decorators/callbacks

No new callbacks are required.

[joehart2001]

Just putting this here do i don't forget and to tag @ElliottKasoar: as the PNAS rdf metric is becoming a common metric to use, we will add a generalised version you'll be able to call. i'll tag you when thats been implemented

[joehart2001]

Hi @ThiagoReschutzegger would you also be able to add diffusivity as a metric? I think this could be really valuable if possible

[ElliottKasoar]

Hi @ThiagoReschutzegger, sorry for the very delayed review.

In addition to the above comments, please can you take a look at our new filtering guidelines: https://ddmms.github.io/ml-peg/developer_guide/filter.html

The principles are relatively simple, but you do have to be a little careful, so again, if anything is unclear, please do ask!

(You'll need to rebase to test this)

Could you also make sure that the tests/pre-commit are passing?

[ThiagoReschutzegger]

Thanks @ElliottKasoar!

I've incorporated all your suggestions to the code. Let me know if anything else is needed!

[ElliottKasoar]

Thanks @ThiagoReschutzegger for addressing so much!

I've left a few more suggestions, which are mostly pretty minor.

The biggest thing is to please take a look at the element filtering guidelines I linked before:

https://ddmms.github.io/ml-peg/developer_guide/filter.html

In particular, the analysis should save an info.json file along with the tables, plots etc.that can be used by the app to enable element filtering, and the analysis should work if some models failed to run some of the MD/crashed/haven't run yet (all of which should give None/NaN for the metric).

Please also make sure you re-run the pre-commit and fix anything that comes up, as this is currently failing.

There seems to be a problem with the custom --system-id too (affecting the liquid densities, not this, but caused by changes here), but I can look into that.

[ElliottKasoar]

Apologies if we've discussed this before, but how much have you considered these thresholds?

If you're happy with them, great, but we're also happy to discuss what they should represent more if you'd like.

[ElliottKasoar]

This is probably ok, but normally what we've done in the past is save the reference data to the calculations outputs directory, and then use that, which means the outputs is self-contained in a sense since it has everything you need for analysis, so unless there's a clear motivation not to do this, I'd consider it

If you keep it as is, it seems that the new key/filename that I'd put in a suggestion got lost?

[ElliottKasoar]

For the custom parser, could you please use:

    try:
        parser.addoption("--system-id", action="store", default=0, type=int)
    except ValueError:
        pass

changing this both in the conftest.py you've added, and also in ml_peg/calcs/molecular_dynamics/liquid_densities/conftest.py?

They run fine independently, but when both are discovered, it raises an error, and I think this is the most straightforward solution.

If it works, it should fix the CI tests.

[joehart2001]

hey @ThiagoReschutzegger, just a ping on this. would be great for us to get some of these running soon