Add battery electrolyte densities benchmark - #778
Open
KMNitesh05 wants to merge 1 commit into
Open
Conversation
NPT density benchmark for 25 Na-ion battery electrolyte and neat solvent systems from arXiv:2603.20183, compared against experimental densities at 298.2 K. Adds calc, analysis and app stages plus documentation. Input structures are downloaded from https://github.com/KMNitesh05/sodium-ion-battery-electrolyte-dataset
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Pre-review checklist for PR author
Summary
NPT density benchmark for 25 battery electrolyte systems, compared against experimental densities at 298.2 K. Six neat glyme and carbonate solvents (DME, DEGDME, TEGDME, PC, DEG, DMC) plus 19 electrolytes formed from NaPF6, NaOTf, NaTFSI and KPF6 at 0.1, 0.5 and 1.0 M. Systems range from roughly 500 to 2200 atoms.
Because the same salts recur across several solvents and concentrations, the benchmark separates general liquid-density performance from the harder question of whether a potential handles concentrated ionic environments.
Structures and reference densities are published at https://github.com/KMNitesh05/sodium-ion-battery-electrolyte-dataset, and the underlying paper is arXiv:2603.20183.
Protocol: NPT with the isotropic MTK barostat at 298.2 K and 1 atm, 1 fs timestep, 150 ps per system. The first 50 ps is discarded and the remaining 100 ps averaged. Metrics are MAE, RMSE and MAPE.
Deliberate differences from
liquid_densitiesFour places where this diverges from the benchmark it is modelled on. All are one-line changes if you would rather I matched upstream:
ADD_D3 = False). The reference densities come from a dispersion-inclusive OMol25-trained potential, so a runtime D3 correction on top would double-count. One consequence worth flagging: the-D3display suffix frombuild_dispersion_name_mapwill be misleading for models not trained on dispersion. Happy to follow your preference on how that is handled.badthresholds of 0.1 g/cm³ rather than 0.4. These densities span 0.861–1.291 g/cm³, so a 0.4 g/cm³ error is a 30–45% miss; at that scale the normalised score saturates near 1.0 for every model and stops discriminating.liquid_densitiesfield-for-field, so the log-parsing route remains available if you prefer consistency across the two benchmarks.AI tool usage
The code in this PR was drafted with AI assistance (Claude), using
liquid_densitiesas the reference implementation, and reviewed by me before submission. The AI also generated the packaging script in the dataset repository that writesexp_densityandexp_temperatureinto the extxyz headers.No external algorithms were introduced. The MD driver is ASE's
IsotropicMTKNPT, MAE and RMSE come fromml_peg.analysis.utils.utils, and the decorators/callbacks used are existing ML-PEG ones.mapeis a five-line helper defined locally in the analysis module becauseanalysis/utils/utils.pyprovidesmaeandrmsebut not MAPE — happy to move it into the shared utils if you would prefer it there.Linked issue
Part of #358.
This PR delivers the 25 Na-ion systems only. The 23 Zn-ion systems also scoped in #358 belong to a dataset owned by ESRA whose paper is not yet published, so I have deliberately written "Part of" rather than "Resolves" to avoid auto-closing #358 on merge. The benchmark directory is named
battery_electrolyte_densitiesso the Zn systems can be added later without a rename. Happy to split #358 into two issues instead if that is tidier.Note also that #358 lists the category as Physicality, whereas this code sits under
molecular_dynamicsalongsideliquid_densitiesandwater_density. I thinkmolecular_dynamicsis the better home for an NPT density benchmark, but say the word and I will move it.(#773 was a duplicate issue I opened by mistake and have now closed.)
Progress
All four stages are implemented, but no production results exist yet — the benchmark has only been exercised with the
mockmodel. The analysis and app stages are a first pass modelled closely onliquid_densities; @joehart2001 offered to help with those two, so please do take them over or rewrite as you see fit.Testing
Only the
mockmodel so far. I do not yet have GPU allocation for 25 systems × N models, so I would welcome guidance on how you would like the production runs done.Verified end to end against stand-in configurations constructed to match the dataset's cell vectors and reference densities exactly:
ase.io.readsystem_id0–24 resolves to the expected systems, and the count check catches a mismatched datasetmetrics.ymlpre-commit run --all-filesis clean, includingnumpydoc-validationThe calc test is marked
very_slow, so CI will not attempt the MD itself.New decorators/callbacks
None. This reuses
plot_parity,build_table,plot_from_table_columnandstruct_from_scatteras they are. The parity plot uses the existingsymbol_byargument to distinguish pure solvents from the 0.1, 0.5 and 1.0 M electrolytes.