Speed markers

[joehart2001]

Pre-review checklist for PR author

PR author must check the checkboxes below when creating the PR.

• I've confirmed the contribution guidelines.
• I have reviewed and understand all AI-generated code in this PR.
• I have added human-written tests for the new logic.
• I have properly cited any upstream algorithms or libraries the AI utilized.
• I have disclosed significant AI tool usage in the PR description.

Summary

• Add test speed markers to each test

• add a test speed summary on the main page, as well on framework pages

• more thorough marking of test speeds, where all tests are marked with fast, medium, slow, very slow, multi-day

• add ability to time tests

• Run fast and medium benchmarks by default

• requires rerun of analysis

Linked issue

Resolves #

Testing

[ElliottKasoar]

From a UI perspective, the numbers (2 hours, 2 hours, 3 min etc.) aren't entirely obvious in terms of what they represent.

I'm particularly confused by the 3 min for the slow tests?

[joehart2001]

From a UI perspective, the numbers (2 hours, 2 hours, 3 min etc.) aren't entirely obvious in terms of what they represent.

I'm particularly confused by the 3 min for the slow tests?

so the NEB tests were marked as slow but they only took 1 min each on gpu, so have now moved to fast. i have also moved elasticity and phonons to slow category now as they are under 10h

[ElliottKasoar]

From a UI perspective, the numbers (2 hours, 2 hours, 3 min etc.) aren't entirely obvious in terms of what they represent.
I'm particularly confused by the 3 min for the slow tests?

so the NEB tests were marked as slow but they only took 1 min each on gpu, so have now moved to fast. i have also moved elasticity and phonons to slow category now as they are under 10h

Some of these are little tricky since whenever there's an optimisation, there's sometimes more than two orders of magnitude difference in runtime between models, depending on how many steps they need (on top of the inference difference).

I agree the Li diffusion is probably still "fast" as is, but ideally we could take this into account a little more somehow, since it's quite different to singlepoint/MD tests where they're a bit more predictable.

[joehart2001]

From a UI perspective, the numbers (2 hours, 2 hours, 3 min etc.) aren't entirely obvious in terms of what they represent.
I'm particularly confused by the 3 min for the slow tests?

so the NEB tests were marked as slow but they only took 1 min each on gpu, so have now moved to fast. i have also moved elasticity and phonons to slow category now as they are under 10h

Some of these are little tricky since whenever there's an optimisation, there's sometimes more than two orders of magnitude difference in runtime between models, depending on how many steps they need (on top of the inference difference).

I agree the Li diffusion is probably still "fast" as is, but ideally we could take this into account a little more somehow, since it's quite different to singlepoint/MD tests where they're a bit more predictable.

yeah i agree with this, but i think we just need to stay true to our baseline. its only rough guidance anyway and in the case of the NEB test, it took under a minute with mace-mp-0a, so the difference would only be a minute or so with different models. i know this can compound, but i think we just have to say "these are the results for X model on Y hardware, hopefully this is useful guidance but take it with a pinch of salt". im also sure we will come up with better ideas down the line, maybe when we have improved logging

[joehart2001]

heres it updated. for the 3 very slow tests, im running them now to see how long (low dimensional relaxation test etc)

i also think this format is much more clear about what everything means