perf: DMRG(2) with smarter adaptive tolerances by lkdvos · Pull Request #455 · QuantumKitHub/MPSKit.jl

lkdvos · 2026-07-10T01:04:49Z

This PR features a somewhat large refactor of the DMRG implementations, and finally fixes #311 hopefully in a more systematic way.

The main idea is that I want to use loose bounds on the krylov solvers, since currently everyone has the wrong assumption that ITensors is a lot faster than MPSKit, which is just due to the default settings.
Nevertheless, while for ITensorMPS you just have to know and read the docs to figure out that if you want to actually converge something for a non-gapped state, I wanted to keep some guarantees on stability and convergence rates.
Therefore, this PR introduces a bunch more knobs to determine adaptive tolerance and krylov settings based on the information that is available:

The tolerance of the krylov solver depends on the global convergence, and oversolving locally (especially for finite systems) has very little benefit.
The tolerance of the krylov solver should not go far below the truncation error incurred in adaptive DMRG schemes, since that error will dominate the eigenvector precision anyways.
The gap of the local effective system can be predicted by investigating the decay rate of the previous sweep, which is theoretically predicted to be geometric for Lanczos solvers. Using this, we can make an educated guess for how gapped the system is and adapt accordingly.

A secondary part of this PR is just me refactoring the DMRG implementations into a more readable and consolidated form, which I think overall just improves the code quailty.

Benchmark summary

Backing up the claims in this PR, I've attached a number of results from investigations I did. The headline is that this new adaptive local eigensolver is an across-the-board improvement over the current main.
Doing a large amount of hyperparameter tests, it seems like the main point is that the results aren't too sensitive to the actual values of the various tolerance factors etc, but the general statement is still true that too high of a precision on the local solvers is not useful at the start of a run, but essential towards the end, specifically for gapless systems. The main thing is that the overhead of switching sides is actually quite large at higher bond dimensions (the SVD, hamiltonian building and environment transfers), such that the extra iterations in the krylov solvers are well worth it because that work has to be done anyways.

I think I'm happy with the current settings, which seems to at least balance having reasonably fast sweeps in the easy regimes and no stagnation in the harder regimes, so for me this PR is now ready.

Benchmark plots

codecov · 2026-07-10T01:28:53Z

Codecov Report

❌ Patch coverage is 90.28571% with 17 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
src/states/orthoview.jl	82.85%	6 Missing ⚠️
src/utility/dynamictols.jl	85.71%	4 Missing ⚠️
src/algorithms/fixedpoint.jl	40.00%	3 Missing ⚠️
src/algorithms/approximate/vomps.jl	75.00%	1 Missing ⚠️
src/algorithms/groundstate/dmrg.jl	98.33%	1 Missing ⚠️
src/algorithms/statmech/vomps.jl	80.00%	1 Missing ⚠️
src/states/finitemps.jl	83.33%	1 Missing ⚠️

Files with missing lines	Coverage Δ
src/MPSKit.jl	`100.00% <ø> (ø)`
src/algorithms/approximate/fvomps.jl	`91.30% <100.00%> (ø)`
src/algorithms/approximate/idmrg.jl	`96.72% <100.00%> (ø)`
src/algorithms/changebonds/optimalexpand.jl	`98.85% <100.00%> (ø)`
src/algorithms/changebonds/randexpand.jl	`63.63% <100.00%> (ø)`
src/algorithms/changebonds/sketchedexpand.jl	`98.43% <100.00%> (ø)`
src/algorithms/excitation/dmrgexcitation.jl	`100.00% <100.00%> (ø)`
src/algorithms/groundstate/idmrg.jl	`99.35% <100.00%> (ø)`
src/algorithms/groundstate/vumps.jl	`81.31% <100.00%> (ø)`
src/algorithms/statmech/idmrg.jl	`97.72% <100.00%> (ø)`
... and 10 more

... and 1 file with indirect coverage changes

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

lkdvos · 2026-07-12T03:19:41Z

Here I've added some results on the comparisons for the sweeptimes, but also on the energy reached after a given walltime, in an attempt to not just measure what the fastest way is to get 10 sweeps done.
What's interesting is that we now consistently outperform ITensorMPS, even without symmetries, but that probably we will have to investigate the single-site version a bit more, since that seems to not perform the way I'd expect.

I'd also say that it seems really hard to make generalizations about the actual walltimes it takes to get to a given energy, since the faster sweeps get compensated with the necessity for more sweeps. In general, this is not faster everywhere, but I think it at least feels faster because the sweeps take a little less time, and it makes a naive benchmark no longer favor the ITensor version that much.

borisdevos

This is amazing work, Lukas! The benchmarks also look very promising.

I was just wondering whether we should be more verbose about how AdaptiveKrylov is now the default everywhere. Also, this is not being compared explicitly in our tests by name, though I'm not sure what the criteria would be.

The refactoring indeed makes the code much easier to follow! The title of the PR could also added "performance" arguably ;)

You should definitely let others review this PR as well, since I'm no expert really, but hopefully I brought up some things with substance.

lkdvos · 2026-07-13T13:28:20Z

Thanks a lot @borisdevos for taking the time to go over this, the feedback is definitely appreciated!

Some more things I would love inputs on:

Looking at the benchmarks right now, it is actually not entirely clear that these looser defaults are really helping. Of course, if the metric is to get fast sweeps this makes them a lot faster, but if the metric is to get to the groundstate faster, this doesn't seem to really be the case, especially in the gapped case or for higher bond dimensions. I might redo an actual hyperparameter sweep in the current implementation and see if I can actually get some more quantitative data on this.
I really don't know what to do with the convergence criterion. The galerkin error seems more principled to me, but especially in the twosite context I'm not even sure if I'm defining this correctly, since the correct thing might be to project onto the twosite left nullspace? Maybe @Jutho can say something more about this. Ultimately I'm happy to in the future work towards making this configurable, but I don't want to tackle that in this PR.
I have a bunch more tricks up my sleeve to make this faster, which mostly have to do with the way the environments are constructed and stored, which arguably will make a much larger difference across the board than this hyperparameter tweaking, so I don't want to spend too much time dwelling on this. I'm probably going to try and add some of these benchmark results to the documentation, where for the comparison with ITensor I will just use the "fixed krylov" settings, which is also what TenPy settled on for their benchmarks.
The conv vs galerkin errors currently have a different scale - the former is 1 - dot(x, y) while the latter is norm = sqrt(dot(...)). However, if I naively square root the former the behavior doesn't really fix my excitations immediately converging issue...

Switch the DMRG/DMRG2 stop test from the overlap-based convergence error `ϵ_conv = 1 - |⟨old|new⟩|` to the Galerkin error (already computed each local update as `ϵ_local`/`ϵ_global`), matching VUMPS/iDMRG. Drop the now-unused `ϵ_conv` bookkeeping from both `local_update!` methods and `find_groundstate!`. Perturb the default `FiniteExcited` initial guess with a small per-site random component. Under the Galerkin criterion an unperturbed ground-state guess can already be (near) an eigenvector of the shifted operator, so convergence would trigger on the first sweep and return the ground state instead of climbing to the excited state. The perturbation preserves spaces (symmetry sector) and bond dimensions. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

DMRG2 reused `HAC2 = Heff * ac2` (needed for the Galerkin `ϵ_local`) as the eigensolver seed. Seeding with H*AC is power-iteration intuition (it amplifies the largest-magnitude eigenvector), which is wrong for `:SR` (smallest algebraic): it de-weights the ground state, and when the effective ground eigenvalue is 0 it annihilates the ground-state component exactly. Since H preserves the ground state's orthogonal complement, the Krylov solve then converges to the smallest nonzero eigenvalue instead — the wrong state. This bites frustration-free/parent Hamiltonians and any H - E0 shift. Seed with the plain two-site center `ac2` instead, consistent with single-site DMRG's `ac_old`. `HAC2` is still computed for the Galerkin gradient only. Verified on a finite TFIM (L=16, g=1) shifted so E0=0: old seeding gives a non-convergent 4.73 (ϵ_conv≈0.88), fixed seeding gives 5e-9≈0 (ϵ_conv≈2e-15); unshifted DMRG2 still matches single-site DMRG to 1e-14. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

…port different metrics and ignore ones that aren't present

`gauge2!` unconditionally promoted the SVD center matrix to complex (`complex(c)`), which for a real-valued state forced a complexify then coerce-back-to-real roundtrip when installing the tensors. Only complexify when the state's scalartype is itself complex. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

The two-site Galerkin error was projected onto BOTH the left and right null spaces (the (null,null) bond-expansion corner), which is exactly what the truncated SVD discards. It therefore collapsed to the truncation floor after a single local solve and spuriously signalled convergence after one sweep, leaving DMRG2 under-converged (e.g. the multifusion Ising ground state stopped 2.6e-6 above exact with variance 3e-5 after one sweep). Project onto the LEFT null space of `AL[pos]` only, matching single-site `calc_galerkin`, so the measure faithfully tracks convergence. The faithful measure floors at the truncation error (the gradient of a truncated state toward the untruncated optimum), so `ϵ_global <= tol` is unreachable under truncation. Converge instead at `max(tol, maxϵ_trunc)` — once the gradient reaches the level the truncation allows. With no truncation (single-site / QR gauge, `ϵ_trunc = 0`) this reduces to the plain `<= tol`. Note: penalized runs (`FiniteExcited`) can still floor above the truncation and hit maxiter; a more general stop criterion is left to a follow-up. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

`updatetol` kept a docstring before the `adapt_solver` refactor; restore one for the deprecated shim so the exported name stays documented. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

lkdvos · 2026-07-17T18:18:19Z

I've updated the implementation and the PR description, benchmark results at the top.

The important changes made here:

using galerkin error also for twosite convergence criterion works, but probably we should make some of this configurable in the future
FiniteExcited should seed from perturbed states to not make that stagnate

lkdvos force-pushed the ld-dynamictols branch 2 times, most recently from 8e22df4 to 8ab7f54 Compare July 12, 2026 00:06

lkdvos marked this pull request as ready for review July 12, 2026 00:06

borisdevos reviewed Jul 13, 2026

View reviewed changes

Jutho reviewed Jul 14, 2026

View reviewed changes

Comment thread src/algorithms/groundstate/dmrg.jl Outdated

Jutho reviewed Jul 14, 2026

View reviewed changes

Comment thread src/algorithms/groundstate/dmrg.jl Outdated

Jutho reviewed Jul 14, 2026

View reviewed changes

Comment thread src/algorithms/groundstate/dmrg.jl

Jutho reviewed Jul 14, 2026

View reviewed changes

Comment thread src/utility/dynamictols.jl

This was referenced Jul 14, 2026

Performance with long range interactions is really poor #459

Open

DMRG3S: strictly single-site DMRG subspace expansion (Hubig et al. 2015) #460

Open

lkdvos changed the title ~~refactor: DMRG(2) with smarter adaptive tolerances~~ perf: DMRG(2) with smarter adaptive tolerances Jul 16, 2026

lkdvos force-pushed the ld-dynamictols branch from f426dd4 to 4e0a673 Compare July 16, 2026 21:47

lkdvos and others added 15 commits July 17, 2026 13:56

refactor fixedpoint to avoid warnings

924c5e0

refactor DMRG implementation

174d864

refactor gauging

abe50dd

add AdaptiveKrylov solver

1caab6f

refactor DMRG2 to be in line with DMRG

89cf07a

small type stability improvements

16797ea

small fixes

69c0d16

restore old DMRG convergence behavior

8d33792

update default values

5c20e82

rework and unify adaptive solver interface - use kwargs to easily sup…

4ee1bc0

…port different metrics and ignore ones that aren't present

update defaults once more

fd9ba08

revert DynamicTol behavior

6d2b2f8

lkdvos and others added 3 commits July 17, 2026 13:56

restore updatetol docstring

4b58685

`updatetol` kept a docstring before the `adapt_solver` refactor; restore one for the deprecated shim so the exported name stays documented. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

cleanup

e3e7012

lkdvos force-pushed the ld-dynamictols branch from 8a5bbd0 to e3e7012 Compare July 17, 2026 18:03

lkdvos merged commit 5bbab08 into main Jul 18, 2026
27 of 30 checks passed

lkdvos deleted the ld-dynamictols branch July 18, 2026 14:51

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

perf: DMRG(2) with smarter adaptive tolerances#455

perf: DMRG(2) with smarter adaptive tolerances#455
lkdvos merged 18 commits into
mainfrom
ld-dynamictols

lkdvos commented Jul 10, 2026 •

edited

Loading

Uh oh!

codecov Bot commented Jul 10, 2026 •

edited

Loading

Uh oh!

lkdvos commented Jul 12, 2026

Uh oh!

borisdevos left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

lkdvos commented Jul 13, 2026 •

edited

Loading

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

lkdvos commented Jul 17, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Uh oh!

Conversation

lkdvos commented Jul 10, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Benchmark summary

Uh oh!

codecov Bot commented Jul 10, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

lkdvos commented Jul 12, 2026

Uh oh!

borisdevos left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

lkdvos commented Jul 13, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

lkdvos commented Jul 17, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

lkdvos commented Jul 10, 2026 •

edited

Loading

codecov Bot commented Jul 10, 2026 •

edited

Loading

lkdvos commented Jul 13, 2026 •

edited

Loading