Add an unopy interface

[robfalck]

Purpose

This pull request adds an interface to Uno to pyoptsparse.
Uno is a C++ package for nonlinear constrained optimization, with a python interface (unopy).
Uno provides another option that provides both SQP and interior-point capability depending on the settings.

Other notes about the Uno interface

• Supports sparse jacobians
• Supports user termination
• Output can be sent to a stream specified by the user using 'logger_stream' option, with verbosity set by the 'logger' option.

This implementation requires unopy 0.4.7 or later.

Expected time until merged

A few weeks (not urgent).

Type of change

• Bugfix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (non-backwards-compatible fix or feature)
• Code style update (formatting, renaming)
• Refactoring (no functional changes, no API changes)
• Documentation update
• Maintenance update
• Other (please describe)

Testing

Running the test suite will test Uno against several existing test problems.

Checklist

• I have run ruff check and ruff format to make sure the Python code adheres to PEP-8 and is consistently formatted
• I have formatted the Fortran code with fprettify or C/C++ code with clang-format as applicable
• I have run unit and regression tests which pass locally with my changes
• I have added new tests that prove my fix is effective or that my feature works
• I have added necessary documentation

Resolves cvanaret/Uno#318

[robfalck]

@cvanaret I would appreciate your review again. Theres a skipped test of problem 109 (TP109) from the Schittkowski test that I wasn't able to get to converge with Uno.

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 82.99%. Comparing base (769adbc) to head (0d02fc9).

Additional details and impacted files

@@           Coverage Diff           @@
##             main     #483   +/-   ##
=======================================
  Coverage   82.99%   82.99%           
=======================================
  Files           1        1           
  Lines         147      147           
=======================================
  Hits          122      122           
  Misses         25       25           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[cvanaret]

@robfalck thanks for the PR!

I have an AMPL model of hs109 and Uno indeed doesn't converge with hessian_model="LBFGS" + the default quasi_newton_memory_size=6. It looks like the Hessian approximation is too positive definite and the solver takes tiny steps.
It does converge for quasi_newton_memory_size=15 but that's just lucky.

I think I should monitor progress and flush the limited memory if the solver stalls. Note that I'm also planning to implement interior points with L-BFGS in the next few days.

Will the exact Hessian callback be available in the near future?

[robfalck]

Thanks for checking. That last question is up to the mdolab folks since it's a significant change.

[cvanaret]

unopy v0.4.1 is out with:

• L-BFGS for interior-point methods:

uno_solver.set_preset("ipopt")
# either set the Hessian model explicitly
uno_solver.set_option("hessian_model", "LBFGS")
# or let Uno default to L-BFGS if no Hessian is provided

• a function set_logger_stream:

uno_solver.set_logger_stream(sys.stdout)
uno_solver.optimize(model)
sys.stdout.flush()

# or

with open("log.txt", "w") as f:
    uno_solver.set_logger_stream(f)
    uno_solver.optimize(model)
    f.flush()

[marcomangano]

Will the exact Hessian callback be available in the near future?

It would be a very cool capability to add since most gradient-based optimizers should support that, but I don't believe that the lab (pinging @A-CGray @eirikurj ) is interested in adding this capability in the near term. Most of the (high-fidelity) codes used for engineering MDAO do not provide second derivatives.

I haven't thought this through yet but I think it would require a change in API of the _masterFunc and _masterFunc2 callback function wrappers and some non-naive way to store sparsity information, so it would not be a trivial implementation. I also know that @ewu63 has been considering some major code refactoring at some point, so this change could fall under that bigger effort.

[A-CGray]

Will the exact Hessian callback be available in the near future?

It would be a very cool capability to add since most gradient-based optimizers should support that, but I don't believe that the lab (pinging @A-CGray @eirikurj ) is interested in adding this capability in the near term. Most of the (high-fidelity) codes used for engineering MDAO do not provide second derivatives.

I wouldn't complain if somebody else implemented it, but we have no plans to. We are rarely (if ever) solving problems where the Hessian is computable in any practical sense. If somebody was going to implement this, we may as well implement the ability to use jacobian-vector products as well.

Thanks for doing this work @robfalck , I have been meaning to try out Uno for a long time, now I might actually get around to it!

[marcomangano]

Hi all, I am working through some testing environment issues on my side and I am currently running stuff on docker to prepare the other PR. Anyway, I kept bumping into this weird error that makes all the Uno tests fail when running testflo:

TypeError: protected_actual_reduction_macheps_coefficient is not of type int triggered on this line

Even if I make sure that the option defined in this line is actually an int, the error persists. Only if I remove the option altogether the tests actually converge. @robfalck do you have any issues like that on your side?

[cvanaret]

Hi @marcomangano, can you try 10. instead of 10?
The option should be a float but it looks like the wrong unopy overload is called.

[marcomangano]

Hi @marcomangano, can you try 10. instead of 10? The option should be a float but it looks like the wrong unopy overload is called.

Ha, that fixes it, good catch! I am not very fluent in cpp, is it because 10 is read as an int by the cpp layer here? Anyway, pushing the quick fix here for now. I cannot commit directly to Rob's branch, but that is an easy fix for later.

[cvanaret]

Ha, that fixes it, good catch! I am not very fluent in cpp, is it because 10 is read as an int by the cpp layer here?

Great!
Yes, it picks the right function given the type of the argument, so with 10 it will naturally pick the integer one. A bit stringent, but this avoids implicit conversions between types.

[robfalck]

Don't have access to a computer but hopefully my update today via GitHub mobile fixed it.

[cvanaret]

unopy v0.4.2 is out. The ipopt preset with L-BFGS Hessian should be able to solve hs109.

[cvanaret]

The callback is available in unopy 0.4.7, see the example file.

[A-CGray]

Thanks @cvanaret , I've been working on getting that major iteration tracking working with that callback. I have it working but there seems to have been some change between unopy v0.4.6 and 0.4.7 that means Uno can no longer converge the rosenbrock test problem. With 0.4.6 it converges in 47 major iterations, whereas with 0.4.7 I killed the process after about 1500 major iterations without convergence. I've attached two log files from the two versions, where the 0.4.7 run was limited to 100 major iterations, I don't see any obvious differences in the algorithm settings.

Uno 046-rosenbrockOutput.log
Uno 047-rosenbrockOutput.log

[ewu63]

I think we discussed making a conda-recipe for uno/unopy to make it a more easily-installed depdendency. @ewu63 have you had a chance to start that?

I have been away for a bit but will get back to this next week. I had started something which compiled fine locally but had trouble running it within the conda-forge docker container ecosystem, where some of the variables were not being passed in correctly for cmake to find library paths for blas etc.

[A-CGray]

Tests will fail until we merge the PR updating our docker images to unopy 0.4.7

[cvanaret]

Thanks @cvanaret , I've been working on getting that major iteration tracking working with that callback. I have it working but there seems to have been some change between unopy v0.4.6 and 0.4.7 that means Uno can no longer converge the rosenbrock test problem. With 0.4.6 it converges in 47 major iterations, whereas with 0.4.7 I killed the process after about 1500 major iterations without convergence. I've attached two log files from the two versions, where the 0.4.7 run was limited to 100 major iterations, I don't see any obvious differences in the algorithm settings.

Uno 046-rosenbrockOutput.log Uno 047-rosenbrockOutput.log

Ah, I changed the initialization of L-BFGS between 0.4.6 and 0.4.7. This is most likely the culprit (no wonder, it's got the number 666 😈 ). I will revert. I will also add a CI workflow that runs the pyOptSparse benchmark with the latest Uno changes, to make sure unopy doesn't regress.

[cvanaret]

Fixed in unopy v0.4.8!

[A-CGray]

I just merged a PR that bumps our docker images up to unopy 0.4.8, usually takes ~4 hours to run all the CI tests and push the new images, at which point the tests here should run successfully.

[ewu63]

Just a heads up that I have managed to get unopy working in conda-forge after some back and forth. PR is here and there will be some adjustments + potential porting of patches upstream. We can coordinate the activities in that PR, and feel free to add yourselves as maintainers.

[A-CGray]

Thanks for all your work on this @robfalck and @cvanaret , this is a great addition to pyOptSparse!

[marcomangano]

Great work, thanks a lot!

[robfalck]

Thanks for the continued support for pyoptsparse. It really is a valuable tool. And @cvanaret, thanks for an open-source tool as capable as Uno.

It's been a pleasure collaborating with all of you on this.

[cvanaret]

Thank you all for your time! I can't wait to improve the existing Uno solvers and try out new ones.