Add PRIMA optimisers

[zaikunzhang]

Dear PINTS maintainers,

This is Dr. Zaikun Zhang from The Hong Kong Polytechnic University. Together with Professor N.I.M. Gould, I am responsible for maintaining the renowned derivative-free optimization solvers of the late Professor M.J.D. Powell, namely COBYLA, UOBYQA, NEWUOA, BOBYQA, and LINCOA. I am the author of PRIMA, which provides the reference implementation for these solvers. They are widely used by engineers and scientists. For instance, see Section 1 of a recent paper on Powell's solvers as well as the Google searches of COBYLA and BOBYQA.

I note that there has been an interest in including these solvers in PINTS, as mentioned in #684 by @MichaelClerx . I hope PRIMA will help you to achieve this goal. I will be happy to assist on the Fortran side if you would like to do so.

Note that the old Fortran 77 implementation of the aforementioned solvers is buggy and not maintained anymore. Even though the original Fortran 77 code is truly a masterpiece, it contains many bugs (mostly due to the language itself), which can lead to segmentation faults or infinite loops. For example, see Section 4.4 of the above paper and many GitHub issues. It is strongly discouraged to use the Fortran 77 version of these solvers anymore.

Thanks and regards,
Zaikun ZHANG
Ph.D. and Assistant Professor
Dept. App. Math., Hong Kong Polytechnic University

[MichaelClerx]

Dear Dr Zhang,

Thank you for getting in touch! It would be very nice to have at least one (e.g. BOBYQA) one these methods available via PINTS.

In the design of PINTS, we have chosen to make it a pure-python package, which means we do not want PINTS to compile any C or fortran extensions, but we don't mind relying on pip-installable packages that do. I think this means we should be able to add PDFO as a dependency and interface with that?

A second issue is that we require every optimiser and sampling method to be useable with a 1-call-per-iteration system, so instead of

result = optimiser.solve(fun, x0, ...)

we stick to a calling procedure like

instance = optimiser(fun, x0, ...)
while stopping_criterion:
x_next = optimiser.ask()
f_next = evaluate_score_function(x_next)
optimiser.tell(f_next)
update stopping_criterion using custom checks and/or optimiser input

the reason for this is we would like to work on large and difficult to parallelise score functions, and a structure like the above allows the user to be responsible for calling the evaluation function in whatever way they think is best.

Do you think it would be possible to use PDFO this way?

Best wishes,
Michael

[zaikunzhang]

Hi @MichaelClerx Michael,

Sure, it is possible to add PDFO as a dependency of PINTS, since PDFO is available via PyPi.

However, PRIMA is not PDFO. We will also provide a Python interface for PRIMA very soon.

To call PDFO or PRIMA solvers in the way you described is nontrivial. It would necessitate a re-implementation of the solvers. Or maybe it can be done via a particular implementation of the objective function, which is a callback to the solvers? I am not sure.

Thank you.

Zaikun