A header-only LBFGS unconstrained optimizer.

LBFGS-Lite is a C/C++ header-only library for unconstrained optimization on twice continuously differentiable functions. This code is a modified version of liblbfgs, so that only necessary part is preserved for simplicity.

• Only one header file "lbfgs.hpp" is required for usage.

• No dependencies except C/C++ standard library.

• The library is an implementation of Limited-Memory Broyden-Fletcher-Goldfarb-Shanno (LBFGS) with More-Thuente Line Search to ensure linear time/space complexity and strong Wolfe conditions in each iteration.

• The objective function is required to be twice continuously differentiable on its domain.

• The library provides an additional callback to utilize externally provided maximum feasible stepsize. It can be helpful when the function is closed on a bounded open domain instead of the whole Euclidean space. The callback avoids function evaluations at infeasible region. This can help a lot for closed functions as long as the Newton step is not always clipped.

• Engineering features such as skipping update at extremely small curvature are also adopted.

• Instruction set dependent parts and L1 regularization parts in original code are removed. Multiple files are reorgainzed here with some additional modification.

See "lbfgs_sample.cpp" for details.

• LBFGS is only proved to be globally convergent (convergent to stationaries for any initial guess) under convexity assumption, while it also works in many nonconvex cases and yields almost the best results. Slight modification in updating can ensure global convergence without convexity assumption, following a work by Fukushima and Li.

• Although More-Thuente line search is already good enough, we plan to compare it with Hager-Zhang line search and Nocedal's zoom line search. Hager-Zhang linear search is reported to be full machine accuracy while More-Thuente's method can only attain the half. Nocedal's zoom is also reported to be reliable. We will compare them and choose the best one if the improvement is good enough.

• Further code optimization (instruction set independent) and reorganization.

LBFGS-Lite is distributed under the term of the MIT license according the previous version by Okazaki and the initial FORTRAN version by Nocedal. Please refer to LICENSE file in the distribution.

If any bug, please contact Zhepei Wang ([email protected]).