This report provides an introduction to the ensmallen numerical optimization library, as well as a deep dive into the technical details of how it works. The library provides a fast and flexible C++ framework for mathematical optimization of arbitrary user-supplied functions. A large set of pre-built optimizers is provided, including many variants of Stochastic Gradient Descent and Quasi-Newton optimizers. Several types of objective functions are supported, including differentiable, separable, constrained, and categorical objective functions. Implementation of a new optimizer requires only one method, while a new objective function requires typically only one or two C++ methods. Through internal use of C++ template metaprogramming, ensmallen provides support for arbitrary user-supplied callbacks and automatic inference of unsupplied methods without any runtime overhead. Empirical comparisons show that ensmallen outperforms other optimization frameworks (such as Julia and SciPy), sometimes by large margins. The library is available at https://ensmallen.org and is distributed under the permissive BSD license.

中文翻译：

---

使用 ensmallen 进行灵活的数值优化

本报告介绍了 ensmallen 数值优化库，并深入探讨了其工作原理的技术细节。该库为任意用户提供的函数的数学优化提供了一个快速灵活的 C++ 框架。提供了大量预先构建的优化器，包括随机梯度下降和拟牛顿优化器的许多变体。支持多种类型的目标函数，包括可微、可分离、约束和分类目标函数。新优化器的实现只需要一种方法，而新的目标函数通常只需要一种或两种 C++ 方法。通过内部使用 C++ 模板元编程，ensmallen 为用户提供的任意回调和未提供方法的自动推理提供支持，而无需任何运行时开销。实证比较表明，ensmallen 的性能优于其他优化框架（例如 Julia 和 SciPy），有时差距很大。该库可在 https://ensmallen.org 获得，并在宽松的 BSD 许可下分发。