Optimization problems in engineering and applied mathematics are typically solved in an iterative fashion, by systematically adjusting the variables of interest until an adequate solution is found. The iterative algorithms that govern these systematic adjustments can be viewed as a control system. In control systems, the output is measured and the input is adjusted using feedback to drive the error to zero. Similarly, in iterative algorithms, the optimization objective is evaluated and the candidate solution is adjusted to drive it toward the optimal point. Choosing an algorithm that works well for a variety of optimization problems is akin to robust controller design. Just as dissipativity theory can be used to analyze the stability properties of control systems, it can also be used to analyze the convergence properties of iterative algorithms. By defining an appropriate notion of “energy” that dissipates with every iteration of the algorithm, the convergence properties of the algorithm can be characterized. This article formalizes the connection between iterative algorithms and control systems and shows through examples how dissipativity theory can be used to analyze the performance of many classes of optimization algorithms. This control-theoretic viewpoint enables the selection and tuning of optimization algorithms to be performed in an automated and systematic way.

中文翻译：

---

优化算法分析：耗散性方法


工程和应用数学中的优化问题通常以迭代方式解决，通过系统地调整感兴趣的变量，直到找到适当的解决方案。控制这些系统调整的迭代算法可以被视为控制系统。在控制系统中，测量输出并使用反馈调整输入以将误差驱动至零。类似地，在迭代算法中，评估优化目标并调整候选解决方案以使其接近最优点。选择适用于各种优化问题的算法类似于鲁棒控制器设计。正如耗散理论可以用来分析控制系统的稳定性特性一样，它也可以用来分析迭代算法的收敛特性。通过定义随着算法的每次迭代而消散的适当的“能量”概念，可以表征算法的收敛特性。本文形式化了迭代算法和控制系统之间的联系，并通过示例展示了如何使用耗散理论来分析多种优化算法的性能。这种控制理论的观点使得优化算法的选择和调整能够以自动化和系统的方式进行。