Autonomous optimization refers to the design of feedback controllers that steer a physical system to a steady state that solves a predefined, possibly constrained, optimization problem. As such, no exogenous control inputs such as set points or trajectories are required. Instead, these controllers are modeled after optimization algorithms that take the form of dynamical systems. The interconnection of this type of optimization dynamics with a physical system is however not guaranteed to be stable unless both dynamics act on sufficiently different timescales. In this paper, we quantify the required timescale separation and give prescriptions that can be directly used in the design of this type of feedback controllers. Using ideas from singular perturbation analysis, we derive stability bounds for different feedback laws that are based on common continuous-time optimization schemes. In particular, we consider gradient descent and its variations, including projected gradient, and Newton gradient. We further give stability bounds for momentum methods and saddle-point flows. Finally, we discuss how optimization algorithms such as subgradient and accelerated gradient descent, while well-behaved in offline settings, are unsuitable for autonomous optimization due to their general lack of robustness.

中文翻译：

---

自主优化中的时间尺度分离


自主优化是指反馈控制器的设计，该反馈控制器将物理系统引导至稳定状态，从而解决预定义的、可能受约束的优化问题。因此，不需要外部控制输入，例如设定点或轨迹。相反，这些控制器是根据动态系统形式的优化算法建模的。然而，这种类型的优化动力学与物理系统的互连不能保证稳定，除非两种动力学作用在足够不同的时间尺度上。在本文中，我们量化了所需的时间尺度分离，并给出了可直接用于此类反馈控制器设计的规定。利用奇异扰动分析的思想，我们推导出基于常见连续时间优化方案的不同反馈定律的稳定性界限。特别是，我们考虑梯度下降及其变化，包括投影梯度和牛顿梯度。我们进一步给出动量方法和鞍点流的稳定性界限。最后，我们讨论次梯度和加速梯度下降等优化算法虽然在离线设置中表现良好，但由于普遍缺乏鲁棒性而不适合自主优化。