1 INTRODUCTION

Real-world systems have to operate in uncertain and changing environments, thus the necessity to design feedback controllers which can adapt to such uncertainties. Controllers which are designed to satisfy this goal are called adaptive controllers. We can classify adaptive control into three main subclasses, for example, Reference : Classical model-based adaptive control, which mainly uses models of the controlled system; learning-based adaptive control, which uses both model-based and data-driven techniques to design modular adaptive controllers; and data-driven adaptive control, which is based on the interaction of the controller with the system. In this special issue, we shall focus on a specific class of data-driven adaptive controllers, namely, extremum seekers.

Recently, there have been a lot of efforts in this direction of control. One of the reasons behind this increasing interest is that, following the pioneering ESC analysis paper,² the field of ESC has reached a certain maturity, which has led to good analysis and understanding of the main properties of the available algorithms. The idea behind ESC is to design a closed-loop dynamical system, such that its flow converges to the optimum of a given static or dynamic cost function, for example, References 3-5. This type of optimization algorithms is also referred to as continuous optimization, for example, Reference 6. The advantages of such optimization approaches comparatively to classical optimization algorithms is the fact that extremum seekers do not require a closed-form expression of the cost function. Moreover, they do not require multiple evaluation of the cost function at each point to estimate the gradient (or higher order derivatives) of the cost function, since they include some internal states which converge to an estimate of the derivatives of the cost over time. Besides, their formulation as a closed-loop control problem allows us to use numerous tools from dynamical systems theory and control theory, for example, time-scale separation, averaging theory, Lyapunov stability theory, to perform rigorous analysis of their convergence and robustness properties w.r.t. initial conditions changes and other hyperparameters tuning.

In this special issue, we have included 10 papers, which could be divided into two main types. The first type, more theoretical, presents new extremum seeking methods designed for different dynamical systems, from systems with delays, and multiagent systems, to distributed systems. The second type of papers deals with real-world applications of extremum seeking controllers to challenging systems, ranging from microalgae cultivation control to heating and ventilation systems. These papers are compiled in a special virtual issue of the journal at the journal homepage. To access all of the papers, please follow the following link (https://onlinelibrary.wiley.com/toc/10991115/2021/35/7).

For instance, Reference 7 presents a novel ESC for batch-to-batch optimal control of unknown time-varying systems. The authors show that the proposed ESC matches the performance of an LQR optimal controller, for the case of known linear dynamics. The applications of this type of data-driven optimal controllers are numerous and range from repetitive robotics tasks to particle accelerators control. In Reference 8, a new gradient-based ESC is introduced for static maps in the presence of infinite-dimensional dynamics described by parabolic partial differential equations (PDEs), where the PDE dynamics contains reaction–advection–diffusion. Generalization to a scalar Newton-based ES algorithm is proposed, which allows to remove the dependence of the convergence rate on the unknown Hessian of the higher derivative of the cost function, which characterizes the standard gradient-based ESC. The problem of maximizing high-order derivatives of unknown maps with known time-varying delays is studied in Reference 9. Convergence rate of the real-time optimizer is shown to be user-assignable, and exponential stability to a small neighborhood of the unknown extremum point of the delayed map is achieved, for locally quadratic derivatives, by using backstepping transformation and averaging theory in infinite dimensions.

In Reference 10, the problem of ESC with finite-time convergence, in continuous-time, is studied for the case of static maps. The finite-time ESC is shown to achieve finite-time practical stability of the optimum of the static cost function. In Reference 11, a novel multivariable ESC is proposed, based on cyclic search direction and monitoring function for sliding-mode control with unknown control direction. The proposed ESC is shown to converge to an arbitrarily small neighborhood of the desired optimal point starting from any initial conditions, that is, global convergence. In Reference 12, the authors tackle the problem of ESC control for nonlinear dynamic systems using Lie bracket approximations, without relying on singular perturbation theory. Instead, the (practical) stability proofs are based on Chen–Fliess series. Finally, to end the theoretical part of the special issue, we find in Reference 13, novel ESC methods that allow to dispense with the time-varying dither excitation signal, needed by standard ESC that rely on a persistence of excitation condition. The authors propose to use the concept of cooperative concurrent learning, which relies on memory that enables the use of information-rich datasets during the optimization process. This concept is first applied to a single system and then extended to the multiagent setting.

On the application side, Reference 14 reports exciting results on the application of ESC to the problem of cultivation of microalgae in photobioreactors (PBRs). The authors explore the use of model-free ESC to optimally regulate the productivity of a continuous PBR. Experimental results for the cultivation of the microalgae Scenedesmus obliquus in a lab-scale flat-panel PBR are discussed.

Finally, the last two applied papers deal with the important problem of indoor heating ventilation and air conditioning systems' control. This problem is indeed important, since it relates to the efficient and optimal control of energy spent in buildings for indoor air quality and comfort regulation. The first paper,¹⁵ focuses on the case of computer room air conditioning systems, which are paramount to the optimal functioning of modern large data centers. In this application, an optimal operation is achieved when the required cooling demand is satisfied at the minimum energy cost. The authors propose to design a supervisory control system, where the higher layer determines the optimal set-points for the local controllers by employing a Newton-based ESC. The controller is validated on a high fidelity simulator, including an indirect adiabatic cooling system, and a computer room. In the second paper,¹⁶ the problem of robust adaptive observer design for thermal-fluid systems is studied, towards an application to efficient energy management in buildings. Indoors airflow/temperature dynamics are modeled using the Boussinesq PDE equations, and a robust reduced-order observer is designed to reconstruct the entire airflow velocity and temperature states under model parametric uncertainties. The observer is then adapted based on real-time measurements from the room, where an ESC is used to estimate the uncertain parameters of the model, and hence improve the overall performance of the observer.

中文翻译：

---

极值控制特刊社论：理论与应用

1 介绍

现实世界的系统必须在不确定和不断变化的环境中运行，因此有必要设计能够适应这种不确定性的反馈控制器。为满足这一目标而设计的控制器称为自适应控制器。我们可以将自适应控制分为三个主要的子类，例如，参考：基于经典模型的自适应控制，主要使用被控系统的模型；基于学习的自适应控制，它使用基于模型和数据驱动的技术来设计模块化自适应控制器；和数据驱动的自适应控制，它基于控制器与系统的交互。在本期特刊中，我们将关注一类特定的数据驱动自适应控制器，即极值搜索器。

最近，在这个控制方向上做了很多努力。这种兴趣增加的原因之一是，继开创性的 ESC 分析论文^{2 之后}，ESC 领域已经达到一定的成熟度，这导致对可用算法的主要属性进行了很好的分析和理解。ESC 背后的想法是设计一个闭环动态系统，使其流程收敛到给定静态或动态成本函数的最优值，例如，参考文献3-5。这类优化算法也称为连续优化，例如参考文献6. 与经典优化算法相比，这种优化方法的优势在于极值搜索器不需要成本函数的封闭形式表达式。此外，它们不需要在每个点对成本函数进行多次评估来估计成本函数的梯度（或高阶导数），因为它们包括一些内部状态，这些状态会随着时间的推移收敛到成本导数的估计。此外，它们作为闭环控制问题的表述使我们能够使用动态系统理论和控制理论中的众多工具，例如时标分离、平均理论、李雅普诺夫稳定性理论，对其收敛性和鲁棒性特性进行严格的分析wrt 初始条件变化和其他超参数调整。

在本期特刊中，我们收录了 10 篇论文，可分为两大类。第一种更具理论性，提出了为不同动态系统设计的新极值寻找方法，从具有延迟的系统、多代理系统到分布式系统。第二种类型的论文涉及极值搜索控制器在具有挑战性的系统中的实际应用，范围从微藻培养控制到加热和通风系统。这些论文汇编在期刊主页上的期刊虚拟特刊中。要访问所有论文，请点击以下链接 (https://onlinelibrary.wiley.com/toc/10991115/2021/35/7)。

例如，参考文献7提出了一种新颖的 ESC，用于未知时变系统的批次间优化控制。作者表明，对于已知线性动力学的情况，所提出的 ESC 与 LQR 最优控制器的性能相匹配。这种类型的数据驱动优化控制器的应用范围很广，从重复的机器人任务到粒子加速器控制。在参考文献8，在存在由抛物线偏微分方程 (PDE) 描述的无限维动力学的情况下，为静态地图引入了一种新的基于梯度的 ESC，其中 PDE 动力学包含反应-平流-扩散。提出了对基于标量牛顿的 ES 算法的推广，该算法允许消除收敛速度对成本函数的高阶导数的未知 Hessian 的依赖性，这是基于标准梯度的 ESC 的特征。在参考文献9 中研究了最大化具有已知时变延迟的未知地图的高阶导数的问题. 实时优化器的收敛率被证明是用户可分配的，并且对于局部二次导数，通过在无限维度上使用反步变换和平均理论，实现了延迟映射未知极值点的小邻域的指数稳定性.

在参考文献10 中，针对静态地图的情况研究了连续时间中具有有限时间收敛的 ESC 问题。有限时间 ESC 被证明可以实现静态成本函数最优值的有限时间实际稳定性。在参考文献11 中，提出了一种基于循环搜索方向和监控功能的新型多变量 ESC，用于控制方向未知的滑模控制。所提出的 ESC 被证明从任何初始条件开始收敛到所需最优点的任意小邻域，即全局收敛。在参考文献12，作者使用李括号近似解决非线性动态系统的 ESC 控制问题，而不依赖于奇异摄动理论。相反，（实际）稳定性证明基于 Chen-Fliess 系列。最后，为了结束特刊的理论部分，我们在参考文献13 中找到了新的 ESC 方法，该方法允许免除时变抖动激励信号，而标准 ESC 需要依赖激励条件的持久性。作者建议使用协作并发学习的概念，该概念依赖于在优化过程中使用信息丰富的数据集的内存。这个概念首先应用于单个系统，然后扩展到多代理设置。

在应用方面，参考文献14报告了将 ESC 应用于光生物反应器 (PBR) 中微藻培养问题的令人兴奋的结果。作者探索了使用无模型 ESC 来优化调节连续 PBR 的生产力。讨论了在实验室规模的平板 PBR 中培养斜栅藻微藻的实验结果。

最后，最后两篇应用论文讨论了室内采暖通风和空调系统控制的重要问题。这个问题确实很重要，因为它涉及有效和最佳地控制建筑物中用于室内空气质量和舒适度调节的能源。第一篇论文，¹⁵专注于机房空调系统的案例，这对于现代大型数据中心的最佳运行至关重要。在此应用中，当所需的冷却需求以最低的能源成本得到满足时，即可实现最佳运行。作者建议设计一个监督控制系统，其中高层通过采用基于牛顿的 ESC 来确定本地控制器的最佳设定点。该控制器在高保真模拟器上进行了验证，包括间接绝热冷却系统和计算机房。在第二篇论文中，¹⁶研究了热流体系统的鲁棒自适应观测器设计问题，以应用于建筑物的高效能源管理。使用 Boussinesq PDE 方程对室内气流/温度动态进行建模，并设计了一个稳健的降阶观测器来重建模型参数不确定性下的整个气流速度和温度状态。然后根据房间的实时测量调整观察器，其中使用 ESC 来估计模型的不确定参数，从而提高观察器的整体性能。