We consider the problem of optimal feedback control of an unknown, noisy, time-varying, dynamic system that is initialized repeatedly. Examples include a robotic manipulator which must perform the same motion, such as assisting a human, repeatedly and accelerating cavities in particle accelerators which are turned on for a fraction of a second with given initial conditions and vary slowly due to temperature fluctuations. We present an approach that applies to systems of practical interest. The method presented here is model independent; does not require knowledge of the objective function; is robust to measurement noise; is applicable for any set of initial conditions; is applicable to simultaneously controlling an arbitrary number of parameters; and may be implemented with a broad range of continuous or discontinuous functions such as sine or square waves. For systems with convex cost functions we prove that our algorithm will produce controllers that approach the minimal cost. For linear systems we reproduce the cost minimizing linear quadratic regulator optimal controller that could have been designed analytically had the system and cost function been known. We demonstrate the effectiveness of the algorithm with simulation studies of noisy and time-varying systems.

中文翻译：

---

具有未知时变系统和未知目标函数的最优控制问题的极值寻求

我们考虑反复初始化的未知、嘈杂、时变、动态系统的最优反馈控制问题。示例包括机器人操纵器，它必须执行相同的运动，例如辅助人类，重复和加速粒子加速器中的腔，这些腔在给定的初始条件下打开几分之一秒，并且由于温度波动而缓慢变化。我们提出了一种适用于具有实际意义的系统的方法。这里介绍的方法与模型无关；不需要了解目标函数；对测量噪声具有鲁棒性；适用于任何一组初始条件；适用于同时控制任意数量的参数；并且可以使用范围广泛的连续或不连续函数（例如正弦波或方波）来实现。对于具有凸成本函数的系统，我们证明我们的算法将产生接近最小成本的控制器。对于线性系统，我们重现了成本最小化线性二次调节器最佳控制器，如果系统和成本函数已知，则可以分析设计该控制器。我们通过对噪声和时变系统的仿真研究证明了该算法的有效性。