We consider a class of finite-time horizon nonlinear stochastic optimal control problem. Although the optimal control admits a path integral representation for this class of control problems, efficient computation of the associated path integrals remains a challenging task. We propose a new Monte Carlo approach that significantly improves upon existing methodology. We tackle the issue of exponential growth in variance with the time horizon by casting optimal control estimation as a smoothing problem for a state-space model, and applying smoothing algorithms based on particle Markov chain Monte Carlo. To further reduce the cost, we then develop a multilevel Monte Carlo method which allows us to obtain an estimator of the optimal control with $\mathcal{O}\left({ϵ}^2\right)$ mean squared error with a cost of $\mathcal{O}({ϵ}^{-2}\log (ϵ{)}^2)$. In contrast, a cost of $\mathcal{O}\left({ϵ}^{-3}\right)$ is required for the existing methodology to achieve the same mean squared error. Our approach is illustrated on two numerical examples.

我们考虑一类有限时间范围的非线性随机最优控制问题。尽管最优控制允许此类控制问题的路径积分表示，但相关路径积分的有效计算仍然是一项具有挑战性的任务。我们提出了一种新的蒙特卡罗方法，该方法显着改进了现有方法。我们通过将最优控制估计转换为状态空间模型的平滑问题，并应用基于粒子马尔可夫链蒙特卡罗的平滑算法来解决随时间范围变化的指数增长问题。为了进一步降低成本，我们开发了一种多级蒙特卡罗方法，该方法使我们能够获得最优控制的估计量$\mathcal{○}\left({\epsilon }^2\right)$均方误差，成本为$\mathcal{○}({\epsilon }^{-2}\mathrm{日志}(\epsilon {)}^2)$. 相比之下，成本为$\mathcal{○}\left({\epsilon }^{-3}\right)$现有方法需要达到相同的均方误差。我们的方法通过两个数值示例进行了说明。