This study proposes an approximate parametric model-based Bayesian reinforcement learning approach for robots, based on online Bayesian estimation and online planning for an estimated model. The proposed approach is designed to learn a robotic task with a few real-world samples and to be robust against model uncertainty, within feasible computational resources. The proposed approach employs two-stage modeling, which is composed of (1) a parametric differential equation model with a few parameters based on prior knowledge such as equations of motion, and (2) a parametric model that interpolates a finite number of transition probability models for online estimation and planning. The proposed approach modifies the online Bayesian estimation to be robust against approximation errors of the parametric model to a real plant. The policy planned for the interpolating model is proven to have a form of theoretical robustness. Numerical simulation and hardware experiments of a planar peg-in-hole task demonstrate the effectiveness of the proposed approach.

中文翻译：

---

基于植物估计的近似贝叶斯强化学习

这项研究基于在线贝叶斯估计和估计模型的在线计划，提出了一种基于近似参数模型的机器人贝叶斯强化学习方法。提出的方法旨在学习具有一些实际示例的机器人任务，并且在可行的计算资源范围内对模型不确定性具有鲁棒性。所提出的方法采用两阶段建模，该模型由（1）基于先验知识（例如运动方程）的带有一些参数的参数微分方程模型和（2）内插有限数量的跃迁概率的参数模型组成在线估算和计划模型。所提出的方法修改了在线贝叶斯估计，以抵抗参数模型对实际工厂的逼近误差。事实证明，为插值模型计划的策略具有某种形式的理论鲁棒性。平面钉入孔任务的数值模拟和硬件实验证明了该方法的有效性。