Soft magnetic miniature robots (SMMRs) have potential biomedical applications due to their flexible size and mobility to access confined environments. However, navigating the robot to a goal site with precise control performance and high repeatability in unstructured environments, especially in flow rate conditions, still remains a challenge. In this study, drawing inspiration from the control requirements of drug delivery and release to the goal lesion site in the presence of dynamic biofluids, we propose a flow rate rejection control strategy based on a deep reinforcement learning (DRL) framework to actuate an SMMR to achieve goal-reaching and hovering in fluidic tubes. To this end, an SMMR is first fabricated, which can be operated by an external magnetic field to realize its desired functionalities. Subsequently, a simulator is constructed based on neural networks to map the relationship between the applied magnetic field and robot locomotion states. With minimal prior knowledge about the environment and dynamics, a gated recurrent unit (GRU)-based DRL algorithm is formulated by considering the designed history state–action and estimated flow rates. In addition, the randomization technique is applied during training to distill the general control policy for the physical SMMR. The results of numerical simulations and experiments are illustrated to demonstrate the robustness and efficacy of the presented control framework. Finally, in-depth analyses and discussions indicate the potentiality of DRL for soft magnetic robots in biomedical applications.

中文翻译：

---

基于深度强化学习框架的软磁微型机器人流量抑制控制


软磁微型机器人（SMMR）因其灵活的尺寸和进入受限环境的移动性而具有潜在的生物医学应用。然而，在非结构化环境中，特别是在流速条件下，以精确的控制性能和高重复性将机器人导航到目标地点仍然是一个挑战。在本研究中，从动态生物流体存在下药物输送和释放到目标病变部位的控制要求中汲取灵感，我们提出了一种基于深度强化学习（DRL）框架的流速拒绝控制策略，以驱动 SMMR实现目标到达并在流体管中悬停。为此，首先制造了SMMR，它可以通过外部磁场操作来实现其所需的功能。随后，基于神经网络构建模拟器来映射所施加的磁场和机器人运动状态之间的关系。利用有关环境和动力学的最少先验知识，通过考虑设计的历史状态动作和估计流量，制定基于门控循环单元（GRU）的 DRL 算法。此外，在训练过程中应用随机化技术来提取物理 SMMR 的通用控制策略。数值模拟和实验的结果证明了所提出的控制框架的鲁棒性和有效性。最后，深入的分析和讨论表明了 DRL 在生物医学应用中软磁机器人的潜力。