Microinstrument contact force sensing based on cable tension using BLSTM–MLP network

Abstract

Minimally invasive surgical robotic systems established the foundation for precise and refined surgery, and the intelligentization of robotic systems is an important direction for future development. Among the methods of intelligentization, microinstrument external force sensing is an open and challenging research area. Force sensing information is used not only to ensure that surgeons apply the appropriate amount of force but also to prevent unintentional tissue damage. Because a microinstrument is a compact and small-sized construction, indirect force sensing method instead of the integration of sensors into the microinstrument is used, yielding better biocompatibility, sterilizability and monetary cost savings. This paper focuses on microinstrument-tissue contact force sensing, and the microinstrument used is a three degrees of freedom cable-driven manipulator. A contact force estimation strategy based on the differences in cable tension is established with consideration of the kinematics, dynamics and friction of the manipulator. A principle prototype of a surgical microinstrument force measurement system is developed, and then zero-drift, hysteresis and force loading experiments are studied. Based on the experimental data of the force loading experiments, the relationship between cable tension and contact forces is established by using a bidirectional long short-term memory plus multilayer perceptron network. The results show that the L2 cost of the network in the training set converges to 0.006 and that the RMSE of the network in the testing set converges to 0.053, and the network can meet the measurement requirements without overfitting. Therefore, the indirect force estimation method is a viable method of measuring forces of cable-driven microinstrument and can be used to integrate force sensing information into surgical robotic systems to improve the operability of surgical robots.

Appendix: Dynamic model detailed results

$$ \begin{aligned} & \boldsymbol{\tau } = \boldsymbol{H}\left( \boldsymbol{q} \right)\boldsymbol{\ddot{q}} + \boldsymbol{C}\left( {\boldsymbol{q},\dot{\boldsymbol{q}}} \right)\dot{\boldsymbol{q}} + \boldsymbol{G}\left( \boldsymbol{q} \right) + \boldsymbol{M}_{f} + \boldsymbol{J}^{\text{T}} \boldsymbol{F} \\ & \left[ {\begin{array}{*{20}c} {\tau_{1m} } \\ {\tau_{2m} } \\ \end{array} } \right] = \left[ {\begin{array}{*{20}c} {{}^{m}h_{11} } & {{}^{m}h_{12} } \\ {{}^{m}h_{21} } & {{}^{m}h_{22} } \\ \end{array} } \right]\left[ {\begin{array}{*{20}c} {\ddot{q}_{1} } \\ {\ddot{q}_{2m} } \\ \end{array} } \right] + \left[ {\begin{array}{*{20}c} {{}^{m}c_{11} } & {{}^{m}c_{12} } \\ {{}^{m}c_{21} } & {{}^{m}c_{22} } \\ \end{array} } \right]\left[ {\begin{array}{*{20}c} {\dot{q}_{1} } \\ {\dot{q}_{2m} } \\ \end{array} } \right] + \left[ {\begin{array}{*{20}c} {g_{1m} } \\ {g_{2m} } \\ \end{array} } \right] + \left[ {\begin{array}{*{20}c} {M_{f1} } \\ {M_{f2m} } \\ \end{array} } \right] + {}^{m}\boldsymbol{J}^{\text{T}} \left[ {\begin{array}{*{20}c} {F_{mx} } \\ {F_{my} } \\ {F_{mz} } \\ \end{array} } \right] \\ \end{aligned} $$

where m = a, b

$$ \begin{aligned} \boldsymbol{J}_{i} & = \left[ {\begin{array}{*{20}c} {\frac{{ - {}^{i - 1}I_{x} + {}^{i - 1}I_{y} + {}^{i - 1}I_{z} }}{2}} & {{}^{i - 1}I_{xy} } & {{}^{i - 1}I_{xz} } & {m_{i} {}^{i - 1}x_{ci} } \\ {{}^{i - 1}I_{xy} } & {\frac{{{}^{i - 1}I_{x} - {}^{i - 1}I_{y} + {}^{i - 1}I_{z} }}{2}} & {{}^{i - 1}I_{yz} } & {m_{i} {}^{i - 1}y_{ci} } \\ {{}^{i - 1}I_{xz} } & {{}^{i - 1}I_{yz} } & {\frac{{{}^{i - 1}I_{x} + {}^{i - 1}I_{y} - {}^{i - 1}I_{z} }}{2}} & {m_{i} {}^{i - 1}z_{ci} } \\ {m_{i} {}^{i - 1}x_{ci} } & {m_{i} {}^{i - 1}y_{ci} } & {m_{i} {}^{i - 1}z_{ci} } & {m_{i} } \\ \end{array} } \right] \\ {}^{m}h_{11} & = {}^{0}I_{y} + {}^{1m}I_{x} + 2a_{1} m_{1} {}^{0}x_{c1} + a_{1}^{2} m_{1} + c_{2m}^{2} \left( {{}^{1m}I_{y} - {}^{1m}I_{x} } \right) - 2s_{2m} c_{2m} {}^{1m}I_{xy} + 2c_{2m} \left( {a_{2} c_{2m} + a_{1} } \right)m_{2m} {}^{1m}x_{c2m} \\ & \quad - 2s_{2m} \left( {a_{2} c_{2m} + a_{1} } \right)m_{2m} {}^{1m}y_{c2m} + \left( {a_{2}^{2} c_{2m}^{2} + 2a_{1} a_{2} c_{2m} + a_{1}^{2} } \right)m_{2m} \\ {}^{m}h_{12} & = {}^{m}h_{21} = - c_{2m} {}^{1m}I_{yz} - s_{2m} {}^{1m}I_{xz} - a_{2} s_{2m} m_{2m} {}^{1m}z_{c2m} \\ {}^{m}h_{22} & = a_{2}^{2} m_{2m} + 2a_{2} m_{2m} {}^{1m}x_{c2m} + {}^{1m}I_{z} \\ {}^{m}c_{11} & = - \left[ {\left( {c_{2m}^{2} - s_{2m}^{2} } \right){}^{1m}I_{xy} + \left( {2s_{2m} c_{2m} a_{2} + s_{2m} a_{1} } \right)m_{2m} {}^{1m}x_{c2m} + \left( {\left( {c_{2m}^{2} - s_{2m}^{2} } \right)a_{2} + c_{2m} a_{1} } \right)m_{2m} {}^{1m}y_{c2m} } \right. \\ & \quad \left. { + \left( {s_{2m} c_{2m} a_{2}^{2} + s_{2m} a_{1} a_{2} } \right)m_{2m} + s_{2m} c_{2m} \left( {{}^{1m}I_{y} - {}^{1m}I_{x} } \right)} \right]\dot{q}_{2m} \\ {}^{m}c_{12} & = - \left[ {\left( {c_{2m}^{2} - s_{2m}^{2} } \right){}^{1m}I_{xy} + \left( {2s_{2m} c_{2m} a_{2} + s_{2m} a_{1} } \right)m_{2m} {}^{1m}x_{c2m} + \left( {\left( {c_{2m}^{2} - s_{2m}^{2} } \right)a_{2} + c_{2m} a_{1} } \right)m_{2m} {}^{1m}y_{c2m} } \right. \\ & \quad \left. { + \left( {s_{2m} c_{2m} a_{2}^{2} + s_{2m} a_{1} a_{2} } \right)m_{2m} + s_{2m} c_{2m} \left( {{}^{1m}I_{y} - {}^{1m}I_{x} } \right)} \right]\dot{q}_{1} + \left[ {s_{2m} {}^{1m}I_{yz} - c_{2m} {}^{1m}I_{xz} - a_{2} c_{2m} m_{2m} {}^{1m}z_{c2m} } \right]\dot{q}_{2m} \\ {}^{m}c_{21} & = \left[ {\left( {c_{2m}^{2} - s_{2m}^{2} } \right){}^{1m}I_{xy} + \left( {2s_{2m} c_{2m} a_{2} + s_{2m} a_{1} } \right)m_{2m} {}^{1m}x_{c2m} + \left( {\left( {c_{2m}^{2} - s_{2m}^{2} } \right)a_{2} + c_{2m} a_{1} } \right)m_{2m} {}^{1m}y_{c2m} } \right. \\ & \quad \left. { + \left( {s_{2m} c_{2m} a_{2}^{2} + s_{2m} a_{1} a_{2} } \right)m_{2m} + s_{2m} c_{2m} \left( {{}^{1m}I_{y} - {}^{1m}I_{x} } \right)} \right]\dot{q}_{1} \\ {}^{m}c_{22} & = 0 \\ g_{1m} & \quad = - \left( {c_{1} a_{1} + c_{1} {}^{0}x_{c1} + s_{1} {}^{0}z_{c1} } \right)m_{1} g - \left( {c_{1} a_{1} + c_{1} c_{2m} a_{2} + c_{1} c_{2m} {}^{1m}x_{c2m} - c_{1} s_{2m} {}^{1m}y_{c2m} + s_{1} {}^{1m}z_{c2m} } \right)m_{2m} g \\ g_{2m} & \quad = \left( {s_{1} s_{2m} a_{2} + s_{1} s_{2m} {}^{1m}x_{c2m} + s_{1} c_{2m} {}^{1m}y_{c2m} } \right)m_{2m} g \\ {}^{m}\boldsymbol{J}^{\text{T}} & \quad = \left[ {\begin{array}{*{20}l} { - s_{1} a_{1} - s_{1} c_{2m} a_{2} } \hfill & {c_{1} a_{1} + c_{1} c_{2m} a_{2} } \hfill & 0 \hfill \\ { - c_{1} s_{2m} a_{2} } \hfill & { - s_{1} s_{2m} a_{2} } \hfill & {c_{2m} a_{2} } \hfill \\ \end{array} } \right] \\ \end{aligned} $$