Elsevier

Neurocomputing

Volume 402, 18 August 2020, Pages 384-394
Neurocomputing

Adaptive visual servoing with an uncalibrated camera using extreme learning machine and Q-leaning

https://doi.org/10.1016/j.neucom.2020.03.049Get rights and content

Abstract

In this paper, a novel image-based visual servoing (IBVS) method using Extreme Learning Machine (ELM) and Q-learning is proposed to solve the problems of complex modeling and selection of the servo gain. First, the pseudoinverse of the interaction matrix is approached by ELM which avoids the singularity of the interaction matrix effectively and is robust to interferences such as feature noises and camera calibration errors. Second, a reinforcement learning method, Q-learning, is adopted to adaptively adjust the servo gain in order to improve the convergence speed and stability. Compared with other methods, ELM has better generalization performance, faster operation speed and a unique optimal solution. Also, Q-learning has self-learning ability without experience in advance. The effectiveness of the proposed method is validated by simulations and experiment on a 6-DOF robot with eye-in-hand configuration.

Introduction

Visual servoing is a servo control method to control the robot or camera pose to the desired position quickly using visual information. There are various applications in visual servoing, such as manipulators [1], [2], mobile robots [3], [4], [5], [6], unmanned aerial vehicles [7], [8], underwater vehicles [9], [10], etc. In industrial applications, visual servoing control system is a very important subsystem for semi-automated and fully automated equipment. For example, it can replace the manpower and carry out qualified inspection of products, such as checking the shape, size and thickness of the product, and greatly improving the detection accuracy. Also, it is used to guide the manipulator to the target position for further assembly and welding operations. This method is not only safe but also greatly improves industrial production efficiency. In the aerospace industry, intelligent robots can be used to explore complex and unknown outer space and complete a series of complex outer space missions. Moreover, the introduction of vision brings great convenience [11], [12]. Visual servoing methods can be mainly classified into position-based visual servoing (PBVS) [13], [14], image-based visual servoing (IBVS) [15], [16] and hybrid visual servoing [17]. Among them, PBVS constitutes a closed-loop control system in 3D cartesian space [18]. IBVS forms a closed-loop system in the 2D image space. Hybrid visual servoing method contains both 3D space and 2D space, so it is called 2.5D visual servoing [19]. Compared with the other two methods, IBVS has been increasingly researched in recent years because of its high steady-state control accuracy [20].

In IBVS approaches, the estimation of image interaction matrix and the choice of gain are two important factors that determine the performance of system. Among them, image interaction matrix which represents the mapping relationship from 3D space to 2D image plane plays an important role in the IBVS system. The parameters of the image interaction matrix are usually obtained by camera calibration and depth estimation methods in the traditional method [21]. However, camera calibration errors, depth estimation errors and feature noise all have an effect on the image interaction matrix. Moreover, solving the pseudoinverse of image interaction matrix is very complicated, and the singularity of the pseudoinverse may be generated which can cause the visual servoing task failure. In order to solve these problems, a fuzzy modeling scheme is used to obtain an inverse of the mapping between image feature variations and joint velocities in [22]. Similarly, an approach for approximating the manipulator dynamic based adaptive fuzzy method is proposed in [23]. But the generalization ability of both fuzzy system is limited [24]. present a method based robust kalman filtering and elman neural network learning. The global mapping relationship between the vision space and the robotic workspace is learned using an elman neural network. However, the ability of anti-interference of it is limited. An interesting solution in [25] design a switching control between neural reinforcement learning controller and traditional IBVS controller without the pseudoinverse of the image interaction matrix. But the velocity signals show huge chatting. Moreover, a recurrent neural controller is brought up to approximate the interaction matrix in [26], but recurrent neural network requires repeated iterations, so that the convergence speed is reduced. In addition, this method is for one point control in this paper. Similarly, a method using adaptive neural network is described in [27]. In order to approximate the pseudoinverse of the interaction matrix, a solution with ELM and fuzzy servo gain is developed in [28]. Although ELM has a good approximation ability, the gain with fuzzy logic is lack of self-learning mechanism. And fuzzy control requires prolific rule experience in advance and the rules determine the control effect directly.

However, most of the above methods do not take into account the effects of servo gain. A fixed gain may lead to system instability and slow convergence. Adaptive gain can increase the convergence speed of the system compared with the traditional fixed gain. The work in [29] put forward an approach that using a monitor to determine the value of gain. But it is only a rough adjustment of the gain, the adaptive ability of gain is weak. In [30], [31], a scheme is introduced that use a set of PID controllers to replace a static gain matrix. However, the PID parameters may be sensitive to the environment. Adaptive state-feedback based method is used to control gain in nonlinear second-order system described in [32]. [33] uses an observer with finite-time to estimate the error, and it is easy to choose a gain since it only depends on the uncertainty of estimation error. However, the computation and the modeling process of these methods is complex. Compared with the above method, a reinforcement learning method, Q-learning, has been widely studied by scholars because of its self-learning ability that does not require experience in advance. Such as [34], the gain adjusted by Q-learning has a strong adaptive ability to the environment. But this method only considers three degree of freedom model.

In this paper, a novel visual servoing with ELM and Q-learning is proposed. On one hand, ELM is used to approximate L^s+e which avoids the singularity of the interaction matrix effectively and is robust to interference. Compared with other classification methods [35], ELM inherits the structural advantages of the single hidden layer feedforward neural network, which has a high learning efficiency that reduces the training time greatly. And the obtained solution is a unique optimal solution, which ensures the generalization ability of the network. On the other hand, Q-learning is used to determine the gain. Compared with other methods, Q-learning does not require any knowledge about the environment and is suitable for decision making. Q-learning is used to select the optimal gain in convergent Q table at each state, so as to achieve the effect of adaptively adjusting the gain to improve the convergence speed.

The structure of this paper is as follows. Section 2 introduces the modeling process of classic visual servoing. In Section 3, the basic principle of ELM is introduced, and ELM is applied to the visual servoing model at the same time. In Section 4, a method of adaptive gain using Q-learning is proposed. Experimental and simulative results are given to verify the effectiveness of the proposed method in Section 5. At last, the conclusion is presented in Section 6.

Section snippets

Image based visual servo control

This section provides a short review of the traditional IBVS system. It presents a visual servoing control of a robot manipulator with eye-in-hand configuration. The central-projection model of a pinhole camera is shown in Fig. 1. Assuming n fixed 3D points with coordinates Pi=(Xi,Yi,Zi)TR3×1, i=1, , n in the camera frame are projected to the image plane, and the 2D coordinates si=(ui,vi)TR2×1, i=1, , n in pixel units issi=[uivi]=[fρu00fρv][XiZiYiZi]+[u0v0]where f is the focal length, (u0, v

Approximation of jacobian matrix by ELM

According to (6), the visual servoing controller is mainly determined by two aspects: pseudoinverse and gain. It is noted that the depth of interaction matrix needs to be estimated. Although the analytical interaction matrix can be obtained after a good depth estimation, it is still affected by camera calibration error and feature noise. It makes sense to find a good way to approximate L^s+e. ELM algorithm is developed on the basis of the Single-Hidden Layer Feedforward Networks (SLFNs) and can

Adaptive serving gain with Q-Learning

The servo gain λ determines the convergence time and the stability of system. An adaptive gain can make the system performance more stable and the convergence time shorter. Because Q-learning dose not require any rulebase about environment in advance and adaptive with the environment, it is suitable to design the gain for decision making. In this study, the camera velocity is Vc=[vx,vy,vz,ωx,ωy,ωz]TR6×1 and the gain matrix of camera velocity is λ=diag{λvx,λvy,λvz,λωx,λωy,λωz}R6×6. Six

System Description and simulation results

In this section, the performance of the proposed method is confirmed by simulating in MATLAB. Four 3D point projections to the camera plane as the image feature. Because the static target mode is considered, the desired feature points sd are unchanged. The task is completed when the feature errors is less than ε pixels. The initial position and desire position aresiniti=[225.99225.99269.99269.9932.0076.0076.0032.00],sd=[138.00138.00182.00182.0098.00142.00142.0098.00]

The data set of ELMs is

Conclusion

In this paper, a novel IBVS system with ELM and Q-learning has been proposed. This method approximates Ls+e by ELM that avoids the singularity of the pseudoinverse of the interaction matrix and is robust to camera calibration errors and feature noise. The servo gain is critical to the performance of the system. Q-learning can learn the gain by multiple trainings, and the Q table eventually converges to the optimal steps of control cycles. The adaptive gain control can be realized by using the

Compliance with Ethical Standards

Conflict of Interest: Meng Kang and Hao Chen are students in Northeastern University. Jiuxiang Dong is a professor in Northeastern University. The authors declare that they have no conflict of interest.

CRediT authorship contribution statement

Meng Kang: Conceptualization, Methodology, Software, Validation, Writing - review & editing. Hao Chen: Software, Data curation, Validation. Jiuxiang Dong: Conceptualization, Methodology, Resources, Supervision.

Declaration of Competing Interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Acknowledgment

This work was supported in part by the National Natural Science Foundation of China under Grant 61873056, Grant 61621004 and Grant 61420106016, the Fundamental Research Funds for the Central Universities in China under Grant N2004001, N2004002, N182608004 and the Research Fund of State Key Laboratory of Synthetical Automation for Process Industries in China under Grant 2013ZCX01.

Meng Kang received the B.S.degrees in Automation from Yanshan university in 2017. She is now pursuing the M.S. degree in College of Information Science and Engineering, Northeastern University, China. Her research interests focus on visual servoing and robot control.

References (36)

  • K. Zhang et al.

    Unified visual servoing tracking and regulation of wheeled mobile robots with an uncalibrated camera

    IEEE/ASME Trans. Mechatron.

    (2018)
  • J. Thomas et al.

    Visual servoing of quadrotors for perching by hanging from cylindrical objects

    IEEE Robot. Autom. Lett.

    (2016)
  • S. Islam et al.

    Robust control of four-rotor unmanned aerial vehicle with disturbance uncertainty

    IEEE Trans. Ind. Electron.

    (2015)
  • J. Gao et al.

    Hierarchical model predictive image-based visual servoing of underwater vehicles with adaptive neural network dynamic control

    IEEE Trans. Cybern.

    (2016)
  • N.R. Gracias et al.

    Mosaic-based navigation for autonomous underwater vehicles

    IEEE J. Oceanic Eng.

    (2003)
  • W.J. Wilson et al.

    Relative end-effector control using cartesian position based visual servoing

    IEEE Trans. Robot. Autom.

    (1996)
  • D. Park et al.

    Novel position-based visual servoing approach to robust global stability under field-of-view constraint

    IEEE Trans. Ind. Electron.

    (2012)
  • K. Hashimoto et al.

    Manipulator Control with Image-based Visual Servo

    Proceedings 1991 IEEE International Conference on Robotics & Automation

    (1991)
  • Cited by (28)

    • Deep learning-based pose prediction for visual servoing of robotic manipulators using image similarity

      2022, Neurocomputing
      Citation Excerpt :

      In [33], Zuo et al. presented a method that predicts the joint points for calculating the pose of a manipulator. Recently, Kang et al. used reinforcement learning to realize visual servoing with an uncalibrated camera in [34]. Inspired by the correlation between the current camera pose and the image observed by a camera, namely, the observation that the more similar the observed images are, the closer the poses of the camera, it is noted that the error between the images is more prominent than the error between the pose labels in the training process.

    • Comparison of visual servoing technologies for robotized aerospace structural assembly and inspection

      2022, Robotics and Computer-Integrated Manufacturing
      Citation Excerpt :

      The system was effective for assembling flexible metal parts with a 0.5 mm tolerance. Besides of using laser for position control in automation, in the last decade there has been a great effort for defining robot trajectories based upon VS [17,18]. Kang et al. [17] used a Q-learning and extreme learning machine (ELM) in addition to visual servoing image-based as a solution to complex modeling problems and servo gains selection.

    • A survey Of learning-Based control of robotic visual servoing systems

      2022, Journal of the Franklin Institute
      Citation Excerpt :

      Shi et al. [79,80] have successfully applied Q-learning to control image-based multi-agent systems and quadrotor systems to adaptively adjust image Jacobian matrix. Q-learning is combined with extreme learning machine to solve the complex modeling and selection problem of servo gain [81]. For RL, it is inevitable to use some estimation methods to estimate the model or parameters in value functions and policies [82].

    View all citing articles on Scopus

    Meng Kang received the B.S.degrees in Automation from Yanshan university in 2017. She is now pursuing the M.S. degree in College of Information Science and Engineering, Northeastern University, China. Her research interests focus on visual servoing and robot control.

    Hao Chen received the B.S.degrees in Electrical Engineering and Automation from Southwest University of Science and Technology in 2016. He received the M.S. degree from College of Information Science and Engineering, Northeastern University, China. His research interests focus on visual servoing and robot control.

    Jiuxiang Dong received the B.S. degree in mathematics and applied mathematics, the M.S. degree in applied mathematics from Liaoning Normal University, China, in 2001 and 2004, respectively. He received the Ph.D. degree in navigation guidance and control from Northeastern University, China in 2009. He is currently a Professor at the College of Information Science and Engineering, Northeastern University. His research interests include fuzzy control, robust control and reliable control. Dr. Dong is an Associate Editor for the International Journal of Control, Automation, and Systems (IJCAS).

    View full text