Adaptive visual servoing with an uncalibrated camera using extreme learning machine and Q-leaning
Introduction
Visual servoing is a servo control method to control the robot or camera pose to the desired position quickly using visual information. There are various applications in visual servoing, such as manipulators [1], [2], mobile robots [3], [4], [5], [6], unmanned aerial vehicles [7], [8], underwater vehicles [9], [10], etc. In industrial applications, visual servoing control system is a very important subsystem for semi-automated and fully automated equipment. For example, it can replace the manpower and carry out qualified inspection of products, such as checking the shape, size and thickness of the product, and greatly improving the detection accuracy. Also, it is used to guide the manipulator to the target position for further assembly and welding operations. This method is not only safe but also greatly improves industrial production efficiency. In the aerospace industry, intelligent robots can be used to explore complex and unknown outer space and complete a series of complex outer space missions. Moreover, the introduction of vision brings great convenience [11], [12]. Visual servoing methods can be mainly classified into position-based visual servoing (PBVS) [13], [14], image-based visual servoing (IBVS) [15], [16] and hybrid visual servoing [17]. Among them, PBVS constitutes a closed-loop control system in 3D cartesian space [18]. IBVS forms a closed-loop system in the 2D image space. Hybrid visual servoing method contains both 3D space and 2D space, so it is called 2.5D visual servoing [19]. Compared with the other two methods, IBVS has been increasingly researched in recent years because of its high steady-state control accuracy [20].
In IBVS approaches, the estimation of image interaction matrix and the choice of gain are two important factors that determine the performance of system. Among them, image interaction matrix which represents the mapping relationship from 3D space to 2D image plane plays an important role in the IBVS system. The parameters of the image interaction matrix are usually obtained by camera calibration and depth estimation methods in the traditional method [21]. However, camera calibration errors, depth estimation errors and feature noise all have an effect on the image interaction matrix. Moreover, solving the pseudoinverse of image interaction matrix is very complicated, and the singularity of the pseudoinverse may be generated which can cause the visual servoing task failure. In order to solve these problems, a fuzzy modeling scheme is used to obtain an inverse of the mapping between image feature variations and joint velocities in [22]. Similarly, an approach for approximating the manipulator dynamic based adaptive fuzzy method is proposed in [23]. But the generalization ability of both fuzzy system is limited [24]. present a method based robust kalman filtering and elman neural network learning. The global mapping relationship between the vision space and the robotic workspace is learned using an elman neural network. However, the ability of anti-interference of it is limited. An interesting solution in [25] design a switching control between neural reinforcement learning controller and traditional IBVS controller without the pseudoinverse of the image interaction matrix. But the velocity signals show huge chatting. Moreover, a recurrent neural controller is brought up to approximate the interaction matrix in [26], but recurrent neural network requires repeated iterations, so that the convergence speed is reduced. In addition, this method is for one point control in this paper. Similarly, a method using adaptive neural network is described in [27]. In order to approximate the pseudoinverse of the interaction matrix, a solution with ELM and fuzzy servo gain is developed in [28]. Although ELM has a good approximation ability, the gain with fuzzy logic is lack of self-learning mechanism. And fuzzy control requires prolific rule experience in advance and the rules determine the control effect directly.
However, most of the above methods do not take into account the effects of servo gain. A fixed gain may lead to system instability and slow convergence. Adaptive gain can increase the convergence speed of the system compared with the traditional fixed gain. The work in [29] put forward an approach that using a monitor to determine the value of gain. But it is only a rough adjustment of the gain, the adaptive ability of gain is weak. In [30], [31], a scheme is introduced that use a set of PID controllers to replace a static gain matrix. However, the PID parameters may be sensitive to the environment. Adaptive state-feedback based method is used to control gain in nonlinear second-order system described in [32]. [33] uses an observer with finite-time to estimate the error, and it is easy to choose a gain since it only depends on the uncertainty of estimation error. However, the computation and the modeling process of these methods is complex. Compared with the above method, a reinforcement learning method, Q-learning, has been widely studied by scholars because of its self-learning ability that does not require experience in advance. Such as [34], the gain adjusted by Q-learning has a strong adaptive ability to the environment. But this method only considers three degree of freedom model.
In this paper, a novel visual servoing with ELM and Q-learning is proposed. On one hand, ELM is used to approximate which avoids the singularity of the interaction matrix effectively and is robust to interference. Compared with other classification methods [35], ELM inherits the structural advantages of the single hidden layer feedforward neural network, which has a high learning efficiency that reduces the training time greatly. And the obtained solution is a unique optimal solution, which ensures the generalization ability of the network. On the other hand, Q-learning is used to determine the gain. Compared with other methods, Q-learning does not require any knowledge about the environment and is suitable for decision making. Q-learning is used to select the optimal gain in convergent Q table at each state, so as to achieve the effect of adaptively adjusting the gain to improve the convergence speed.
The structure of this paper is as follows. Section 2 introduces the modeling process of classic visual servoing. In Section 3, the basic principle of ELM is introduced, and ELM is applied to the visual servoing model at the same time. In Section 4, a method of adaptive gain using Q-learning is proposed. Experimental and simulative results are given to verify the effectiveness of the proposed method in Section 5. At last, the conclusion is presented in Section 6.
Section snippets
Image based visual servo control
This section provides a short review of the traditional IBVS system. It presents a visual servoing control of a robot manipulator with eye-in-hand configuration. The central-projection model of a pinhole camera is shown in Fig. 1. Assuming n fixed 3D points with coordinates n in the camera frame are projected to the image plane, and the 2D coordinates n in pixel units iswhere f is the focal length, (u0, v
Approximation of jacobian matrix by ELM
According to (6), the visual servoing controller is mainly determined by two aspects: pseudoinverse and gain. It is noted that the depth of interaction matrix needs to be estimated. Although the analytical interaction matrix can be obtained after a good depth estimation, it is still affected by camera calibration error and feature noise. It makes sense to find a good way to approximate . ELM algorithm is developed on the basis of the Single-Hidden Layer Feedforward Networks (SLFNs) and can
Adaptive serving gain with Q-Learning
The servo gain λ determines the convergence time and the stability of system. An adaptive gain can make the system performance more stable and the convergence time shorter. Because Q-learning dose not require any rulebase about environment in advance and adaptive with the environment, it is suitable to design the gain for decision making. In this study, the camera velocity is and the gain matrix of camera velocity is . Six
System Description and simulation results
In this section, the performance of the proposed method is confirmed by simulating in MATLAB. Four 3D point projections to the camera plane as the image feature. Because the static target mode is considered, the desired feature points sd are unchanged. The task is completed when the feature errors is less than ε pixels. The initial position and desire position are
The data set of ELMs is
Conclusion
In this paper, a novel IBVS system with ELM and Q-learning has been proposed. This method approximates by ELM that avoids the singularity of the pseudoinverse of the interaction matrix and is robust to camera calibration errors and feature noise. The servo gain is critical to the performance of the system. Q-learning can learn the gain by multiple trainings, and the Q table eventually converges to the optimal steps of control cycles. The adaptive gain control can be realized by using the
Compliance with Ethical Standards
Conflict of Interest: Meng Kang and Hao Chen are students in Northeastern University. Jiuxiang Dong is a professor in Northeastern University. The authors declare that they have no conflict of interest.
CRediT authorship contribution statement
Meng Kang: Conceptualization, Methodology, Software, Validation, Writing - review & editing. Hao Chen: Software, Data curation, Validation. Jiuxiang Dong: Conceptualization, Methodology, Resources, Supervision.
Declaration of Competing Interest
The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.
Acknowledgment
This work was supported in part by the National Natural Science Foundation of China under Grant 61873056, Grant 61621004 and Grant 61420106016, the Fundamental Research Funds for the Central Universities in China under Grant N2004001, N2004002, N182608004 and the Research Fund of State Key Laboratory of Synthetical Automation for Process Industries in China under Grant 2013ZCX01.
Meng Kang received the B.S.degrees in Automation from Yanshan university in 2017. She is now pursuing the M.S. degree in College of Information Science and Engineering, Northeastern University, China. Her research interests focus on visual servoing and robot control.
References (36)
- et al.
Visual tracking via dynamic tensor analysis with mean update
Neurocomputing
(2011) - et al.
Stochastic resonance in visual cortical neurons: does the eye-tremor actually improve visual acuity
Neurocomputing
(2002) - et al.
A neural controller for image-based visual servoing of manipulators with physical constraints
IEEE Trans. Neural Netw. Learn. Syst.
(2018) Intelligent visual servoing with extreme learning machine and fuzzy logic
Expert Syst. Appl.
(2017)- et al.
Partially decoupled image-based visual servoing using different sensitive features
IEEE Trans. Syst. Man Cybern. Syst.
(2017) - et al.
Orthogonal image features for visual servoing of a 6-DOF manipulator with uncalibrated stereo cameras
IEEE Trans. Rob.
(2016) - et al.
A hybrid switched reactive-based visual servo control of 5-DOF robot manipulators for pick-and-place tasks
IEEE Syst. J.
(2015) - et al.
Vision-based model predictive control for steering of a nonholonomic mobile robot
IEEE Trans. Control Syst. Technol.
(2016) - et al.
Adaptive image-based trajectory tracking control of wheeled mobile robots with an uncalibrated fixed camera
IEEE Trans. Control Syst. Technol.
(2015) - et al.
Visual servoing trajectory tracking of nonholonomic mobile robots without direct position measurement
IEEE Trans. Rob.
(2014)
Unified visual servoing tracking and regulation of wheeled mobile robots with an uncalibrated camera
IEEE/ASME Trans. Mechatron.
Visual servoing of quadrotors for perching by hanging from cylindrical objects
IEEE Robot. Autom. Lett.
Robust control of four-rotor unmanned aerial vehicle with disturbance uncertainty
IEEE Trans. Ind. Electron.
Hierarchical model predictive image-based visual servoing of underwater vehicles with adaptive neural network dynamic control
IEEE Trans. Cybern.
Mosaic-based navigation for autonomous underwater vehicles
IEEE J. Oceanic Eng.
Relative end-effector control using cartesian position based visual servoing
IEEE Trans. Robot. Autom.
Novel position-based visual servoing approach to robust global stability under field-of-view constraint
IEEE Trans. Ind. Electron.
Manipulator Control with Image-based Visual Servo
Proceedings 1991 IEEE International Conference on Robotics & Automation
Cited by (28)
Deep learning-based pose prediction for visual servoing of robotic manipulators using image similarity
2022, NeurocomputingCitation Excerpt :In [33], Zuo et al. presented a method that predicts the joint points for calculating the pose of a manipulator. Recently, Kang et al. used reinforcement learning to realize visual servoing with an uncalibrated camera in [34]. Inspired by the correlation between the current camera pose and the image observed by a camera, namely, the observation that the more similar the observed images are, the closer the poses of the camera, it is noted that the error between the images is more prominent than the error between the pose labels in the training process.
Comparison of visual servoing technologies for robotized aerospace structural assembly and inspection
2022, Robotics and Computer-Integrated ManufacturingCitation Excerpt :The system was effective for assembling flexible metal parts with a 0.5 mm tolerance. Besides of using laser for position control in automation, in the last decade there has been a great effort for defining robot trajectories based upon VS [17,18]. Kang et al. [17] used a Q-learning and extreme learning machine (ELM) in addition to visual servoing image-based as a solution to complex modeling problems and servo gains selection.
A survey Of learning-Based control of robotic visual servoing systems
2022, Journal of the Franklin InstituteCitation Excerpt :Shi et al. [79,80] have successfully applied Q-learning to control image-based multi-agent systems and quadrotor systems to adaptively adjust image Jacobian matrix. Q-learning is combined with extreme learning machine to solve the complex modeling and selection problem of servo gain [81]. For RL, it is inevitable to use some estimation methods to estimate the model or parameters in value functions and policies [82].
Visual Servoing Trajectory Tracking and Depth Identification for Mobile Robots with Velocity Saturation Constraints
2024, IEEE Transactions on Industrial ElectronicsIHUVS: Infinite Homography-Based Uncalibrated Methodology for Robotic Visual Servoing
2024, IEEE Transactions on Industrial Electronics
Meng Kang received the B.S.degrees in Automation from Yanshan university in 2017. She is now pursuing the M.S. degree in College of Information Science and Engineering, Northeastern University, China. Her research interests focus on visual servoing and robot control.
Hao Chen received the B.S.degrees in Electrical Engineering and Automation from Southwest University of Science and Technology in 2016. He received the M.S. degree from College of Information Science and Engineering, Northeastern University, China. His research interests focus on visual servoing and robot control.
Jiuxiang Dong received the B.S. degree in mathematics and applied mathematics, the M.S. degree in applied mathematics from Liaoning Normal University, China, in 2001 and 2004, respectively. He received the Ph.D. degree in navigation guidance and control from Northeastern University, China in 2009. He is currently a Professor at the College of Information Science and Engineering, Northeastern University. His research interests include fuzzy control, robust control and reliable control. Dr. Dong is an Associate Editor for the International Journal of Control, Automation, and Systems (IJCAS).