Peg-in-hole assembly in live-line maintenance based on generative mapping and searching network☆
Introduction
Uninterrupted power supply plays an important role in people’s daily life. In order to ensure the continuous and stable power supply, it is necessary for workers to do dangerous live-line maintenance work at high altitudes. Obviously, developing an intelligent robot to do the maintenance work autonomously can avoid operation risks and improve the work efficiency, which is of great significance [1], [2], [3].
However, the scene of live-line maintenance is unstructured, and the typical parts such as the cable is flexible and prone to deformation. The task which involves peg-in-hole assembly, requires high accuracy of control [4]. Therefore, robots should have not only the ability to detect objects and scenes similar to human eyes, but also the dexterous operation ability similar to human hands, as well as the learning, adaptation and decision-making ability similar to human brain. It is a very challenging task to develop a live-line maintenance robot which can complete such complex task.
Lightning arrester replacement is one of the common tasks in live-line maintenance. The lower screw of the arrester is inserted into the hole of the cross arm and fixed by the nut, and the upper screw is used as the terminal to fix end of the cable. The most difficult problem for replacing the lightning arrester is to install the crimp terminal on the upper screw of the arrester, which can be regarded as a peg-in-hole assembly, as the upper screw of the arrester can be regarded as a “peg”, and the hole of the crimp terminal is regarded as a “hole”. Fig. 1(a) shows the typical parts of the peg-in-hole assembly. In the installation process, when the crimp terminal is close to the screw, it will inevitably block the screw, and it is difficult for the manipulator to distinguish the peg and the hole under the visual detection, as shown in Fig. 1(b). Furthermore, when the manipulator grasps the flexible cable, it is difficult to build an accurate force model for the cable due to its deformation, as shown in Fig. 1(c).
In this paper, the peg-in-hole assembly strategy based on vision and force are studied, taking robotic lightning arrester replacement as an example, and aiming at the difficult problem of installing the crimp terminal on the screw. The main contribution of this paper is to propose an approach for the manipulator to autonomously complete the assembly of the crimp terminal and the screw. Contribution details are as follows:
- •
We divide the task of installing the crimp terminal into two phases, the rough alignment by visual servo and the precise hole-searching by visual/force fusion. The visual servo in the phase of rough alignment is designed to make the crimp terminal close to the screw rapidly when the distance between them is far away.
- •
We establish a two-dimensional hole-searching model. The positional relationship between the crimp terminal and the screw can be judged by the visual and force sensor.
- •
We propose a hole-searching policy-Generative Mapping and Searching Network (GMSN), which estimates the optimal routine from a known two-dimensional state map. The training result of GMSN is applied in the actual environment, as it was trained only in the virtual environment. Through this network, the time consuming of hole searching can be effectively reduced and the success rate is improved.
The rest of this paper is organized as follows. In Section 2, the related work in this area is presented. The proposed approach is given in detail in Section 3. In Section 4, we perform the experimental analysis, which is followed by a summary and prospects for future research in Section 5.
Section snippets
Related work
The peg-in-hole assembly with manipulator has been widely used in the operation of production lines, which is a very difficult task [5], [6]. Strategies of the solution can be divided into four types, including visual servo, force/torque servo, control without additional sensors, and hole searching policy.
The peg-in-hole assembly can be accomplished by using the position control under structured and sample scenes. The position difference between the target hole and the peg is obtained as the
Methodology
In order to automatically accomplish the peg-in-hole assembly, the manipulator needs to have the ability to identify and locate the crimp terminal and the screw spatially. The location information can be obtained by depth-camera-based target detection and ranging. Nevertheless, the information is not accurate enough to complete high-precision hole alignment. In addition, when the crimp terminal is close to the screw, it will block the screw, resulting in signal loss. As a result, it is
Experiments
We set up the experimental platform by referring to the actual distribution lines, and took pictures of the actual scene to make the background panel. The experimental platform was equipped with a manipulator (UR5), a depth camera (Intel Realsense D435) behind the manipulator, a force/torque sensor(Robotiq FT300), the lightning arrester and desktop computer, as shown in Fig. 7. The proposed network is trained in Titan XP, and experiments are conducted in an indoor environment. In this paper,
Conclusion
This paper presents a novel framework for the peg-in-hole assembly in live-line maintenance. YOLOv3 network and visual servo are used to realize the rough alignment of the crimp terminal and the screw. A two-dimensional hole-searching model is established by fusing signals of vision and force. Through this model, the state of each waypoint on the search routine can be obtained. Furthermore, we propose a new semi-supervised learning network-GMSN, which contains the G-net for estimating the true
Declaration of Competing Interest
The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.
Acknowledgments
This work was supported by National Natural Science Foundation of China (61973167), and Key Research and Development Project of Jiangsu Province (BE2017161).
Wei Wu received the B.Sc. degrees in 2015 and is currently pursuing the Ph.D. degree in Control Science and Engineering at Nanjing University of Science and Technology. His research interests focus on the visual servo and autonomous control of robots.
References (33)
- et al.
Dcbot: An autonomous hot-line working robot for 110 kv substation
Robot. Auton. Syst.
(2019) - et al.
State-of-the-art control strategies for robotic pih assembly
Robot. Comput.-Integr. Manuf.
(2020) Development of an intelligent transformer insertion system using a robot arm
Robot. Comput.-Integr. Manuf.
(2018)Robotic assembly of smartphone back shells with eye-in-hand visual servoing
Robot. Comput.-Integr. Manuf.
(2018)- et al.
A machine learning approach for visual recognition of complex parts in robotic manipulation
Procedia Manuf.
(2017) - et al.
A constraint-based programming approach for robotic assembly skills implementation
Robot. Comput.-Integr. Manuf.
(2019) - et al.
An approach for peg-in-hole assembling using intuitive search algorithm based on human behavior and carried by sensors guided industrial robot
IFAC-PapersOnLine
(2015) - et al.
SCARA based peg-in-hole assembly using compliant IPMC micro gripper
Robot. Auton. Syst.
(2013) - et al.
Robotics in power systems: Enabling a more reliable and safe grid
IEEE Ind. Electron. Mag.
(2017) - et al.
A vision-based broken strand detection method for a power-line maintenance robot
IEEE Trans. Power Deliv.
(2014)
Research on mechanism configuration and coordinated control for power distribution network live working robot
Ind. Robot: Int. J. Robot. Res. Appl.
Vision based pose estimation of multiple peg-in-hole for robotic assembly
Hole detection algorithm for chamferless square peg-in-hole based on shape recognition using f/t sensor
Int. J. Precis. Eng. Manuf.
On contact models for assembly tasks: Experimental investigation beyond the peg-in-hole problem on the example of force-torque maps
A hybrid contact state analysis methodology for robotic-based adjustment of cylindrical pair
Int. J. Adv. Manuf. Technol.
Sensing and control for simultaneous precision peg-in-hole assembly of multiple objects
IEEE Trans. Autom. Sci. Eng.
Cited by (7)
Design of a robot system for reorienting and assembling irregular parts
2024, International Journal of Advanced Robotic Systems3D Reconstruction of deformable linear objects based on cylindrical fitting
2023, Signal, Image and Video ProcessingA Novel Disassembly Strategy of Hexagonal Screws Based on Robot Vision and Robot-Tool Cooperated Motion
2023, Applied Sciences (Switzerland)
Wei Wu received the B.Sc. degrees in 2015 and is currently pursuing the Ph.D. degree in Control Science and Engineering at Nanjing University of Science and Technology. His research interests focus on the visual servo and autonomous control of robots.
Hui Zhou received the B.S. degree in control science and engineering, the M.S. degree in instrumentation and automation, and the Ph.D. degree in control theory and engineering from Huazhong University of Science and Technology, Wuhan, China, in 2005, 2008, and 2012, respectively. Since 2012, he has been an Assistant Professor in the Research Center for Neural Engineering, Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences, Shenzhen, China. Since 2018, he has been an Associate Professor in the School of Automation, Nanjing University of Science and Technology, Nanjing, China. His research interests include rehabilitation robots, motion control and modeling, and human machine interaction.
Yu Guo received the B.Sc. and M.Sc. degrees in Automation, both from Huazhong University of Science and Technology, Wuhan, China, in 1984 and 1987, respectively, and her Ph.D. degree in Control Science and Engineering from Nanjing University of Science and Technology. In 1987, she joined the faculty of the School of Automation, Nanjing University of Science and Technology, and is currently a Professor of Automatic Control there. Her main research interests include intelligent robot control, optimization for complicated systems and so forth.
Yifei Wu received the Ph.D. degree in Automation at Nanjing University of Science and Technology and is currently an Associate Professor of Nanjing University of Science and Technology. His research interests focus on servo motor control and intelligent robot.
Jian Guo received the Ph.D. degree in Automation at Nanjing University of Science and Technology in 2002. He was a visiting scholar at Purdue University from 2008 to 2009 and is currently a Professor of Automatic Control at Nanjing University of Science and Technology. His research interests focus on robot system design and automatic control.