Robotic seam tracking system based on vision sensing and human-machine interaction for multi-pass MAG welding
Introduction
Along with the development of social productivity and the progress of science and technology, welding robots are more and more widely used in industrial production. Welding robots can not only greatly improve the production efficiency and quality, but also can greatly reduce the amount of labor and reduce the technical requirements on the operation of workers. Welding robots have the advantages of stability, reliability and high repetition accuracy. Automation, digitization and intelligentization will be the important development direction of welding process, and the research of seam tracking has become one of the significant research fields of welding equipment [1,2].
At present, there are many researches on robotic welding sensing and control. Observing the 3D weld pool surface by structured laser, directly viewing the work-piece by a camera, tracking the operator’s motions with the help of the Leap sensor, Liu and Zhang [3] proposed a predictive robot arm teleoperation scheme to achieve the virtual welding. The scheme could help transfer human knowledge to welding robots. Liu and Zhang [4] proposed a fuzzy weighting based fusion approach to combine different machine and human intelligent models for developing intelligent welding robot in gas tungsten arc welding (GTAW). It has strong robustness. Liu and Zhang [5] also proposed a supervised adaptive neuro-fuzzy inference system model, which correlated the 3-D weld pool characteristic parameters and adjustment on the welding speed by welder. The model can drive the welding robot to perform automated welding task with an intelligent controller under different welding current as well as under disturbances in speed and measurement. Wang et al. [6] established a mapping model between arc voltage, arc length and welding current. They then proposed a virtual reality human-robot collaborative welding system based on the model, which allowed the robot to track the weld seam automatically. Wang et al. [7] also presented a virtual reality human–robot interaction welding system that allowed human welders to manipulate a welding robot and undertake welding tasks naturally and intuitively via consumer-grade virtual reality hardware. All these foregoing researches promoted the intelligentization of welding processes and the application of robots in welding field. Human intelligence was introduced to help robots / machine to finish some intelligent jobs in welding which are difficult or impossible for robots / machine now. Researches on modeling the behaviors of operator by machine learning, further to enhance the intelligence of robots / machine, started to draw attentions in the field of intelligent welding.
In actual welding production, the strong arc light, spatter and fumes etc. often make the characteristic in the images needed for seam tracking to be indistinct. It makes it difficult for industrial robots to correct the deviation of welding torch and affects the weld quality. In the field of seam tracking, new vision sensing technology, new control theory, and neural network technology are all applied in welding seam tracking technology [[8], [9], [10], [11], [12]]. In the current application of seam tracking technology, through-the-arc method and vision sensing method are relatively common and proven. Arc voltage and arc signal obtained by the arc sensor, and the weld groove information obtained by the visual sensor are often used in seam tracking applications, which greatly improves the seam tracking technology [13,14].
When the welding wire swings horizontally in the welding groove, the distance between the workpiece and the end of the welding wire will change, which will lead to changes in welding current and voltage. The swing arc sensor is based on that to conduct signal sensing to implement the seam tracking. Kodama et al. [15,16] designed a simple and efficient electromagnetic high speed swing arc sensor. They used the sensor to obtain the welding current and arc voltage signals based on the principle of high frequency oscillation arc, and to improve the accuracy of welding seam tracking greatly. Xu et al. [17] extracted the characteristic of arc voltage and established the linear relationship model between arc voltage and arc length by using an appropriate denoising algorithm. The model met the requirements of the seam tracking and controlling during the robotic GTAW process. Based on a well-known seam-tracking algorithm, Baek et al. [18] designed a special measurement device for the arc voltage, and proposed an automatic seam-tracking and weaving width control algorithm to predict weld position more accurately during the GTAW process. But with the increase in the welding speed, the signal to noise ratio of the swing arc sensor will decrease and the weld shape will go worse, because of the low swing frequency. Using a high-speed rotating arc sensor directly driven by a hollow shaft motor, Jia et al. [19] employed the characteristic harmonic detection method to identify the deviation of torch to conduct the tracking test for metal inert gas arc welding of V-groove, obtaining a good effectiveness. Taking the arc voltage waveform as the characteristic signal, Kim [20] developed an arc sensor for reciprocating wire feed gas metal arc welding, which was used to detect the torch height and track weld seams.
Although the through-the-arc method is often affected by the shape of weld groove, the swing frequency of arc and other factors, it is still applied in the industrial applications due to its advantages, for example, no extra sensor is needed; it is unaffected by the arc light or high temperature; it can adapt to noise intensity and other complex environments, and so on.
The other most widely used seam tracking technology in industrial production is the vision sensing method. It can obtain numerous information from a capacious detection space. Its accuracy, real-time performance, versatility are good.
Section snippets
Current status of seam tracking based on vision sensing
According to the light sources, the vision sensing methods in welding field are mainly divided into active vision method and passive vision method. The active vision method uses extra light source as measurement tools or illumination, while the passive vision method uses the light sources only from the welding process, such as arc and weld pool radiation, to assist sensing.
Overall design
KUKA robot (Model: KR 16 L8 arc HW) was selected to be the actuator. When it runs, weld correction is implemented through the Robot Sensor Interface (RSI) of its controller. In the multi-pass welding process, the deviation judgement is difficult in the complex groove / weld pass conditions, thus, human intelligent is proposed to solve this problem. The operator transmits the analog deviation signal to RSI, which enables the robot to conduct a small range of adjustment based on the preset path,
Conclusions
Seam tracking was first reviewed in detail and the challenges in multi-pass welding were summarized. A robotic seam tracking system for multi-pass MAG welding based on vision sensing and human-machine interaction was proposed and tested in this work, and the following conclusions can be drawn:
- (1)
The industrial camera was placed in front of the welding torch to monitor the welding area from the upper left view, which distorts the welding images in the monitoring interface. The position of weld
Declaration of interests
The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.
Acknowledgements
Thanks for the financial support from Tianhe Mechanical Equipment Manufacturing Co., Ltd.
References (28)
- et al.
Fusing machine algorithm with welder intelligence for adaptive welding robots
J Manuf Process
(2017) - et al.
Virtual reality human-robot collaborative welding: a case study of weaving gas tungsten arc welding
J Manuf Process
(2019) - et al.
Weld image deep learning-based on-line defects detection using convolutional neural networks for Al alloy in robotic arc welding
J Manuf Process
(2019) - et al.
Real-time seam tracking control technology during welding robot GTAW process based on passive vision sensor
J Mater Process Technol
(2012) - et al.
A study on automatic seam tracking in pulsed laser edge welding by using a vision sensor without an auxiliary light source
J Manuf Syst
(2002) - et al.
The development and the application of the industrial robot technology
Mach Build Autom
(2015) - et al.
Research status and application of welding intelligence
Electr Weld Mach
(2019) - et al.
Toward welding robot with human knowledge: a remotely-controlled approach
IEEE Trans Autom Sci Eng
(2015) - et al.
Supervised learning of human welder behaviors for intelligent robotic welding
IEEE Trans Autom Sci Eng
(2017) - et al.
Modeling of human welders’ operations in virtual reality human-robot interaction
IEEE Robot Autom Lett
(2019)
Welding seam profiling techniques based on active vision sensing for intelligent robotic welding
Int J Adv Manuf Technol
Fuzzy logic based image de-noising and enhancement for grayscale images
Int J Comput Appl
A new fuzzy logic image de-noising algorithm based on gradient detection
International conference on fuzzy systems & knowledge discovery. IEEE
Weld feature extraction based on fully convolutional network
China J Lasers
Cited by (28)
A novel welding path generation method for robotic multi-layer multi-pass welding based on weld seam feature point
2023, Measurement: Journal of the International Measurement ConfederationResearch on robust laser vision feature extraction method for fillet welds with different reflective materials under uncertain interference
2023, Optics and Laser TechnologyCitation Excerpt :Robot welding has been widely used in shipbuilding, automobile, aircraft, petrochemical industry and other industrial fields due to its advantages of continuous long time working and stable welding quality [1–5].
Trajectory planning method with grinding compensation strategy for robotic propeller blade sharpening application
2023, Journal of Manufacturing ProcessesSeam tracking system based on laser vision and CGAN for robotic multi-layer and multi-pass MAG welding
2022, Engineering Applications of Artificial IntelligenceCitation Excerpt :In order to realize the seam tracking system for robotic multi-layer and multi-pass welding based on linear laser vision, the real-time motion control for the robot must be solved. Xue et al. (2021) proposed a seam tracking system based on passive vision sensing and human–machine interaction for multi-pass welding, and the position correction was realized through the KUKA robot sensor interface (RSI). However, the proposed system cannot realize the fully automatic seam tracking because a human operator was needed to determine the position deviation and then adjust the welding torch position.
Application of sensing technology in intelligent robotic arc welding: A review
2022, Journal of Manufacturing ProcessesA teaching-free welding position guidance method for fillet weld based on laser vision sensing and EGM technology
2022, OptikCitation Excerpt :Hou et al. [5] used an image processing method based on prior knowledge with laser vision to realize teaching-free experiments on V-grooves and fillet welds. Xue et al. [19] proposed a teaching-free method based on sensing and human-machine interaction for multi-pass MAG welding, the torch position by the hint on the positioning ruler can be adjusted by the operators in real-time. Zheng et al. [20] developed a knowledge-based program generation approach that can increase programming efficiency and is successfully proven in manufacturing cases.