Robotic seam tracking system based on vision sensing and human-machine interaction for multi-pass MAG welding

https://doi.org/10.1016/j.jmapro.2020.02.026Get rights and content

Abstract

In the seam tracking for the metal active gas (MAG) welding of heavy plate, there are many problems, such as the strong arc light due to the large current, spatter and fumes, the losses of centerline information and complex conditions in groove induced by previous passes, etc. The problems often make the characteristic in the images needed for seam tracking indistinct and make it difficult for industrial robots to correct the deviation of multi-pass welding. Regarding this issue, a robotic seam tracking system based on vision sensing and human-machine interaction for the multi-pass MAG welding was proposed. The groove edges were used as reference to tell the deviation and to control it according to a simple, fast and effective control strategy based on human-machine interaction. The deviation could be judged by comparing the left or right groove edge alternatively with the corresponding specified positioning ruler from the real-time images obtained by industrial camera. The operators could adjust the torch position in real time by the hint on the positioning ruler, which followed the control strategy. The accuracy, response rate and smoothness of the system were verified by reasonable path planning in multi-pass MAG welding. By comparing the deviation correction of robot with the actual deviation of weld seam, the accuracy of system was rated, which had an average error less than 0.5 mm and was acceptable. The comparison between the actual weld height and width after correction and the designed height and width showed that the system worked smoothly and steadily and the system could track the weld in real time.

Introduction

Along with the development of social productivity and the progress of science and technology, welding robots are more and more widely used in industrial production. Welding robots can not only greatly improve the production efficiency and quality, but also can greatly reduce the amount of labor and reduce the technical requirements on the operation of workers. Welding robots have the advantages of stability, reliability and high repetition accuracy. Automation, digitization and intelligentization will be the important development direction of welding process, and the research of seam tracking has become one of the significant research fields of welding equipment [1,2].

At present, there are many researches on robotic welding sensing and control. Observing the 3D weld pool surface by structured laser, directly viewing the work-piece by a camera, tracking the operator’s motions with the help of the Leap sensor, Liu and Zhang [3] proposed a predictive robot arm teleoperation scheme to achieve the virtual welding. The scheme could help transfer human knowledge to welding robots. Liu and Zhang [4] proposed a fuzzy weighting based fusion approach to combine different machine and human intelligent models for developing intelligent welding robot in gas tungsten arc welding (GTAW). It has strong robustness. Liu and Zhang [5] also proposed a supervised adaptive neuro-fuzzy inference system model, which correlated the 3-D weld pool characteristic parameters and adjustment on the welding speed by welder. The model can drive the welding robot to perform automated welding task with an intelligent controller under different welding current as well as under disturbances in speed and measurement. Wang et al. [6] established a mapping model between arc voltage, arc length and welding current. They then proposed a virtual reality human-robot collaborative welding system based on the model, which allowed the robot to track the weld seam automatically. Wang et al. [7] also presented a virtual reality human–robot interaction welding system that allowed human welders to manipulate a welding robot and undertake welding tasks naturally and intuitively via consumer-grade virtual reality hardware. All these foregoing researches promoted the intelligentization of welding processes and the application of robots in welding field. Human intelligence was introduced to help robots / machine to finish some intelligent jobs in welding which are difficult or impossible for robots / machine now. Researches on modeling the behaviors of operator by machine learning, further to enhance the intelligence of robots / machine, started to draw attentions in the field of intelligent welding.

In actual welding production, the strong arc light, spatter and fumes etc. often make the characteristic in the images needed for seam tracking to be indistinct. It makes it difficult for industrial robots to correct the deviation of welding torch and affects the weld quality. In the field of seam tracking, new vision sensing technology, new control theory, and neural network technology are all applied in welding seam tracking technology [[8], [9], [10], [11], [12]]. In the current application of seam tracking technology, through-the-arc method and vision sensing method are relatively common and proven. Arc voltage and arc signal obtained by the arc sensor, and the weld groove information obtained by the visual sensor are often used in seam tracking applications, which greatly improves the seam tracking technology [13,14].

When the welding wire swings horizontally in the welding groove, the distance between the workpiece and the end of the welding wire will change, which will lead to changes in welding current and voltage. The swing arc sensor is based on that to conduct signal sensing to implement the seam tracking. Kodama et al. [15,16] designed a simple and efficient electromagnetic high speed swing arc sensor. They used the sensor to obtain the welding current and arc voltage signals based on the principle of high frequency oscillation arc, and to improve the accuracy of welding seam tracking greatly. Xu et al. [17] extracted the characteristic of arc voltage and established the linear relationship model between arc voltage and arc length by using an appropriate denoising algorithm. The model met the requirements of the seam tracking and controlling during the robotic GTAW process. Based on a well-known seam-tracking algorithm, Baek et al. [18] designed a special measurement device for the arc voltage, and proposed an automatic seam-tracking and weaving width control algorithm to predict weld position more accurately during the GTAW process. But with the increase in the welding speed, the signal to noise ratio of the swing arc sensor will decrease and the weld shape will go worse, because of the low swing frequency. Using a high-speed rotating arc sensor directly driven by a hollow shaft motor, Jia et al. [19] employed the characteristic harmonic detection method to identify the deviation of torch to conduct the tracking test for metal inert gas arc welding of V-groove, obtaining a good effectiveness. Taking the arc voltage waveform as the characteristic signal, Kim [20] developed an arc sensor for reciprocating wire feed gas metal arc welding, which was used to detect the torch height and track weld seams.

Although the through-the-arc method is often affected by the shape of weld groove, the swing frequency of arc and other factors, it is still applied in the industrial applications due to its advantages, for example, no extra sensor is needed; it is unaffected by the arc light or high temperature; it can adapt to noise intensity and other complex environments, and so on.

The other most widely used seam tracking technology in industrial production is the vision sensing method. It can obtain numerous information from a capacious detection space. Its accuracy, real-time performance, versatility are good.

Section snippets

Current status of seam tracking based on vision sensing

According to the light sources, the vision sensing methods in welding field are mainly divided into active vision method and passive vision method. The active vision method uses extra light source as measurement tools or illumination, while the passive vision method uses the light sources only from the welding process, such as arc and weld pool radiation, to assist sensing.

Overall design

KUKA robot (Model: KR 16 L8 arc HW) was selected to be the actuator. When it runs, weld correction is implemented through the Robot Sensor Interface (RSI) of its controller. In the multi-pass welding process, the deviation judgement is difficult in the complex groove / weld pass conditions, thus, human intelligent is proposed to solve this problem. The operator transmits the analog deviation signal to RSI, which enables the robot to conduct a small range of adjustment based on the preset path,

Conclusions

Seam tracking was first reviewed in detail and the challenges in multi-pass welding were summarized. A robotic seam tracking system for multi-pass MAG welding based on vision sensing and human-machine interaction was proposed and tested in this work, and the following conclusions can be drawn:

  • (1)

    The industrial camera was placed in front of the welding torch to monitor the welding area from the upper left view, which distorts the welding images in the monitoring interface. The position of weld

Declaration of interests

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Acknowledgements

Thanks for the financial support from Tianhe Mechanical Equipment Manufacturing Co., Ltd.

References (28)

  • J. Muhammad et al.

    Welding seam profiling techniques based on active vision sensing for intelligent robotic welding

    Int J Adv Manuf Technol

    (2017)
  • S. Kamya et al.

    Fuzzy logic based image de-noising and enhancement for grayscale images

    Int J Comput Appl

    (2013)
  • L.R. Tang et al.

    A new fuzzy logic image de-noising algorithm based on gradient detection

    International conference on fuzzy systems & knowledge discovery. IEEE

    (2007)
  • Y.S. Zhang et al.

    Weld feature extraction based on fully convolutional network

    China J Lasers

    (2019)
  • Cited by (28)

    • Research on robust laser vision feature extraction method for fillet welds with different reflective materials under uncertain interference

      2023, Optics and Laser Technology
      Citation Excerpt :

      Robot welding has been widely used in shipbuilding, automobile, aircraft, petrochemical industry and other industrial fields due to its advantages of continuous long time working and stable welding quality [1–5].

    • Seam tracking system based on laser vision and CGAN for robotic multi-layer and multi-pass MAG welding

      2022, Engineering Applications of Artificial Intelligence
      Citation Excerpt :

      In order to realize the seam tracking system for robotic multi-layer and multi-pass welding based on linear laser vision, the real-time motion control for the robot must be solved. Xue et al. (2021) proposed a seam tracking system based on passive vision sensing and human–machine interaction for multi-pass welding, and the position correction was realized through the KUKA robot sensor interface (RSI). However, the proposed system cannot realize the fully automatic seam tracking because a human operator was needed to determine the position deviation and then adjust the welding torch position.

    • A teaching-free welding position guidance method for fillet weld based on laser vision sensing and EGM technology

      2022, Optik
      Citation Excerpt :

      Hou et al. [5] used an image processing method based on prior knowledge with laser vision to realize teaching-free experiments on V-grooves and fillet welds. Xue et al. [19] proposed a teaching-free method based on sensing and human-machine interaction for multi-pass MAG welding, the torch position by the hint on the positioning ruler can be adjusted by the operators in real-time. Zheng et al. [20] developed a knowledge-based program generation approach that can increase programming efficiency and is successfully proven in manufacturing cases.

    View all citing articles on Scopus
    View full text