To read this content please select one of the options below:

Industrial robot programming by demonstration using stereoscopic vision and inertial sensing

João Pedro C. de Souza (DEEC, University of Porto and CRIIS, INESC TEC, Porto, Portugal)
António M. Amorim (CRIIS, INESC TEC, Porto, Portugal)
Luís F. Rocha (CRIIS, INESC TEC, Porto, Portugal)
Vítor H. Pinto (DEEC, University of Porto and CRIIS, INESC TEC, Porto, Portugal)
António Paulo Moreira (DEEC, University of Porto and CRIIS, INESC TEC, Porto, Portugal)

Industrial Robot

ISSN: 0143-991x

Article publication date: 17 August 2021

Issue publication date: 3 January 2022

175

Abstract

Purpose

The purpose of this paper is to present a programming by demonstration (PbD) system based on 3D stereoscopic vision and inertial sensing that provides a cost-effective pose tracking system, even during error-prone situations, such as camera occlusions.

Design/methodology/approach

The proposed PbD system is based on the 6D Mimic innovative solution, whose six degrees of freedom marker hardware had to be revised and restructured to accommodate an IMU sensor. Additionally, a new software pipeline was designed to include this new sensing device, seeking the improvement of the overall system’s robustness in stereoscopic vision occlusion situations.

Findings

The IMU component and the new software pipeline allow the 6D Mimic system to successfully maintain the pose tracking when the main tracking tool, i.e. the stereoscopic vision, fails. Therefore, the system improves in terms of reliability, robustness, and accuracy which were verified by real experiments.

Practical implications

Based on this proposal, the 6D Mimic system reaches a reliable and low-cost PbD methodology. Therefore, the robot can accurately replicate, on an industrial scale, the artisan level performance of highly skilled shop-floor operators.

Originality/value

To the best of the authors’ knowledge, the sensor fusion between stereoscopic images and IMU applied to robot PbD is a novel approach. The system is entirely designed aiming to reduce costs and taking advantage of an offline processing step for data analysis, filtering and fusion, enhancing the reliability of the PbD system.

Keywords

Acknowledgements

This research work is supported by the Portuguese Foundation for Science and Technology (FCT) project COBOTIS (PTDC/EME-EME/32595/2017) and by the FCT PhD program grant DFA/BD/5352/2020. This work is also co-financed by the ERDF – European Regional Development Fund through the Operational Programme for Competitiveness and Internationalization – COMPETE 2020 and the Lisboa2020 under the PORTUGAL 2020 Partnership Agreement and through the Portuguese National Innovation Agency (ANI) as a part of project PRODUTECH SIF: POCI-01–0247-FEDER-024541.

Citation

de Souza, J.P.C., Amorim, A.M., Rocha, L.F., Pinto, V.H. and Moreira, A.P. (2022), "Industrial robot programming by demonstration using stereoscopic vision and inertial sensing", Industrial Robot, Vol. 49 No. 1, pp. 96-107. https://doi.org/10.1108/IR-02-2021-0043

Publisher

:

Emerald Publishing Limited

Copyright © 2021, Emerald Publishing Limited

Related articles