Three-dimensional mapping and immersive human–robot interfacing utilize Kinect-style depth cameras and virtual reality for agricultural mobile robots,Journal of Field Robotics

当前位置： X-MOL 学术 › J. Field Robot. › 论文详情

Our official English website, www.x-mol.net, welcomes your feedback! (Note: you will need to create a separate account there.)

Three-dimensional mapping and immersive human–robot interfacing utilize Kinect-style depth cameras and virtual reality for agricultural mobile robots
Journal of Field Robotics ( IF 8.3 ) Pub Date : 2024-01-28 , DOI: 10.1002/rob.22294
Dong Wang _{1,

2} , Baohua Zhang ₁ , Jun Zhou ₃ , Yingjun Xiong ₁ , Longshen Liu ₁ , Qianqiu Tan ₁

Affiliation

Navigation and autonomous operation in farmland or greenhouse environments present significant challenges for agricultural mobile robots on account of unstructured scene conditions and complex task requirements. The immersive interfaces based on human–robot interaction have been extensively developed, which bridge the high-level decision of human beings and the precise sensing and high motion ability of robots. In this study, we propose a human–robot interaction framework based on three-dimensional mapping and virtual reality (VR) visualization. On the construction client, dense three-dimensional point-cloud maps are created based on the simultaneous localization and mapping real-time appearance-based mapping algorithm with a Kinect-style depth camera. Subsequently, point-cloud filtering and mesh creation methods are introduced for postprocessing the map models. A data communication network is employed to transmit the optimized models to the VR-based exploration client. Utilizing VR visualization, the operator can gain an intuitive and comprehensive understanding of the environments to be explored. The three-dimensional agricultural scenes are mapped into VR models through the system, which can combine the physical worlds with the VR spaces in a better manner. Compared with the video streaming-based method for three-dimensional mapping, the map models with colors and textures are entirely displayed in the VR headset, thus providing robust and global human–robot interaction interfaces and enhancing the immersive exploration for human beings to a considerable extent. Experimental results indicate that the three-dimensional map models exhibit relatively sufficient construction integrity and yield favorable optimization effects. Additionally, according to a user questionnaire survey, immersive VR-based interfaces can be more effectively utilized for the three-dimensional mapping of mobile robots. Research evidence and results demonstrate that our proposed system framework offers positive assistance and support for enhancing human–robot interaction and executing specific robotic tasks in unstructured field agricultural environments. Code implementation and related data sets are shared at the link https://github.com/WangDongBUAA/Mapping_Interfacing.

更新日期：2024-01-29

点击分享查看原文

点击收藏

阅读更多本刊最新论文本刊介绍/投稿指南

全部期刊列表>>