This study presents a multisensory machine learning architecture for object recognition by employing a novel dataset that was constructed with the iCub robot, which is equipped with three cameras and a depth sensor. The proposed architecture combines convolutional neural networks to form representations (i.e., features) for grayscaled color images and a multi-layer perceptron algorithm to process depth data. To this end, we aimed to learn joint representations of different modalities (e.g., color and depth) and employ them for recognizing objects. We evaluate the performance of the proposed architecture by benchmarking the results obtained with the models trained separately with the input of different sensors and a state-of-the-art data fusion technique, namely decision level fusion. The results show that our architecture improves the recognition accuracy compared with the models that use inputs from a single modality and decision level multimodal fusion method.

中文翻译：

---

一种用于旋转不变物体识别的多感官学习架构

本研究通过采用由 iCub 机器人构建的新型数据集，提出了一种用于物体识别的多感官机器学习架构，该数据集配备了三个摄像头和一个深度传感器。所提出的架构结合了卷积神经网络以形成灰度彩色图像的表示（即特征）和处理深度数据的多层感知器算法。为此，我们旨在学习不同模态（例如颜色和深度）的联合表示，并将它们用于识别对象。我们通过对使用不同传感器的输入和最先进的数据融合技术（即决策级融合）分别训练的模型获得的结果进行基准测试来评估所提出架构的性能。