当前位置: X-MOL 学术J. Opt. Soc. Am. A › 论文详情
Our official English website, www.x-mol.net, welcomes your feedback! (Note: you will need to create a separate account there.)
Mid-fusion of road scene polarization images on pretrained RGB neural networks
Journal of the Optical Society of America A ( IF 1.9 ) Pub Date : 2021-03-10 , DOI: 10.1364/josaa.413604
Khalid Omer , Meredith Kupinski

This work presents a mid-fusion pipeline that can increase the detection performance of a convolutional neural network (RetinaNet) by including polarimetric images even though the network is trained on a large-scale database containing RGB and monochromatic images (Microsoft COCO). Here, the average precision (AP) for each object class quantifies performance. The goal of this work is to evaluate the usefulness of polarimetry for object detection and recognition of road scenes and determine the conditions that will increase AP. Shadows, reflections, albedo, and other object features that reduce RGB image contrast also decrease the AP. This work demonstrates specific cases for which the AP increases using linear Stokes and polarimetric flux images. Images are fused during the neural network evaluation pipeline, which is referred to as mid-fusion. Here, the AP of polarimetric mid-fusion is greater than the RGB AP in 54 out of 80 detection instances. The recall values for cars and buses are similar for RGB and polarimetry, but values increase from 36% to 38% when using polarimetry for detecting people. Videos of linear Stokes images for four different scenes are collected at three different times of the day for two driving directions. Despite this limited dataset and the use of a pretrained network, this work demonstrates selective enhancement of object detection through mid-fusion of polarimetry to neural networks trained on RGB images.

中文翻译:

预训练RGB神经网络上道路场景极化图像的融合

这项工作提出了一种融合中的管道,即使该网络在包含RGB和单色图像的大型数据库(Microsoft COCO)上进行了训练,也可以通过包括极化图像来提高卷积神经网络(RetinaNet)的检测性能。在这里,每个对象类别的平均精度(AP)可以量化性能。这项工作的目的是评估极化仪对目标检测和道路场景识别的有用性,并确定将增加AP的条件。降低RGB图像对比度的阴影,反射,反照率和其他对象特征也会降低AP。这项工作演示了使用线性斯托克斯和极化通量图像增加AP的特定情况。在神经网络评估流程中将图像融合,这称为中间融合。在此,在80个检测实例中的54个检测实例中,偏光融合的AP大于RGB AP。汽车和公共汽车的召回值在RGB和偏振法上相似,但是当使用偏振法检测人员时,召回值从36%增加到38%。在一天中的三个不同时间,针对两个行驶方向,收集了四个不同场景的线性斯托克斯图像的视频。尽管数据集有限并且使用了预训练的网络,但这项工作证明了通过极化技术与在RGB图像上训练的神经网络的中间融合,可以选择性地增强对象检测。在一天中的三个不同时间,针对两个行驶方向,收集了四个不同场景的线性斯托克斯图像的视频。尽管数据集有限并且使用了预训练的网络,但这项工作证明了通过极化技术与在RGB图像上训练的神经网络的中间融合,可以选择性地增强对象检测。在一天中的三个不同时间,针对两个行驶方向,收集了四个不同场景的线性斯托克斯图像的视频。尽管数据集有限并且使用了预训练的网络,但这项工作证明了通过极化技术与在RGB图像上训练的神经网络的中间融合,可以选择性地增强对象检测。
更新日期:2021-04-01
down
wechat
bug