Integrating information collected by different types of sensors observing the same or related phenomenon can lead to more accurate and robust decision making. The purpose of this article is to review sensor fusion approaches to achieve passive radio frequency (RF) and electro-optical (EO) sensor fusion and to present the proposed fusion of EO/RF neural network (FERNN). While research has been conducted to integrate complementary data collected by EO and RF modalities, the processing of RF data usually applies traditional features, such as Doppler. This article explores the viability of using the histogram of I/Q (in-phase and quadrature) data for the purposes of augmenting the detection accuracy that EO input alone is incapable of achieving. Specifically, by processing the histogram of I/Q data via deep learning and enhancing feature input for neural network fusion. Using the simulated data from the Digital Imaging and Remote Sensing Image Generation dataset, FERNN can achieve 95% accuracy in vehicle detection and scenario categorization, which is a 23% improvement over the accuracy achieved by a stand-alone EO sensor.

中文翻译：

---

用于无源 RF 和 EO 信息集成的多模态传感器融合综述


整合不同类型的传感器观察相同或相关现象所收集的信息可以导致更准确和稳健的决策。本文的目的是回顾实现无源射频 (RF) 和电光 (EO) 传感器融合的传感器融合方法，并提出所提出的 EO/RF 神经网络 (FERNN) 融合。虽然已经进行了研究以整合 EO 和 RF 模式收集的互补数据，但 RF 数据的处理通常应用传统功能，例如多普勒。本文探讨了使用 I/Q（同相和正交）数据直方图来提高仅 EO 输入无法实现的检测精度的可行性。具体来说，通过深度学习处理 I/Q 数据的直方图，并增强神经网络融合的特征输入。使用来自数字成像和遥感图像生成数据集的模拟数据，FERNN 在车辆检测和场景分类方面可以实现 95% 的准确率，这比独立 EO 传感器实现的准确率提高了 23%。