Near-infrared (NIR) band sensors capture achromatic images that contain complementary details of a scene which are diminished in visible (VS) band images when the scene is obscured by haze, mist, or fog. To exploit these complementary details, an integrated FPGA architecture and implementation of a video processing system are proposed in this paper. This system performs VS-NIR video fusion and produces an enhanced VS video in real-time. The proposed FPGA architecture and implementation effectively handle the challenges associated with the simultaneous processing of video signals obtained from different sources such as the inevitable delay among corresponding frames and time-varying deviation among frame rates. Moreover, the proposed implementation is efficiently designed and able to produce the fused video at the same frame rate as the input videos, i.e. in real-time, regardless of the resolution of the input videos while the consumed FPGA resources are kept small. This is achieved by data and calculations reuse, besides performing operations concurrently in parallel and pipelined fashions at both the data and task levels. The proposed implementation is synthesized, validated on a low-end FPGA device, and compared to three other implementations. The comparison shows the superiority of the proposed implementation in terms of the consumed resources which have a direct industrial impact in the case of integration in modern smart-phones and cameras.

近红外 (NIR) 波段传感器捕获包含场景互补细节的消色差图像，当场景被雾霾、薄雾或雾气遮挡时，这些细节在可见 (VS) 波段图像中会减弱。为了利用这些互补的细节，本文提出了一个集成的 FPGA 架构和视频处理系统的实现。该系统执行 VS-NIR 视频融合并实时生成增强的 VS 视频。所提出的 FPGA 架构和实现有效地处理了与同时处理从不同来源获得的视频信号相关的挑战，例如相应帧之间不可避免的延迟和帧速率之间的时变偏差。而且，所提出的实现经过有效设计，能够以与输入视频相同的帧速率（即实时）生成融合视频，而不管输入视频的分辨率如何，同时消耗的 FPGA 资源保持较小。这是通过数据和计算重用来实现的，除了在数据和任务级别以并行和流水线方式并发执行操作之外。建议的实现是综合的，在低端 FPGA 设备上进行了验证，并与其他三个实现进行了比较。比较显示了拟议实施在消耗资源方面的优越性，这些资源在现代智能手机和相机集成的情况下具有直接的工业影响。这是通过数据和计算重用来实现的，除了在数据和任务级别以并行和流水线方式并发执行操作之外。建议的实现是综合的，在低端 FPGA 设备上进行了验证，并与其他三个实现进行了比较。比较显示了拟议实施在消耗资源方面的优越性，这些资源在现代智能手机和相机集成的情况下具有直接的工业影响。这是通过数据和计算重用来实现的，除了在数据和任务级别以并行和流水线方式并发执行操作之外。建议的实现是综合的，在低端 FPGA 设备上进行了验证，并与其他三个实现进行了比较。比较显示了所提议的实施方案在消耗资源方面的优越性，这些资源在现代智能手机和相机集成的情况下具有直接的工业影响。