Event cameras are biologically-inspired sensors that gather the temporal evolution of the scene. They capture pixel-wise brightness variations and output a corresponding stream of asynchronous events. Despite having multiple advantages with respect to conventional cameras, their use is limited due to the scarce compatibility of asynchronous event streams with traditional data processing and vision algorithms. In this regard, we present a framework that synthesizes RGB frames from the output stream of an event camera and an initial or a periodic set of color key-frames. The deep learning-based frame synthesis framework consists of an adversarial image-to-image architecture and a recurrent module. Two public event-based datasets, DDD17 and MVSEC, are used to obtain qualitative and quantitative per-pixel and perceptual results. In addition, we converted into event frames two additional well-known datasets, namely Kitti and Cityscapes, in order to present semantic results, in terms of object detection and semantic segmentation accuracy. Extensive experimental evaluation confirms the quality and the capability of the proposed approach of synthesizing frame sequences from color key-frames and sequences of intermediate events.

中文翻译：

---

结合传统相机和事件相机的视频帧合成

事件摄像机是受生物启发的传感器，可收集场景的时间演变。它们捕获逐像素的亮度变化并输出相应的异步事件流。尽管相对于传统相机具有多种优势，但由于异步事件流与传统数据处理和视觉算法的兼容性不足，它们的使用受到限制。在这方面，我们提出了一个框架，它从事件相机的输出流和初始或周期性的一组颜色关键帧中合成 RGB 帧。基于深度学习的帧合成框架由对抗性图像到图像架构和循环模块组成。两个基于事件的公共数据集，DDD17 和MVSEC, 用于获得定性和定量的每像素和感知结果。此外，我们将另外两个众所周知的数据集转换为事件框架，即基蒂和城市景观，为了呈现语义结果，在对象检测和语义分割精度方面。广泛的实验评估证实了所提出的从彩色关键帧和中间事件序列合成帧序列的方法的质量和能力。