Event cameras or neuromorphic cameras mimic the human perception system as they measure the per-pixel intensity change rather than the actual intensity level. In contrast to traditional cameras, such cameras capture new information about the scene at MHz frequency in the form of sparse events. The high temporal resolution comes at the cost of losing the familiar per-pixel intensity information. In this work we propose a variational model that accurately models the behaviour of event cameras, enabling reconstruction of intensity images with arbitrary frame rate in real-time. Our method is formulated on a per-event-basis, where we explicitly incorporate information about the asynchronous nature of events via an event manifold induced by the relative timestamps of events. In our experiments we verify that solving the variational model on the manifold produces high-quality images without explicitly estimating optical flow. This paper is an extended version of our previous work (Reinbacher et al. in British machine vision conference (BMVC), 2016) and contains additional details of the variational model, an investigation of different data terms and a quantitative evaluation of our method against competing methods as well as synthetic ground-truth data.

中文翻译：

---

使用流形正则化的事件相机的实时强度图像重建

事件相机或神经形态相机模拟人类感知系统，因为它们测量每个像素的强度变化而不是实际的强度水平。与传统相机相比，此类相机以稀疏事件的形式以 MHz 频率捕获有关场景的新信息。高时间分辨率是以丢失熟悉的每像素强度信息为代价的。在这项工作中，我们提出了一个变分模型，可以准确地模拟事件相机的行为，从而能够实时重建具有任意帧速率的强度图像。我们的方法是基于每个事件制定的，我们通过事件的相对时间戳引起的事件流形明确地合并有关事件异步性质的信息。在我们的实验中，我们验证了在流形上求解变分模型会产生高质量的图像，而无需明确估计光流。这篇论文是我们之前工作（Reinbacher 等人在英国机器视觉会议 (BMVC)，2016 年）的扩展版本，包含变分模型的更多细节、不同数据项的调查以及我们的方法对抗竞争的定量评估方法以及合成的地面实况数据。