Event-based cameras are novel, efficient sensors inspired by the human vision system, generating an asynchronous, pixel-wise stream of data. Learning from such data is generally performed through heavy preprocessing and event integration into images. This requires buffering of possibly long sequences and can limit the response time of the inference system. In this work, we instead propose to directly use events from a DVS camera, a stream of intensity changes and their spatial coordinates. This sequence is used as the input for a novel \emph{asynchronous} RNN-like architecture, the Input-filtering Neural ODEs (INODE). This is inspired by the dynamical systems and filtering literature. INODE is an extension of Neural ODEs (NODE) that allows for input signals to be continuously fed to the network, like in filtering. The approach naturally handles batches of time series with irregular time-stamps by implementing a batch forward Euler solver. INODE is trained like a standard RNN, it learns to discriminate short event sequences and to perform event-by-event online inference. We demonstrate our approach on a series of classification tasks, comparing against a set of LSTM baselines. We show that, independently of the camera resolution, INODE can outperform the baselines by a large margin on the ASL task and it's on par with a much larger LSTM for the NCALTECH task. Finally, we show that INODE is accurate even when provided with very few events.

中文翻译：

---

使用输入过滤神经 ODE 对短事件相机流进行实时分类

基于事件的相机是受人类视觉系统启发的新型高效传感器，可生成异步的像素级数据流。从这些数据中学习通常是通过大量的预处理和事件集成到图像中来执行的。这需要缓冲可能很长的序列，并可能限制推理系统的响应时间。在这项工作中，我们建议直接使用来自 DVS 相机的事件、强度变化流及其空间坐标。该序列用作新型 \emph{asynchronous} RNN 类架构的输入，即输入过滤神经 ODE（INODE）。这是受到动态系统和过滤文献的启发。INODE 是 Neural ODE (NODE) 的扩展，它允许将输入信号连续馈送到网络，就像在过滤中一样。该方法通过实现批量前向欧拉求解器自然地处理具有不规则时间戳的批量时间序列。INODE 像标准 RNN 一样进行训练，它学习区分短事件序列并执行逐个事件的在线推理。我们在一系列分类任务上展示了我们的方法，并与一组 LSTM 基线进行比较。我们表明，独立于相机分辨率，INODE 在 ASL 任务上可以大大优于基线，并且在 NCALTECH 任务中与更大的 LSTM 相当。最后，我们表明即使提供很少的事件，INODE 也是准确的。它学习区分短事件序列并执行逐个事件的在线推理。我们在一系列分类任务上展示了我们的方法，并与一组 LSTM 基线进行比较。我们表明，独立于相机分辨率，INODE 在 ASL 任务上可以大大优于基线，并且在 NCALTECH 任务中与更大的 LSTM 相当。最后，我们表明即使提供很少的事件，INODE 也是准确的。它学习区分短事件序列并执行逐个事件的在线推理。我们在一系列分类任务上展示了我们的方法，并与一组 LSTM 基线进行比较。我们表明，独立于相机分辨率，INODE 在 ASL 任务上可以大大优于基线，并且在 NCALTECH 任务中与更大的 LSTM 相当。最后，我们表明即使提供很少的事件，INODE 也是准确的。