Optical flow estimation forms the core of several computer vision tasks and its estimation requires accurate spatial and temporal gradient information. However, if there are fast-moving objects in the scene or if the camera moves rapidly, then the acquired images will suffer from motion blur, which will lead to poor optical flow estimation. Such challenging cases can be handled by event sensors which are a novel generation of sensors that acquire pixel-level brightness changes as binary events at a very high temporal resolution. Brightness constancy constraint, which is the basis of several optical flow algorithms cannot be directly used on event sensors making it challenging to estimate optical flow. We overcome this challenge by imposing brightness constancy constraint on intensity images predicted from event sensor data. For this task, we design a recurrent neural network that jointly predicts a sparse optical flow and intensity images from the event data. While intensity estimation is supervised using ground truth frames, optical flow estimation is self-supervised using the predicted intensity frames. However, in our case the temporal resolution of the ground truth intensity frames is far lower than the temporal resolution of the predicted intensity frames, making it challenging to supervise. As we use recurrent neural network, such a challenge can be overcome by sharing the weights for each of the predicted intensity frames. Quantitatively our predicted optical flow is better than previously proposed algorithms for optical flow estimation from event sensors. We also show our algorithm’s robustness against challenging cases of fast motion and high dynamic range scenes.

光流估计构成了一些计算机视觉任务的核心，其估计需要准确的空间和时间梯度信息。但是，如果场景中有快速移动的物体或相机快速移动，则采集的图像将遭受运动模糊，这将导致光流估计不佳。事件传感器可以处理这种具有挑战性的情况，事件传感器是新一代的传感器，可以在非常高的时间分辨率下将像素级亮度变化作为二进制事件来获取。亮度恒定性约束是几种光流算法的基础，不能直接在事件传感器上使用，这给估算光流带来了挑战。我们通过对从事件传感器数据预测的强度图像施加亮度恒定约束来克服此挑战。对于此任务，我们设计了一个递归神经网络，可以根据事件数据共同预测稀疏的光流和强度图像。虽然使用地面真帧对强度估计进行监督，但使用预测的强度帧对光流估计进行自我监督。然而，在我们的情况下，地面真实强度帧的时间分辨率远低于预测强度帧的时间分辨率，这使得监督变得困难。当我们使用递归神经网络时，可以通过共享每个预测强度框架的权重来克服这种挑战。从数量上来说，我们预测的光流要比先前提出的用于从事件传感器进行光流估计的算法更好。我们还展示了我们算法在快速运动和高动态范围场景下具有挑战性的情况下的鲁棒性。