One key challenge to learning-based video compression is that motion predictive coding, a very effective tool for video compression, can hardly be trained into a neural network. In this paper, we propose the concept of PixelMotionCNN (PMCNN) which includes motion extension and hybrid prediction networks. PMCNN can model spatiotemporal coherence to effectively perform predictive coding inside the learning network. On the basis of PMCNN, we further explore a learning-based framework for video compression with additional components of iterative analysis/synthesis and binarization. The experimental results demonstrate the effectiveness of the proposed scheme. Although entropy coding and complex configurations are not employed in this paper, we still demonstrate superior performance compared with MPEG-2 and achieve comparable results with H.264 codec. The proposed learning-based scheme provides a possible new direction to further improve compression efficiency and functionalities of future video coding.

中文翻译：

---

学习视频压缩

基于学习的视频压缩的一个关键挑战是运动预测编码，一种非常有效的视频压缩工具，很难训练成神经网络。在本文中，我们提出了 PixelMotionCNN (PMCNN) 的概念，它包括运动扩展和混合预测网络。PMCNN 可以对时空相干性进行建模，以在学习网络内部有效地执行预测编码。在 PMCNN 的基础上，我们进一步探索了一个基于学习的视频压缩框架，其中包含迭代分析/合成和二值化的附加组件。实验结果证明了所提出方案的有效性。尽管本文没有采用熵编码和复杂配置，但与 MPEG-2 相比，我们仍然展示了优越的性能，并取得了与 H.264 编解码器相当的结果。