With the development of video coding technology, the high efficiency video coding (HEVC) provides better coding efficiency compared to its predecessors H.264/AVC. HEVC improves rate distortion (RD) performance significantly with increased encoding complexity. Due to the adoption of a large variety of coding unit (CU) sizes, at RD optimization level, the quadtree partition of the CU consumes a large proportion of the encoding complexity. Hence, the computational complexity cost remains a critical issue that must be properly considered in the optimization task. In this paper, two machine learning-based fast CU partition method for inter-mode HEVC are proposed, to optimize the complexity allocation at CU level. First, we propose an online support vector machine (SVM)-based fast CU algorithm for reducing HEVC complexity. The later was trained in an online way. Second, a deep convolutional neural network (CNN) is designed to predict the CU partition, in which large-scale training database including substantial CU partition data is considered. Experimental results demonstrate that the proposed online SVM can achieve a time saving of 52.28% with a degradation of 1.928% in the bitrate (BR). However, the proposed deep CNN can reduce the encoding time by 53.99% with 0.195% BR degradation. Compared to the state-of-the art, the two proposed approaches outperform the related works in terms of both RD performance and complexity reduction at inter-mode.

中文翻译：

---

基于快速CU分区的机器学习方法可降低HEVC复杂度

随着视频编码技术的发展，高效视频编码（HEVC）与之前的H.264 / AVC相比提供了更好的编码效率。HEVC通过增加编码复杂性来显着提高速率失真（RD）性能。由于采用了多种编码单元（CU）大小，因此在RD优化级别上，CU的四叉树分区消耗了很大一部分编码复杂度。因此，计算复杂度成本仍然是关键问题，必须在优化任务中适当考虑。本文针对跨模式HEVC，提出了两种基于机器学习的快速CU分割方法，以优化CU级的复杂度分配。首先，我们提出了一种基于在线支持向量机（SVM）的快速CU算法，以降低HEVC的复杂度。后者接受了在线培训。其次，设计了深度卷积神经网络（CNN）来预测CU分区，其中考虑了包含大量CU分区数据的大规模训练数据库。实验结果表明，提出的在线SVM可以节省52.28％的时间，而比特率（BR）降低1.928％。但是，提出的深度CNN可以将编码时间减少53.99％，而BR降低0.195％。与最新技术相比，两种建议的方法在跨模态下的RD性能和复杂性降低方面均胜过相关工作。实验结果表明，提出的在线SVM可以节省52.28％的时间，而比特率（BR）降低1.928％。但是，提出的深层CNN可以将编码时间减少53.99％，而BR降低0.195％。与最新技术相比，两种建议的方法在跨模态下的RD性能和复杂性降低方面均胜过相关工作。实验结果表明，提出的在线SVM可以节省52.28％的时间，而比特率（BR）降低1.928％。但是，提出的深度CNN可以将编码时间减少53.99％，而BR降低0.195％。与最新技术相比，两种建议的方法在跨模态下的RD性能和复杂性降低方面均胜过相关工作。