In this paper, a novel High Efficiency Video Coding (HEVC)-compliant perceptual rate-distortion optimization (RDO) scheme is proposed based on motion attention and visual distortion sensitivity models, which both fully utilize in-loop coding information of HEVC. In detail, the motion attention model is designed by using the motion vectors (MVs) estimated during the inter-prediction process. The MV field is refined based on maximum a posteriori (MAP) estimation to remove MV outliers and improve the model’s efficiency. In addition, the visual distortion sensitivity is modeled by using the spatiotemporal energy of AC coefficients, which are obtained from HEVC transform process. Then, these two models are incorporated together into the RDO process. As a result, the Lagrange multiplier and quantization parameter are adjusted adaptively in an analytical way. Since the two models are calculated within the HEVC coding loop, the complexity increase is limited. The experimental results indicate that the proposed perceptual RDO scheme can achieve significantly better rate-VQM performance than the conventional RDO scheme. Specifically, the BD-rate can reach a maximum 24.45% and an average 13.68% reduction in terms of the Bjontegaard Delta metric compared to HEVC practical encoder x265.

中文翻译：

---

HEVC实时编码器基于环内感知模型的速率失真优化

本文基于运动注意力和视觉失真敏感度模型，提出了一种新的高效视频编码（HEVC）兼容的感知率失真优化（RDO）方案，两者都充分利用了HEVC的环路编码信息。详细地，通过使用在帧间预测过程中估计的运动矢量（MV）来设计运动注意模型。MV字段基于最大后验来精炼（MAP）估计以消除MV异常值并提高模型的效率。此外，通过使用从HEVC变换过程获得的AC系数的时空能量对视觉失真敏感度进行建模。然后，将这两个模型一起整合到RDO流程中。结果，以分析方式自适应地调整拉格朗日乘数和量化参数。由于这两个模型是在HEVC编码循环内计算的，因此复杂度的增加受到了限制。实验结果表明，与传统的RDO方案相比，提出的感知RDO方案可以实现明显更好的rate-VQM性能。具体而言，与HEVC实用编码器x265相比，按照Bjontegaard Delta度量标准，BD速率最高可达到24.45％，平均降低13.68％。