Compared with deadzone hard-decision quantization (HDQ), rate-distortion optimized quantization (RDOQ) in HEVC brings non-negligible coding gain, however consumes considerable computations caused by exhaustive search over multiple candidates to determine optimal output level. Benefiting from efficient prediction in HEVC, transform blocks are frequently quantized to all zero, especially in small-size blocks. It is worthwhile to detect all zero block (AZB) for transform blocks to bypass subsequent computation-intensive RDOQ. Traditional thresholding based AZB detection algorithms are well-suited for deadzone quantized blocks, however miss partial optimal results in RDOQ and suffer from more or less accuracy degradation in RDOQ. This paper proposes a novel multi-stage AZB detection algorithm for RDOQ blocks with good tradeoff between complexity and accuracy. At the first stage, genuine all zero blocks (G_AZB) which are quantized to all zero both in HDQ and RDOQ are prejudged by comparison with conservative threshold determined by mathematical derivation for deadzone HDQ. At the second stage, an adaptive threshold model is built using adaptive deadzone offset by simulating the behavior patterns existing in RDOQ, aiming to further detect the pseudo AZB (P_AZB) which are quantized to all zero in RDOQ however not all zero in HDQ. At the final stage, machine learning based detection is proposed to classify the remaining “cunning” all zero blocks using eight distinguished RDO-related features, by which subtle working mechanism in RDOQ is leveraged. The experimental results demonstrate that the proposed algorithm achieves up to 7.471% total coding computation saving with 0.064% BD-RATE increment compared with RDOQ on average. Moreover, the average FNR and FPR detection accuracies are 6.3% and 6.5% respectively.

与死区硬决策量化（HDQ）相比，HEVC中的速率失真优化量化（RDOQ）带来了不可忽略的编码增益，但是，由于对多个候选进行穷举搜索来确定最佳输出电平，因此会消耗大量计算量。得益于HEVC中的高效预测，变换块经常被量化为全零，尤其是在小型块中。值得为变换块检测所有零块（AZB），以绕过后续的计算密集型RDOQ。传统的基于阈值的AZB检测算法非常适合死区量化块，但是错过了RDOQ中的部分最优结果，并在RDOQ中或多或少地降低了精度。本文针对RDOQ块提出了一种新颖的多级AZB检测算法，该算法在复杂度和准确性之间取得了良好的折衷。在第一阶段，通过与由死区HDQ的数学推导确定的保守阈值比较，来预先判断在HDQ和RDOQ中都被量化为全零的真正全零块（G_AZB）。在第二阶段，通过模拟RDOQ中存在的行为模式，使用自适应死区偏移构建自适应阈值模型，旨在进一步检测伪AZB（P_AZB），这些伪AZB在RDOQ中量化为全零，但在HDQ中并非全为零。在最后阶段，提出了基于机器学习的检测方法，以使用八种与RDO相关的独特功能对所有“零”剩余块进行“分类”，从而利用RDOQ中的细微工作机制。实验结果表明，与平均RDOQ相比，该算法可节省多达7.471％的总编码计算量，且BD-RATE增量为0.064％。此外，平均FNR和FPR检测准确度分别为6.3％和6.5％。