Skip to main content
SpringerLink
Log in

Content adaptive pre-filtering for video compression

  • Original Paper
  • Published:
Signal, Image and Video Processing Aims and scope Submit manuscript

Abstract

Bitrate reduction with little to no degradation in visual perception is a long-standing challenge in video coding. This paper targets this challenge by adaptively filtering the content prior to video compression and in the preprocessing stage. This is done by applying a bilateral filter where the filter parameters are selected according to regional content complexity and estimated visual importance besides bitrate and quality requirements. A multi-scale metric based on 2D gradient is employed to determine bandwidth requirements of different regions. A random forest regression model is trained to predict distortion and bit requirements for a block, if it is filtered and encoded at a given quality. The predicted distortion and bit requirements are used to select filter parameters considering a cost function. The proposed approach is applied to both H.264 and HEVC encoders, with different GOP structures. The results show up to 60% bitrate reduction in terms of BD-Rate (about 20% on average) for the attempted test cases with little to no noticeable quality degradation.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

Notes

  1. Python sklearn notations are used for different forest parameters.

  2. Consider all possible combinations of i, j, f in \(B_{i,j}^f\), and also different quality (Q) values and different content types.

  3. Test sequences are from [2] and http://media.xiph.org/video/derf/.

References

  1. Bjontegaard, G.: Calculation of average PSNR differences between RD-curves. In: Proceedings of the ITU-T Video Coding Experts Group, Thirteenth Meeting (2001)

  2. Bossen, F.: Common test conditions and software reference configurations. JCT-VC, Technical report I1100 (2012)

  3. Breiman, L.: Random forests. Mach. Learn. 45(1), 5–32 (2001)

    Article  Google Scholar 

  4. Chen, Z., Liu, H.: JND modeling: approaches and applications. In: International Conference on Digital Signal Processing, pp. 827–830 (2014)

  5. Duvar, R., Akbulut, O., Urhan, O.: Fast inter mode decision exploiting intra-block similarity in HEVC. Signal Process. Image Commun. 78, 503–510 (2019)

    Article  Google Scholar 

  6. Grois, D., Nguyen, T., Marpe, D.: Performance comparison of AV1, JEM, VP9, and HEVC encoders. In: Proceedings of SPIE, 10396, (2018)

  7. Guo, L., Cock, J.D., Aaron, A.: Compression performance comparison of x264, x265, libvpx and aomenc for on-demand adaptive streaming applications. In: Picture Coding Symposium, pp. 26–30 (2018)

  8. Heidari, B., Ramezanpour, M.: Reduction of intra-coding time for HEVC based on temporary direction map. J. Real-Time Image Process. (2018). https://doi.org/10.1007/s11554-018-0815-7

    Article  Google Scholar 

  9. Huang, X., Zhang, Q., Zhao, X., Zhang, W., Zhang, Y., Gan, Y.: Fast inter-prediction mode decision algorithm for HEVC. Signal Image Video Process. 11(1), 33–40 (2017)

    Article  Google Scholar 

  10. Jenab, M., Amer, I., Ivanovic, B., Saeedi, M., Liu, Y., Sines, G., Shirani, S.: Content-adaptive resolution control to improve video coding efficiency. In: IEEE International Conference on Multimedia Expo Workshops, pp. 1–4 (2018)

  11. Karunaratne, P.V., Segall, C.A., Katsaggelos, A.K.: A rate-distortion optimal video pre-processing algorithm. In: International Conference on Image Processing, 1, 481–484 (2001)

  12. Kerofsky, L.J., Vanam, R., Reznik, Y.A.: Improved adaptive video delivery system using a perceptual pre-processing filter. In: Signal and Information Processing, pp. 1058–1062 (2014)

  13. Ki, S., Bae, S., Kim, M., Ko, H.: Learning-based just-noticeable-quantization-distortion modeling for perceptual video coding. IEEE Trans. Image Process. 27(7), 3178–3193 (2018)

    Article  MathSciNet  Google Scholar 

  14. Kim, W.J., Yi, J.W., Kim, S.D.: A bit allocation method based on picture activity for still image coding. IEEE Trans. Image Process. 8(7), 974–977 (1999)

    Article  Google Scholar 

  15. Lee, J., Ebrahimi, T.: Perceptual video compression: a survey. IEEE J.Sel. Top. Signal Process. 6(6), 684–697 (2012)

    Article  Google Scholar 

  16. Li, S., Zhang, F., Ma, L., Ngan, K.N.: Image quality assessment by separately evaluating detail losses and additive impairments. IEEE Trans. Multimed. 13(5), 935–949 (2011)

    Article  Google Scholar 

  17. Li, Z., Aaron, A., Katsavounidis, I., Moorthy, A., Manohara, M.: Toward a practical perceptual video quality metric. Netflix Technology Blog (2016)

  18. Li, Z., Bampis, C., Novak, J., Aaron, A., Swanson, K., Moorthy, A., Cock, J.D.: VMAF: The journey continues. Netflix Technology Blog (2018)

  19. Lu, S.P., Zhang, S.R.: Saliency-based fidelity adaptation preprocessing for video coding. J. Comput. Sci. Technol. 26(1), 195–202 (2011)

    Article  Google Scholar 

  20. Najafabadi, N., Ramezanpour, M.: Mass center direction-based decision method for intraprediction in HEVC standard. J. Real-Time Image Process. (2019). https://doi.org/10.1007/s11554-019-00864-z

    Article  Google Scholar 

  21. Nguyen, T., Marpe, D.: Future video coding technologies: a performance evaluation of AV1, JEM, VP9, and HM. In: Picture Coding Symposium, pp. 31–35 (2018)

  22. Oh, H., Kim, W.: Video processing for human perceptual visual quality-oriented video coding. IEEE Trans. Image Process. 22(4), 1526–1535 (2013)

    Article  MathSciNet  Google Scholar 

  23. Ramezanpour, M., Zargari, F.: Fast CU size and prediction mode decision method for HEVC encoder based on spatial features. Signal Image Video Process. 10(7), 1233–1240 (2016)

    Article  Google Scholar 

  24. Rassool, R.: VMAF reproducibility: validating a perceptual practical video quality metric. In: IEEE International Symposium on Broadband Multimedia Systems and Broadcasting, pp. 1–2 (2017)

  25. Shaw, M.Q., Allebach, J.P., Delp, E.J.: Color difference weighted adaptive residual preprocessing using perceptual modeling for video compression. Image Commun. 39, 355–368 (2015)

    Google Scholar 

  26. Sheikh, H.R., Bovik, A.C.: Image information and visual quality. IEEE Trans. Image Process. 15(2), 430–444 (2006)

    Article  Google Scholar 

  27. Tomasi, C., Manduchi, R.: Bilateral filtering for gray and color images. In: International Conference on Computer Vision, pp. 839–846 (1998)

  28. Vanam, R., Kerofsky, L.J., Reznik, Y.A.: Perceptual pre-processing filter for adaptive video on demand content delivery. In: IEEE International Conference on Image Processing, pp. 2537–2541 (2014)

  29. Vidal, E., Sturmel, N., Guillemot, C., Corlay, P., Coudoux, F.X.: New adaptive filters as perceptual preprocessing for rate-quality performance optimization of video coding. Signal Process. Image Commun. 52, 124–137 (2017)

    Article  Google Scholar 

  30. Yang, X., Ling, W., Lu, Z., Ong, E.P., Yao, S.: Just noticeable distortion model and its applications in video coding. Signal Process. Image Commun. 20(7), 662–680 (2005)

    Article  Google Scholar 

  31. Zhang, X., Lin, W., Wang, S., Liu, J., Ma, S., Gao, W.: Fine-grained quality assessment for compressed images. IEEE Trans Image Process. 28(3), 1163–1175 (2019)

    Article  MathSciNet  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Advanced Micro Devices, Inc. (AMD), 1 Commerce Valley Dr E, Markham, ON, L3T 7X6, Canada

    Mehdi Saeedi, Boris Ivanovic, Tomasz Stolarczyk, Ihab Amer & Gabor Sines

Authors
  1. Mehdi Saeedi
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Boris Ivanovic
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Tomasz Stolarczyk
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Ihab Amer
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Gabor Sines
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Mehdi Saeedi.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Saeedi, M., Ivanovic, B., Stolarczyk, T. et al. Content adaptive pre-filtering for video compression. SIViP 14, 935–943 (2020). https://doi.org/10.1007/s11760-019-01625-y

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11760-019-01625-y

Keywords

Search

Navigation