Skip to main content
SpringerLink
Log in

An automated model for the assessment of QoE of adaptive video streaming over wireless networks

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Nowadays, heterogeneous devices are widely utilizing Hypertext Transfer Protocol (HTTP) to transfer the data. Furthermore, HTTP adaptive video streaming (HAS) technology transmits the video data over wired and wireless networks. In adaptive technology services, a client’s application receives a streaming video through the adaptation of its quality to the network condition. However, such a technology has increased the demand for Quality of Experience (QoE) in terms of prediction and assessment. It can also cause a challenging behavior regarding subjective and objective QoE evaluations of HTTP adaptive video over time since each Quality of Service (QoS) parameter affects the QoE of end-users separately. This paper introduces a methodology design for the evaluation of subjective QoE in adaptive video streaming over wireless networks. Besides, some parameters are considered such as video characteristics, segment length, initial delay, switch strategy, stalls, as well as QoS parameters. The experiment’s evaluation demonstrated that objective metrics can be mapped to the most significant subjective parameters for user’s experience. The automated model could function to demonstrate the importance of correlation for network behaviors’ parameters. Consequently, it directly influences the satisfaction of the end-user’s perceptual quality. In comparison with other recent related works, the model provided a positive Pearson Correlation value. Simulated results give a better performance between objective Structural Similarity (SSIM) and subjective Mean Opinion Score (MOS) evaluation metrics for all video test samples.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Similar content being viewed by others

References

  1. Abar T, Letaifa AB, Elasmi S (2018) Enhancing QoE based on machine learning and DASH in SDN networks. 32nd International Conference on Advanced Information Networking and Applications Workshops (WAINA), IEEE, Krakow, Poland, pp. 258–263, https://doi.org/10.1109/WAINA.2018.00095

  2. Absolute Category Rating (ACR) ITU-R BT.500–12. Recommendation ITU-R BT.500-13Methodology for the subjective assessment of the quality of television pictures (Jan. 2012), [Accessed November 2019 Online]

  3. Barman N, Martini MG (2019) QoE modeling for HTTP adaptive video streaming–a survey and open challenges. IEEE Access 7:30831–30859. https://doi.org/10.1109/ACCESS.2019.2901778

    Article  Google Scholar 

  4. Barman N, Zadtootaghaj S, Schmidt S, Martini MG, Möller S (2018) An objective and subjective quality assessment study of passive gaming video streaming. International Journal of Network Management, pp. e2054. DOI: https://doi.org/10.1002/nem.2054

  5. Barman N, Jammeh E, Ghorashi SA, Martini MG (2019) No-reference video quality estimation based on machine learning for passive gaming video streaming applications. IEEE Access 7:74511–74527. https://doi.org/10.1109/ACCESS.2019.2920477

    Article  Google Scholar 

  6. Bulkan U, Dagiuklas T (2019) Predicting quality of experience for online video service provisioning. Multimed Tools Appl 78(13):18787–18811. https://doi.org/10.1007/s11042-019-7164-9

    Article  Google Scholar 

  7. Chen Y, Wu K, Zhang Q (2014) From QoS to QoE: a tutorial on video quality assessment. IEEE Commun Surveys Tutorials 17(2):1126–1165. https://doi.org/10.1109/COMST.2014.2363139

    Article  Google Scholar 

  8. Chikkerur S, Sundaram V, Reisslein M, Karam LJ (2011) Objective video quality assessment methods: a classification, review, and performance comparison. IEEE Trans Broadcast 57(2):165–182. https://doi.org/10.1109/TBC.2011.2104671

    Article  Google Scholar 

  9. Cisco report, [Accessed November 2019 Online]: http://bit.ly/2gg1F6B

  10. Claeys M, Latre S, Famaey J, De Turck F (2014) Design and evaluation of a self-learning HTTP adaptive video streaming client. IEEE Commun Lett 18(4):716–719. https://doi.org/10.1109/LCOMM.2014.020414.132649

    Article  Google Scholar 

  11. Cofano G, De Cicco L, Zinner T, Nguyen-Ngoc A, Tran-Gia P, Mascolo S (2016) Design and experimental evaluation of network-assisted strategies for HTTP adaptive streaming. In Proceedings of the 7th International Conference on Multimedia Systems, pp. 1–12. https://doi.org/10.1145/2910017.2910597

  12. Duanmu Z, Zeng K, Ma K, Rehman A, Wang Z (2016) A quality-of-experience index for streaming video. IEEE J Selected Topics Sign Process 11(1):154–166. https://doi.org/10.1109/10.1109/JSTSP.2016.2608329

    Article  Google Scholar 

  13. García L, Lloret J, Turro C, Taha M (2016) QoE assesment of MPEG-DASH in polimedia e-learning system. International Conference on Advances in Computing, Communications and Informatics (ICACCI), IEEE, pp. 1117–1123. DOI: https://doi.org/10.1109/ICACCI.2016.7732194

  14. García B, López-Fernández L, Gortázar F, Gallego M (2019) Practical Evaluation of VMAF Perceptual Video Quality for WebRTC Aepplications. Journal of Electronics, vol. 9, no. 2. https://doi.org/10.3390/electronics8080854

  15. Guan-MingSu XS, Bai Y, Wang M, Vasilakos AV, Wang H (2016) QoE in video streaming over wireless networks: perspectives and research challenges. Wirel Netw 22(5):1571–1593. https://doi.org/10.1007/s11276-015-1028-7

    Article  Google Scholar 

  16. Gutterman C, Guo K, Arora S, Wang X, Wu L, Katz-Bassett E, Zussman G (2019) Requet: Real-time QoE detection for encrypted YouTube traffic. In Proceedings of the 10th ACM Multimedia Systems Conference, pp. 48–59. https://doi.org/10.1145/3304109.3306226

  17. Huawei report, [Accessed December 2019 Online]: https://www.huawei.com/minisite/hwmbbf15/img/video_coverage_whitepaper_en.pdf.

  18. ITU-T Rec. P.1203. (2017) Parametric bitstream-based quality assessment of progressive download and adaptive audiovisual streaming services over reliable transport. ITU-T Rec. P.1203

  19. Juluri P, Tamarapalli V, Medhi D (2015) Measurement of quality of experience of video-on-demand services: a survey. IEEE Commun Surveys Tutorials 18(1):401–418. https://doi.org/10.1109/COMST.2015.2401424

    Article  Google Scholar 

  20. Juluri P, Tamarapalli V, Medhi D (2016) Measurement of quality of experience of video-on-demand services: a survey. IEEE Commun Surveys Tutorials 18(1):401–418. https://doi.org/10.1109/COMST.2015.2401424

    Article  Google Scholar 

  21. Liu X, Nan Z, Richard Yu F, Chen Y, Tang J, Leung VCM (2018) Cooperative video transmission strategies via caching in small-cell networks. IEEE Trans Veh Technol 67(12):12204–12217. https://doi.org/10.1109/TVT.2018.2874258

    Article  Google Scholar 

  22. MingfuLi C-LY, Shao-Yu L (2018) Real-time QoE monitoring system for video streaming services with adaptive media playout. Int J Digital Multimed Broadcast 2018:1–11. https://doi.org/10.1155/2018/2619438

    Article  Google Scholar 

  23. Mok R, Luo X, Chan E, Chang R (2012) QDASH: A QoE-aware DASH system. Proceedings of the 3rd Multimedia Systems Conference, pp. 11–22. https://doi.org/10.1145/2155555.2155558

  24. Moorthy AK, Seshadrinathan K, Soundararajan R, Bovik AC (2010) Wireless video quality assessment: a study of subjective scores and objective algorithms. IEEE trans Circuits Syst Vid Technol 20(4):587–599. https://doi.org/10.1109/TCSVT.2010.2041829

    Article  Google Scholar 

  25. Nam H, Kim K-H, Schulzrinne H (2016) QoE matters more than QoS: Why people stop watching cat videos. In IEEE INFOCOM 2016-The 35th Annual IEEE International Conference on Computer Communications, IEEE, pp. 1–9. https://doi.org/10.1109/INFOCOM.2016.7524426

  26. Nunome T, Mizutani K (2019) The joint effect of wireless LAN reliable groupcast and a rate-adaptation mechanism on QoE of audio and video transmission. in Proc. IEEE 26th International Conference on Telecommunications (ICT), IEEE, pp. 149–153. https://doi.org/10.1109/ICT.2019.8798643

  27. Nunome T, Tani H (2017) The effect of seeking operation on QoE of HTTP adaptive streaming services. Int J Comput Netw Commun (IJCNC) 9(2):1–18. https://doi.org/10.5121/ijcnc.2017.9201

    Article  Google Scholar 

  28. Orsolic I, Pevec D, Suznjevic M, Skorin-Kapov L (2017) A machine learning approach to classifying YouTube QoE based on encrypted network traffics. Multimed Tools Appl 76(21):22267–22301. https://doi.org/10.1007/s11042-017-4728-4

    Article  Google Scholar 

  29. Pal D, Vanijja V (2017) Effect of network QoS on user QoE for a mobile video streaming service using H. 265/VP9 codec. Proced Comput Sci 111:214–222. https://doi.org/10.1016/j.procs.2017.06.056

    Article  Google Scholar 

  30. Petrangeli S, Van Der Hooft J, Wauters T, De Turck F (2018) Quality of experience-centric management of adaptive video streaming services: status and challenges. ACM Trans Multimed Comput Commun Appl (TOMM) 14(2):1–29. https://doi.org/10.1145/3165266

    Article  Google Scholar 

  31. Poojary S, El-Azouzi R, Altman E, Sunny A, Triki I, Haddad M, Jimenez T, Valentin S, Tsilimantos D (2018) Analysis of QoE for adaptive video streaming over wireless networks. 16th International Symposium on Modeling and Optimization in Mobile, Ad Hoc, and Wireless Networks (WiOpt), IEEE, pp. 1–8, https://doi.org/10.23919/WIOPT.2018.8362846

  32. Retal S, Bagaa M, Taleb T, Flinck H (2017) Content delivery network slicing: QoE and cost awareness. IEEE International Conference on Communications (ICC), IEEE, pp. 1–6, https://doi.org/10.1109/ICC.2017.7996499

  33. Rodrigues F, Sousa I, Queluz MP, Rodrigues A (2018) QoE-aware scheduling algorithm for adaptive HTTP video delivery in wireless networks. Wireless Communications and Mobile computing, vol. 2018. DIO: https://doi.org/10.1155/2018/9736360

  34. Schatz R, Sackl A, Timmerer C, Gardlo B (2017) Towards subjective quality of experience assessment for omnidirectional video streaming." Ninth International Conference on Quality of Multimedia Experience (QoMEX), IEEE, no. 6, pp. 1-, https://doi.org/10.1109/QoMEX.2017.7965657

  35. Seufert M, Egger S, Slanina M, Zinner T, Hoßfeld T, Tran-Gia P (2014) A survey on quality of experience of HTTP adaptive streaming. IEEE Commun Surveys Tutorials 17(1):469–492. https://doi.org/10.1109/COMST.2014.2360940

    Article  Google Scholar 

  36. Singh S, Andrews JG, de Veciana G (2012) Interference shaping for improved quality of experience for real-time video streaming. IEEE J Selected Areas Commun 30(7):1259–1269. https://doi.org/10.1109/JSAC.2012.120811

    Article  Google Scholar 

  37. Stensen JMG (2012) Evaluating QoS and QoE Dimensions in Adaptive Video Streaming. Master's thesis, Institutt for telematikk, https://doi.org/10.12142/ZTECOM.201901004

  38. Streijl RC, Winkler S, Hands DS (2016) Mean opinion score (MOS) revisited: methods and applications, limitations and alternatives. Multimedia Systems 22(2):213–227. https://doi.org/10.1007/s00530-014-0446-1

    Article  Google Scholar 

  39. Taha M (2016) A novel CDN testbed for fast deploying HTTP adaptive video streaming. Proceedings of the 9th EAI International Conference on Mobile Multimedia Communications, pp. 65–71. DOI: https://doi.org/10.4108/eai.18-6-2016.2264163

  40. Taha M, Lloret J, Ali A, Garcia L (2018) Adaptive video streaming testbed design for performance study and assessment of QoE. Int J Commun Syst 31(9):e3551. https://doi.org/10.1002/dac.3551

    Article  Google Scholar 

  41. Thang TC, Le HT, Nguyen HX, Pham AT, Kang JW, Ro YM (2013) Adaptive video streaming over HTTP with dynamic resource estimation. J Commun Netw 15(6):635–644. https://doi.org/10.1109/JCN.2013.000112

    Article  Google Scholar 

  42. Timmerer C, Zabrovskiy A (2019) Automating QoS and QoE Evaluation of HTTP Adaptive Streaming Systems. ZTE COMMUNICATIONS, vol. 17, no. 1

  43. Trakas P, Adelantado F, Zorba N, Verikoukis C (2017) A quality of experience-aware association algorithm for 5G heterogeneous networks. In 2017 IEEE International Conference on Communications (ICC), IEEE, pp. 1–6. DOI: https://doi.org/10.1109/ICC.2017.7996869

  44. Villa BJ, Heegaard PE (2012) Improving Fairness in QoS and QoE domains for Adaptive Video Streaming. International Journal on Advances in Networks and Services vol. 5, no. 3, pp. 4

  45. Qingyong Wang, Hong-Ning Dai, Di Wu, and Hong Xiao (2018) Data analysis on video streaming QoE over mobile networks. Eurasip J Wireless Commun Netw, no. 1, pp.173, https://doi.org/10.1186/s13638-018-1180-8

  46. Wenjing L, Yu P, Wang R, Lei F, Dong O, Xuesong Q (2017) Quality of experience evaluation of HTTP video streaming based on user interactive behaviors. J China Univ Posts Telecommun 24(3):24–32. https://doi.org/10.1016/S1005-8885(17)60208-5

    Article  Google Scholar 

  47. Xu Y, Zhou Y, Chiu D-M (2014) Analytical QoE models for bit-rate switching in dynamic adaptive streaming systems. IEEE Trans Mob Comput 13(12):2734–2748. https://doi.org/10.1109/TMC.2014.2307323

    Article  Google Scholar 

  48. Zeng H, Fang Y (2013) Implementation of video transcoding client based on FFMPEG. In Advanced Materials Research, vol. 756, Trans Tech Publications Ltd, pp. 1748–1752. 10.4028/www.scientific.net/AMR.756-759.1748

  49. Zhang W, Wen Y, Chen Z, Khisti A (2013) QoE-driven cache management for HTTP adaptive bit rate streaming over wireless networks. IEEE Trans Multimed 15(6):1431–1445. https://doi.org/10.1109/TMM.2013.2247583

    Article  Google Scholar 

  50. Zhao T, Liu Q, Chen CW (2016) QoE in video transmission: a user experience-driven strategy. IEEE Commun Surveys Tutorials 19(1):285–302. https://doi.org/10.1109/COMST.2016.2619982

    Article  Google Scholar 

Download references

Acknowledgments

This work has been partially supported by the “Ministerio de Economía y Competitividad” in the “Programa Estatal de Fomento de la Investigación Científica y Técnica de Excelencia, Subprograma Estatal de Generación de Conocimiento” within the Project under Grant TIN2017-84802-C2-1-P. This study has been partially done in the computer science departments at the (University of Sulaimani and Halabja).

Author information

Authors and Affiliations

  1. Department of Computer Science, University of Sulaimani, Sulaymaniyah, Iraq

    Miran Taha & Aree Ali

  2. Integrated Management Coastal Research Institute, Universitat Politécnica de Valencia, Valencia, Spain

    Miran Taha, Jaime Lloret, Paulo R. L. Gondim & Alejandro Canovas

  3. Computer Department, College of Science, University of Halabja, Halabja, Iraq

    Aree Ali

  4. University of Brasilia, Brasilia, Brazil

    Paulo R. L. Gondim

Authors
  1. Miran Taha
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Aree Ali
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jaime Lloret
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Paulo R. L. Gondim
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Alejandro Canovas
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jaime Lloret.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Taha, M., Ali, A., Lloret, J. et al. An automated model for the assessment of QoE of adaptive video streaming over wireless networks. Multimed Tools Appl 80, 26833–26854 (2021). https://doi.org/10.1007/s11042-021-10934-9

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-021-10934-9

Keywords

Search

Navigation