The Impact of Temporal Impairment on Quality of Experience (QoE) in Video Streaming: A No Reference (NR) Subjective and Objective Study

Live video streaming is one of the most widely used service among end users, yet it is a big challenge for the network operators in terms of quality. The only way to provide excellent Quality of Experience (QoE) to the end users is continuous monitoring of live video streaming. For this purpose, there are several objective algorithms available that monitor the quality of the video in a live stream. Subjective tests play a very important role in fine tuning the results of objective algorithms. As human perception is considered to be the most reliable source for assessing the quality of a video stream subjective tests are conducted in order to develop more reliable objective algorithms. Temporal impairments in a live video stream can have a negative impact on the end users. In this paper we have conducted subjective evaluation tests on a set of video sequences containing temporal impairment known as frame freezing. Frame Freezing is considered as a transmission error as well as a hardware error which can result in loss of video frames on the reception side of a transmission system. In our subjective tests, we have performed tests on videos that contain a single freezing event and also for videos that contain multiple freezing events. We have recorded our subjective test results for all the videos in order to give a comparison on the available No Reference (NR) objective algorithms. Finally, we have shown the performance of no reference algorithms used for objective evaluation of videos and suggested the algorithm that works better. The outcome of this study shows the importance of QoE and its effect on human perception. The results for the subjective evaluation can serve the purpose for validating objective algorithms.




References:
[1] Forecast, Cisco VNI. "Cisco Visual Networking Index: Global Mobile
data Traffic Forecast Update 2009-2014." Cisco Public Information,
February 9 (2010).
[2] Video Quality Experts Group. "Final report from the Video Quality
Experts Group on the validation of objective models of video quality
assessment, Phase II(FR_TV2).", 2003.
[3] Cisco, I. "Cisco visual networking index: Forecast and methodology,
2011–2016." CISCO White Paper (2012): 2011-2016.
[4] Usman, Muhammad Arslan, Muhammad Rehan Usman, and Soo Young
Shin. "Performance Analysis of a No-Reference Temporal Quality
Assessment Metric for Videos Impaired by Frame Freezing Artefacts."
[5] “Subjective video quality assessment methods for multimedia
applications,” September 1999, iTU-T, Recommendation ITU-R P910.
[6] “ITU-R Radio communication Sector of ITU, Recommendation ITU-R
BT.500-12,” 2009.
[7] Wang, Zhou, et al. "Image quality assessment: from error visibility to
structural similarity." Image Processing, IEEE Transactions on 13.4
(2004): 600-612.
[8] Stephen Wolf, “A no reference (nr) and reduced reference (rr) metric for
detecting dropped video frames,” in Second International Workshop on
Video Processing and Quality Metrics for Consumer Electronics
(VPQM), 2009.
[9] Quan Huynh-Thu and M. Ghanbari, “No-reference temporal quality
metric for video impaired by frame freezing artefacts,” in IEEE
International Conference on Image Processing (ICIP), 2009, pp. 2221–
2224.
[10] Borer, Silvio. "A model of jerkiness for temporal impairments in video
transmission." Quality of Multimedia Experience (QoMEX), 2010
Second International Workshop on. IEEE, 2010.
[11] Shahid, Muhammad, et al. "Subjective quality assessment of H.
264/AVC encoded low resolution videos." Image and Signal Processing
(CISP), 2012 5th International Congress on. IEEE, 2012.
[12] Nightingale, James, et al. "The impact of network impairment on quality
of experience (QoE) in H. 265/HEVC video streaming." Consumer
Electronics, IEEE Transactions on 60.2 (2014): 242-250.
[13] Ou, Yen-Fu, et al. "Modeling the impact of frame rate on perceptual
quality of video." City 70.80 (2008): 90.
[14] S. van Kester, T. Xiao, R. E. Kooij, K. Brunnstr¨om, and O. K. Ahmed,
“Estimating the impact of single and multiple freezes on video quality,”
2011, vol. 7865, pp. 78650O–78650O–10.
[15] Ponomarenko, N., et al. "Color image database for evaluation of image
quality metrics." Multimedia Signal Processing, 2008 IEEE 10th
Workshop on. IEEE, 2008.