The Impact of Temporal Impairment on Quality of Experience (QoE) in Video Streaming: A No Reference (NR) Subjective and Objective Study
Live video streaming is one of the most widely used
service among end users, yet it is a big challenge for the network
operators in terms of quality. The only way to provide excellent
Quality of Experience (QoE) to the end users is continuous
monitoring of live video streaming. For this purpose, there are several
objective algorithms available that monitor the quality of the video in
a live stream. Subjective tests play a very important role in fine
tuning the results of objective algorithms. As human perception is
considered to be the most reliable source for assessing the quality of a
video stream subjective tests are conducted in order to develop more
reliable objective algorithms. Temporal impairments in a live video
stream can have a negative impact on the end users. In this paper we
have conducted subjective evaluation tests on a set of video
sequences containing temporal impairment known as frame freezing.
Frame Freezing is considered as a transmission error as well as a
hardware error which can result in loss of video frames on the
reception side of a transmission system. In our subjective tests, we
have performed tests on videos that contain a single freezing event
and also for videos that contain multiple freezing events. We have
recorded our subjective test results for all the videos in order to give a
comparison on the available No Reference (NR) objective
algorithms. Finally, we have shown the performance of no reference
algorithms used for objective evaluation of videos and suggested the
algorithm that works better. The outcome of this study shows the
importance of QoE and its effect on human perception. The results
for the subjective evaluation can serve the purpose for validating
objective algorithms.
[1] Forecast, Cisco VNI. "Cisco Visual Networking Index: Global Mobile
data Traffic Forecast Update 2009-2014." Cisco Public Information,
February 9 (2010).
[2] Video Quality Experts Group. "Final report from the Video Quality
Experts Group on the validation of objective models of video quality
assessment, Phase II(FR_TV2).", 2003.
[3] Cisco, I. "Cisco visual networking index: Forecast and methodology,
2011–2016." CISCO White Paper (2012): 2011-2016.
[4] Usman, Muhammad Arslan, Muhammad Rehan Usman, and Soo Young
Shin. "Performance Analysis of a No-Reference Temporal Quality
Assessment Metric for Videos Impaired by Frame Freezing Artefacts."
[5] “Subjective video quality assessment methods for multimedia
applications,” September 1999, iTU-T, Recommendation ITU-R P910.
[6] “ITU-R Radio communication Sector of ITU, Recommendation ITU-R
BT.500-12,” 2009.
[7] Wang, Zhou, et al. "Image quality assessment: from error visibility to
structural similarity." Image Processing, IEEE Transactions on 13.4
(2004): 600-612.
[8] Stephen Wolf, “A no reference (nr) and reduced reference (rr) metric for
detecting dropped video frames,” in Second International Workshop on
Video Processing and Quality Metrics for Consumer Electronics
(VPQM), 2009.
[9] Quan Huynh-Thu and M. Ghanbari, “No-reference temporal quality
metric for video impaired by frame freezing artefacts,” in IEEE
International Conference on Image Processing (ICIP), 2009, pp. 2221–
2224.
[10] Borer, Silvio. "A model of jerkiness for temporal impairments in video
transmission." Quality of Multimedia Experience (QoMEX), 2010
Second International Workshop on. IEEE, 2010.
[11] Shahid, Muhammad, et al. "Subjective quality assessment of H.
264/AVC encoded low resolution videos." Image and Signal Processing
(CISP), 2012 5th International Congress on. IEEE, 2012.
[12] Nightingale, James, et al. "The impact of network impairment on quality
of experience (QoE) in H. 265/HEVC video streaming." Consumer
Electronics, IEEE Transactions on 60.2 (2014): 242-250.
[13] Ou, Yen-Fu, et al. "Modeling the impact of frame rate on perceptual
quality of video." City 70.80 (2008): 90.
[14] S. van Kester, T. Xiao, R. E. Kooij, K. Brunnstr¨om, and O. K. Ahmed,
“Estimating the impact of single and multiple freezes on video quality,”
2011, vol. 7865, pp. 78650O–78650O–10.
[15] Ponomarenko, N., et al. "Color image database for evaluation of image
quality metrics." Multimedia Signal Processing, 2008 IEEE 10th
Workshop on. IEEE, 2008.
[1] Forecast, Cisco VNI. "Cisco Visual Networking Index: Global Mobile
data Traffic Forecast Update 2009-2014." Cisco Public Information,
February 9 (2010).
[2] Video Quality Experts Group. "Final report from the Video Quality
Experts Group on the validation of objective models of video quality
assessment, Phase II(FR_TV2).", 2003.
[3] Cisco, I. "Cisco visual networking index: Forecast and methodology,
2011–2016." CISCO White Paper (2012): 2011-2016.
[4] Usman, Muhammad Arslan, Muhammad Rehan Usman, and Soo Young
Shin. "Performance Analysis of a No-Reference Temporal Quality
Assessment Metric for Videos Impaired by Frame Freezing Artefacts."
[5] “Subjective video quality assessment methods for multimedia
applications,” September 1999, iTU-T, Recommendation ITU-R P910.
[6] “ITU-R Radio communication Sector of ITU, Recommendation ITU-R
BT.500-12,” 2009.
[7] Wang, Zhou, et al. "Image quality assessment: from error visibility to
structural similarity." Image Processing, IEEE Transactions on 13.4
(2004): 600-612.
[8] Stephen Wolf, “A no reference (nr) and reduced reference (rr) metric for
detecting dropped video frames,” in Second International Workshop on
Video Processing and Quality Metrics for Consumer Electronics
(VPQM), 2009.
[9] Quan Huynh-Thu and M. Ghanbari, “No-reference temporal quality
metric for video impaired by frame freezing artefacts,” in IEEE
International Conference on Image Processing (ICIP), 2009, pp. 2221–
2224.
[10] Borer, Silvio. "A model of jerkiness for temporal impairments in video
transmission." Quality of Multimedia Experience (QoMEX), 2010
Second International Workshop on. IEEE, 2010.
[11] Shahid, Muhammad, et al. "Subjective quality assessment of H.
264/AVC encoded low resolution videos." Image and Signal Processing
(CISP), 2012 5th International Congress on. IEEE, 2012.
[12] Nightingale, James, et al. "The impact of network impairment on quality
of experience (QoE) in H. 265/HEVC video streaming." Consumer
Electronics, IEEE Transactions on 60.2 (2014): 242-250.
[13] Ou, Yen-Fu, et al. "Modeling the impact of frame rate on perceptual
quality of video." City 70.80 (2008): 90.
[14] S. van Kester, T. Xiao, R. E. Kooij, K. Brunnstr¨om, and O. K. Ahmed,
“Estimating the impact of single and multiple freezes on video quality,”
2011, vol. 7865, pp. 78650O–78650O–10.
[15] Ponomarenko, N., et al. "Color image database for evaluation of image
quality metrics." Multimedia Signal Processing, 2008 IEEE 10th
Workshop on. IEEE, 2008.
@article{"International Journal of Information, Control and Computer Sciences:70594", author = "Muhammad Arslan Usman and Muhammad Rehan Usman and Soo Young Shin", title = "The Impact of Temporal Impairment on Quality of Experience (QoE) in Video Streaming: A No Reference (NR) Subjective and Objective Study", abstract = "Live video streaming is one of the most widely used
service among end users, yet it is a big challenge for the network
operators in terms of quality. The only way to provide excellent
Quality of Experience (QoE) to the end users is continuous
monitoring of live video streaming. For this purpose, there are several
objective algorithms available that monitor the quality of the video in
a live stream. Subjective tests play a very important role in fine
tuning the results of objective algorithms. As human perception is
considered to be the most reliable source for assessing the quality of a
video stream subjective tests are conducted in order to develop more
reliable objective algorithms. Temporal impairments in a live video
stream can have a negative impact on the end users. In this paper we
have conducted subjective evaluation tests on a set of video
sequences containing temporal impairment known as frame freezing.
Frame Freezing is considered as a transmission error as well as a
hardware error which can result in loss of video frames on the
reception side of a transmission system. In our subjective tests, we
have performed tests on videos that contain a single freezing event
and also for videos that contain multiple freezing events. We have
recorded our subjective test results for all the videos in order to give a
comparison on the available No Reference (NR) objective
algorithms. Finally, we have shown the performance of no reference
algorithms used for objective evaluation of videos and suggested the
algorithm that works better. The outcome of this study shows the
importance of QoE and its effect on human perception. The results
for the subjective evaluation can serve the purpose for validating
objective algorithms.", keywords = "Objective evaluation, subjective evaluation, quality
of experience (QoE), video quality assessment (VQA).", volume = "9", number = "8", pages = "1906-8", }