Picture-wise just noticeable difference prediction model for JPEG image quality assessment
Abstract
Introduction/purpose: The paper presents interesting research related to the performance analysis of the picture-wise just noticeable difference (JND) prediction model and its application in the quality assessment of images with JPEG compression.
Methods: The performance analysis of the JND model was conducted in an indirect way by using the publicly available results of subject-rated image datasets with the separation of images into two classes (above and below the threshold of visible differences). In the performance analysis of the JND prediction model and image quality assessment, five image datasets were used, four of which come from the visible wavelength range, and one dataset is intended for remote sensing and surveillance with images from the infrared part of the electromagnetic spectrum.
Results: The paper shows that using a picture-wise JND model, subjective image quality assessment scores can be estimated with better accuracy, leading to significant performance improvements of the traditional peak signal-to-noise ratio (PSNR). The gain achieved by introducing the picture-wise JND model in the objective assessment depends on the chosen dataset and the results of the initial simple to compute PSNR measure, and it was obtained on all five datasets. The mean linear correlation coefficient (for five datasets) between subjective and PSNR objective quality estimates increased from 74% (traditional PSNR) to 90% (picture-wise JND PSNR).
Conclusion: Further improvement of the JND-based objective measure can be obtained by improving the picture-wise model of JND prediction.
References
Ahar, A., Mahmaoudpour, S., Van Wallendael, G., Paridaens, T., Lambert, P. & Schelkens, P. 2018. A just noticeable difference subjective test for high dynamic range images. In: Proceedings of Tenth International Conference on Quality of Multimedia Experience (QoMEX), Cagliari, Italy, pp.1-6, May 29-June 1. Available at: https://doi.org/10.1109/QoMEX.2018.8463429
Bondžulić, B.P., Pavlović, B.Z., Andrić, M.S. & Petrović, V.S. 2017. Comments on objective quality assessment of JPEG images with visible differences. In: Proceedings of 13th International Conference on Advanced Technologies, Systems and Services in Telecommunications (TELSIKS), Niš, Serbia, pp.455-458, October 18-20. Available at: https://doi.org/10.1109/TELSKS.2017.8246323
Bondžulić, B., Pavlović, B. & Petrović, V. 2018. Performance analysis of full-reference objective image and video quality assessment metrics. Vojnotehnički glasnik/Military Technical Courier, 66(2), pp.322-350. Available at: https://doi.org/10.5937/vojtehg66-12708
Bondzulic, B., Pavlovic, B., Petrovic, V. & Andric, M. 2016. Performance of peak signal-to-noise ratio quality assessment in video streaming with packet losses. Electronics Letters, 52(6), pp.454-456. Available at: https://doi.org/10.1049/el.2015.3784
Bondžulić, B., Stojanović, N., Petrović, V. & Zelmati, O. 2020. Using objective image quality assessment metrics in detection just noticeable differences of JPEG images. In: Proceedings of XXVI Conference and Exhibition YU INFO 2020, Kopaonik, Serbia, pp.203-208, March 8-11 [online]. Available at: http://www.yuinfo.org/ZBORNIK_YU_INFO_2020.pdf (in Serbian) [Accessed: 1 November 2021].
Bondžulić, B., Stojanović, N., Petrović, V., Pavlović, B. & Miličević, Z. 2021. Efficient prediction of the first just noticeable difference point for JPEG compressed images. Acta Polytechnica Hungarica, 18(8), pp.201-220. Available at: https://doi.org/10.12700/APH.18.8.2021.8.11
Egiazarian, K., Astola, J., Ponomarenko, N., Lukin, V., Battisti, F. & Carli, M. 2006. Two new full-reference quality metrics based on HVS. In: Proceedings of 2nd International Workshop on Video Processing and Quality Metrics for Consumer Electronics – VPQM, Scottsdale, Arizona, USA, pp.1-4, January 22-24.
Fan, C., Lin, H., Hosu, V., Zhang, Y., Jiang, Q., Hamzaoui, R. & Saupe, D. 2019. SUR-Net: Predicting the satisfied user ratio curve for image compression with deep learning. In: Proceedings of 11th International Conference on Quality of Multimedia Experience, Berlin, Germany, pp.1-6, June 5-7. Available at: https://doi.org/10.1109/QoMEX.2019.8743204
Gonzalez, R.C. & Woods, R.E. 2018. Digital image processing, 4th Edition. London: Pearson Education, Inc. ISBN-13: 9780133356724.
Huang, J., Feng, H., Xu, Z., Li, Q. & Chen, Y. 2018. A robust deblurring algorithm for noisy images with just noticeable blur. Optik – International Journal for Light and Electron Optics, 168, pp.577-589. Available at: https://doi.org/10.1016/j.ijleo.2018.04.052
Hudson, G., Leger, A., Niss, B. & Sebestyen, I. 2017. JPEG at 25: Still going strong. IEEE MultiMedia, 24(2), pp.96-103. Available at: https://doi.org/10.1109/MMUL.2017.38
Hudson, G., Leger, A., Niss, B., Sebestyen, I. & Vaaben, J. 2018. JPEG-1 standard 25 years: Past, present, and future reasons for a success. Journal of Electronic Imaging, 27(4), art.number:040901. Available at: https://doi.org/10.1117/1.JEI.27.4.040901
Huynh-Thu, Q. & Ghanbari, M. 2008. Scope of validity of PSNR in image/video quality assessment. Electronics Letters, 44(13), pp.800-801. Available at: https://doi.org/10.1049/el:20080522
-ITU-T (Telecommunication Standardization Sector of ITU). 2004. Objective perceptual assessment of video quality: Full reference television. [online]. Available at: https://www.itu.int/ITU-T/studygroups/com09/docs/tutorial_opavc.pdf [Accessed: 1 November 2021].
Jin, L., Lin, J.Y., Hu, S., Wang, H., Wang, P., Katsavounidis, I., Aaron, A. & Kuo, C.-C. J. 2016. Statistical study on perceived JPEG image quality via MCL-JCI dataset construction and analysis. In: Proceedings of IS&T International Symposium on Electronic Imaging – Image Quality and System Performance XIII, San Francisco, CA, USA, art.number:IQSP-222, February 14-18. Available at: https://doi.org/10.2352/ISSN.2470-1173.2016.13.IQSP-222
Larson, E.C. & Chandler, D.M. 2010. Most apparent distortion: Full-reference image quality assessment and the role of strategy. Journal of Electronic Imaging, 19(1), art.number:011006. Available at: https://doi.org/10.1117/1.3267105
Li, H., Jenadeleh, M., Chen, G., Reips, U-D., Hamzaoui, R. & Saupe, D. 2020. Subjective assessment of global picture-wise just noticeable difference. In: Proceedings of IEEE International Conference on Multimedia & Expo Workshops, London, UK, pp.1-6, July 6-10. Available at: https://doi.org/10.1109/ICMEW46912.2020.9106058
Lin, H., Hosu, V., Fan, C., Zhang, Y., Mu, Y., Hamzaoui, R. & Saupe, D. 2020. SUR-FeatNet: Predicting the satisfied user ratio curve for image compression with deep feature learning. Quality and User Experience, 5(5), pp.1-23. Available at: https://doi.org/10.1007/s41233-020-00034-1
Liu, X., Chen, Z., Wang, X., Jiang, J. & Kwong, S. 2018. JND-Pano: database for just noticeable difference of JPEG compressed panoramic images. In: Hong, R., Cheng, W.H., Yamasaki, T., Wang, M. & Ngo, C.W. (Eds.) Advances in Multimedia Information Processing. PCM 2018. Lecture Notes in Computer Science, 11164, pp.458-468. Springer, Cham. Available at: https://doi.org/10.1007/978-3-030-00776-8_42
Liu, H., Zhang, Y., Zhang, H., Fan, C., Kwong, S., Kuo, C-C.J. & Fan, X. 2020. Deep learning based picture-wise just noticeable prediction model for image compression. IEEE Transactions on Image Processing, 29, pp.641-656. Available at: https://doi.org/10.1109/TIP.2019.2933743
Lu, G., Zhang, X., Ouyang, W., Chen, L., Gao, Z. & Xu, G. 2021. An end-to-end learning framework for video compression. IEEE Transactions on Pattern Analysis and Machine Intelligence, 43(10), pp.3292-3308. Available at: https://doi.org/10.1109/TPAMI.2020.2988453
Merrouche, S., Bondžulić, B., Andrić, M. & Bujaković, D. 2018. Description and analysis of infrared image database – Subjective and objective image quality assessment. In: Proceedings of 8th International Scientific Conference on Defensive Technologies – OTEH, Belgrade, Serbia, pp.307-313, October 11-12 [online]. Available at: http://www.vti.mod.gov.rs/oteh18/elementi/rad/058.htm [Accessed: 1 November 2021].
Pennebaker, W.B. & Mitchell, J.L. 1993. JPEG: Still image data compression standard. New York: Van Nostrand Reinhold Publishers. ISBN: 0-442-01272-1.
Ponomarenko, N., Lukin, V., Astola, J. & Egiazarian, K. 2015. Analysis of HVS-metrics’ properties using color image database TID2013. In: Battiato, S., Blanc-Talon, J., Gallo, G., Philips, W., Popescu, D. & Scheunders, P. (Eds.) Advanced Concepts for Intelligent Vision Systems. ACIVS 2015. Lecture Notes in Computer Science, 9386, pp.613-624. Springer, Cham. Available at: https://doi.org/10.1007/978-3-319-25903-1_53
Ponomarenko, N., Silvestri, F., Egiazarian, K., Carli, M., Astola, J. & Lukin, V. 2007. On between-coefficient contrast masking of DCT basis functions. In: Proceedings of 3rd International Workshop on Video Processing and Quality Metrics for Consumer Electronics – VPQM, Scottsdale, Arizona, USA, pp.1-4, January 25-26 [online]. Available at: http://ponomarenko.info/vpqm07_p.pdf [Accessed: 1 November 2021].
Seo, S., Ki, S. & Kim, M. 2021. A novel just-noticeable-difference-based saliency-channnel attention residual network for full-reference image quality predictions. IEEE Transactions on Circuits and Systems for Video Technology, 31(7), pp.2602-2616. Available at: https://doi.org/10.1109/TCSVT.2020.3030895
Sheikh, H.R., Sabir M.F. & Bovik, A.C. 2006. A statistical evaluation of recent full reference image quality assessment algorithms. IEEE Transactions on Image Processing, 15(11), pp.3440-3451. Available at: https://doi.org/10.1109/TIP.2006.881959
Tian, T., Wang, H., Zuo, L., Kuo, C-C.J. & Kwong, S. 2020. Just noticeable difference level prediction for perceptual image compression. IEEE Transactions on Broadcasting, 66(3), pp.690-700. Available at: https://doi.org/10.1109/TBC.2020.2977542
Toprak, S. & Yalman, Y. 2017. A new full-reference image quality metric based on just noticeable difference. Computer Standards & Interfaces, 50, pp.18-25. Available at: https://doi.org/10.1016/j.csi.2016.08.003
Wallace, G.K. 1992. The JPEG still picture compression standard. IEEE Transactions on Consumer Electronics, 38(1), pp.18-34. Available at: https://doi.org/10.1109/30.125072
Wang, Z., Tran, T-H., Muthappa, P.K. & Simon, S. 2019. A JND-based pixel-domain algorithm and hardware architecture for perceptual image coding. Journal of Imaging, 5(50), pp.1-29. Available at: https://doi.org/10.3390/jimaging5050050
Yu, H. & Winkler, S. 2013. Image complexity and spatial information. In: Proceedings of 5th International Workshop on Quality of Multimedia Experience – QoMEX, Klagenfurt am Worthersee, Austria, pp.12-17, July 3-5. Available at: https://doi.org/10.1109/QoMEX.2013.6603194
Zarić, A., Tatalović, N., Brajković, N., Hlevnjak, H., Lončarić, M., Dumić, E. & Grgić, S. 2012. VCL@FER image quality assessment database. Automatika, 53(4), pp.344-354. Available at: https://doi.org/10.7305/automatika.53-4.241
Proposed Creative Commons Copyright Notices
Proposed Policy for Military Technical Courier (Journals That Offer Open Access)
Authors who publish with this journal agree to the following terms:
Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under a Creative Commons Attribution License that allows others to share the work with an acknowledgement of the work's authorship and initial publication in this journal.
- Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgement of its initial publication in this journal.
- Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (See The Effect of Open Access).