当前位置: X-MOL 学术J. Visual Commun. Image Represent. › 论文详情
Our official English website, www.x-mol.net, welcomes your feedback! (Note: you will need to create a separate account there.)
Learning to predict the quality of distorted-then-compressed images via a deep neural network
Journal of Visual Communication and Image Representation ( IF 2.6 ) Pub Date : 2021-01-01 , DOI: 10.1016/j.jvcir.2020.103004
Bowen Li , Meng Tian , Weixia Zhang , Hongtai Yao , Xianpei Wang

Being captured by amateur photographers, reciprocally propagated through multimedia pipelines, and compressed with different levels, real-world images usually suffer from a wide variety of hybrid distortions. Faced with this scenario, full-reference (FR) image quality assessment (IQA) algorithms can not deliver promising predictions due to the inferior references. Meanwhile, existing no-reference (NR) IQA algorithms remain limited in their efficacy to deal with different distortion types. To address this obstacle, we explore a NR-IQA metric by predicting the perceptual quality of distorted-then-compressed images using a deep neural network (DNN). First, we propose a novel two-stream DNN to handle both authentic distortions and synthetic compressions and adopt effective strategies to pre-train the two branches of the network. Specifically, we transfer the knowledge learned from in-the-wild images to account for authentic distortions by utilizing a pre-trained deep convolutional neural network (CNN) to provide meaningful initializations. Meanwhile, we build a CNN for synthetic compressions and pre-train it on a dataset including synthetic compressed images. Subsequently, we bilinearly pool these two sets of features as the image representation. The overall network is fine-tuned on an elaborately-designed auxiliary dataset, which is annotated by a reliable objective quality metric. Furthermore, we integrate the output of the authentic-distortion-aware branch with that of the overall network following a two-step prediction manner to boost the prediction performance, which can be applied in the distorted-then-compressed scenario when the reference image is available. Extensive experimental results on several databases especially on the LIVE Wild Compressed Picture Quality Database show that the proposed method achieves state-of-the-art performance with good generalizability and moderate computational complexity.



中文翻译:

通过深度神经网络学习预测变形然后压缩的图像的质量

现实世界的图像由业余摄影师捕获,通过多媒体管道相互传播,并经过不同级别的压缩,因此通常会遭受各种各样的混合失真。面对这种情况,由于参考质量较低,全参考(FR)图像质量评估(IQA)算法无法提供有前途的预测。同时,现有的无参考(NR)IQA算法在处理不同失真类型方面的功效仍然受到限制。为了解决这一障碍,我们通过使用深度神经网络(DNN)预测扭曲然后压缩的图像的感知质量来探索NR-IQA指标。首先,我们提出了一种新颖的两流DNN,以处理真实的失真和合成压缩,并采用有效的策略来预训练网络的两个分支。具体来说,我们通过使用预先训练的深度卷积神经网络(CNN)来提供有意义的初始化,从而将从野生图像中获得的知识转移到真实的失真中。同时,我们构建了用于合成压缩的CNN,并在包含合成压缩图像的数据集上对其进行了预训练。随后,我们将这两组特征双线性合并为图像表示。整个网络在精心设计的辅助数据集上进行了微调,并通过可靠的客观质量指标进行了注释。此外,我们采用两步预测的方式将真实失真感知分支的输出与整个网络的输出进行集成,以提高预测性能,该方法可以应用于参考图像为失真然后压缩的场景。可用的。

更新日期:2021-02-28
down
wechat
bug