Most existing convolutional neural network (CNN) based models designed for natural image quality assessment (IQA) employ image patches as training samples for data augmentation, and obtain final quality score by averaging all predicted scores of image patches. This brings two problems when applying these methods for screen content image (SCI) quality assessment. Firstly, SCI contains more complex content compared to natural image. As a result, qualities of SCI patches are different, and the subjective differential mean opinion score (DMOS) is not appropriate as qualities of all image patches. Secondly, the average score of image patches does not represent the quality of entire SCI since the human visual system (HVS) is sensitive to image patches containing texture and edge information. In this paper, we propose a novel quadratic optimized model based on the deep convolutional neural network (QODCNN) for full-reference (FR) and no-reference (NR) SCI quality assessment to overcome these two problems. The contribution of our algorithm can be concluded as follows: 1) Considering the characteristics of SCIs, a valid network architecture is designed for both NR and FR visual quality evaluation of SCIs, which makes the networks learn the feature differences for FR-IQA; 2) with the consideration of correlation between local quality and DMOS, a training data selection method is proposed to fine-tune the pre-trained model with valid SCI patches; 3) an adaptive pooling approach is employed to fuse patch quality to obtain image quality, owns strong noise robust and effects on both FR and NR IQA. Experimental results verify that our model outperforms both current no-reference and full-reference image quality assessment methods on the benchmark screen content image quality assessment database (SIQAD). Cross-database evaluation shows high generalization ability and high effectiveness of our model.

设计用于自然图像质量评估（IQA）的大多数现有的基于卷积神经网络（CNN）的模型都将图像斑块用作数据增强的训练样本，并通过平均所有图像斑块的预测得分来获得最终质量得分。当将这些方法应用于屏幕内容图像（SCI）质量评估时，会带来两个问题。首先，与自然图像相比，SCI包含更复杂的内容。结果，SCI补丁的质量不同，并且主观差分平均意见得分（DMOS）不适合作为所有图像补丁的质量。其次，图像补丁的平均分数不能代表整个SCI的质量，因为人类视觉系统（HVS）对包含纹理和边缘信息的图像补丁很敏感。在本文中，我们提出了一种基于深度卷积神经网络（QODCNN）的新型二次优化模型，用于全参考（FR）和无参考（NR）SCI质量评估，以克服这两个问题。该算法的贡献可以归纳为：1）考虑到SCI的特性，设计了一种有效的网络架构，用于SCI的NR和FR视觉质量评估，使网络了解FR-IQA的特征差异。2）考虑到局部质量与DMOS的相关性，提出了一种训练数据选择方法，以对带有有效SCI补丁的预训练模型进行微调。3）采用自适应池化方法融合补丁质量以获得图像质量，具有强大的噪声鲁棒性，并且对FR和NR IQA都有影响。实验结果证明，我们的模型优于基准屏幕内容图像质量评估数据库（SIQAD）上的当前无参考图像和全参考图像质量评估方法。跨数据库评估显示了我们模型的高泛化能力和高有效性。