Please use this identifier to cite or link to this item:
Title: Waterloo exploration database : new challenges for image quality assessment models
Authors: Ma, K
Duanmu, Z
Wu, Q
Wang, Z
Yong, H 
Li, H
Zhang, L 
Keywords: Discriminable image pair
Image database
Image quality assessment
Listwise ranking consistency
Mean opinion score
Pairwise preference consistency
Issue Date: 2017
Publisher: Institute of Electrical and Electronics Engineers
Source: IEEE transactions on image processing, 2017, v. 26, no. 2, p. 1004-1016 How to cite?
Journal: IEEE transactions on image processing 
Abstract: The great content diversity of real-world digital images poses a grand challenge to image quality assessment (IQA) models, which are traditionally designed and validated on a handful of commonly used IQA databases with very limited content variation. To test the generalization capability and to facilitate the wide usage of IQA techniques in real-world applications, we establish a large-scale database named the Waterloo Exploration Database, which in its current state contains 4744 pristine natural images and 94 880 distorted images created from them. Instead of collecting the mean opinion score for each image via subjective testing, which is extremely difficult if not impossible, we present three alternative test criteria to evaluate the performance of IQA models, namely, the pristine/distorted image discriminability test, the listwise ranking consistency test, and the pairwise preference consistency test (P-test). We compare 20 well-known IQA models using the proposed criteria, which not only provide a stronger test in a more challenging testing environment for existing models, but also demonstrate the additional benefits of using the proposed database. For example, in the P-test, even for the best performing no-reference IQA model, more than 6 million failure cases against the model are 'discovered' automatically out of over 1 billion test pairs. Furthermore, we discuss how the new database may be exploited using innovative approaches in the future, to reveal the weaknesses of existing IQA models, to provide insights on how to improve the models, and to shed light on how the next-generation IQA models may be developed.
ISSN: 1057-7149
EISSN: 1941-0042
DOI: 10.1109/TIP.2016.2631888
Appears in Collections:Journal/Magazine Article

View full-text via PolyU eLinks SFX Query
Show full item record


Last Week
Last month
Citations as of Aug 13, 2018


Last Week
Last month
Citations as of Aug 11, 2018

Page view(s)

Last Week
Last month
Citations as of Aug 13, 2018

Google ScholarTM



Items in DSpace are protected by copyright, with all rights reserved, unless otherwise indicated.