Please use this identifier to cite or link to this item: http://hdl.handle.net/10397/27953
Title: Building a framework for the probability ranking principle by a family of expected weighted rank
Authors: Dang, EKF
Wu, HC
Luk, RWP 
Wong, KF
Keywords: Optimization
Probability ranking principle
Issue Date: 2009
Publisher: Assoc Computing Machinery
Source: ACM transactions on information systems, 2009, v. 27, no. 4, 20 How to cite?
Journal: ACM Transactions on Information Systems 
Abstract: A new principles framework is presented for retrieval evaluation of ranked outputs. It applies decision theory to model relevance decision preferences and shows that the Probability Ranking Principle (PRP) specifies optimal ranking. It has two new components, namely a probabilistic evaluation model and a general measure of retrieval effectiveness. Its probabilities may be interpreted as subjective or objective ones. Its performance measure is the expected weighted rank which is the weighted average rank of a retrieval list. Starting from this measure, the expected forward rank and some existing retrieval effectiveness measures (e.g., top n precision and discounted cumulative gain) are instantiated using suitable weighting schemes after making certain assumptions. The significance of these instantiations is that the ranking prescribed by PRP is shown to be optimal simultaneously for all these existing performance measures. In addition, the optimal expected weighted rank may be used to normalize the expected weighted rank of retrieval systems for (summary) performance comparison (across different topics) between systems. The framework also extends PRP and our evaluation model to handle graded relevance, thereby generalizing the discussed, existing measures (e.g., top n precision) and probabilistic retrieval models for graded relevance.
URI: http://hdl.handle.net/10397/27953
DOI: 10.1145/1629096.1629098
Appears in Collections:Journal/Magazine Article

Access
View full-text via PolyU eLinks SFX Query
Show full item record

SCOPUSTM   
Citations

1
Last Week
0
Last month
0
Citations as of Feb 20, 2017

WEB OF SCIENCETM
Citations

1
Last Week
0
Last month
0
Citations as of Feb 21, 2017

Page view(s)

23
Last Week
1
Last month
Checked on Feb 26, 2017

Google ScholarTM

Check

Altmetric



Items in DSpace are protected by copyright, with all rights reserved, unless otherwise indicated.