Please use this identifier to cite or link to this item: http://hdl.handle.net/10397/34840
Title: Beyond bag-of-words : bigram-enhanced context-dependent term weights
Authors: Dang, EK
Luk, RW 
Allan, J
Keywords: Information retrieval
Issue Date: 2014
Publisher: Wiley
Source: Journal of the association for information science and technology, 2014, v. 65, no. 6, p. 1134-1148 How to cite?
Journal: Journal of the association for information science and technology
Abstract: While term independence is a widely held assumption in most of the established information retrieval approaches, it is clearly not true and various works in the past have investigated a relaxation of the assumption. One approach is to use n-grams in document representation instead of unigrams. However, the majority of early works on n-grams obtained only modest performance improvement. On the other hand, the use of information based on supporting terms or “contexts” of queries has been found to be promising. In particular, recent studies showed that using new context-dependent term weights improved the performance of relevance feedback (RF) retrieval compared with using traditional bag-of-words BM25 term weights. Calculation of the new term weights requires an estimation of the local probability of relevance of each query term occurrence. In previous studies, the estimation of this probability was based on unigrams that occur in the neighborhood of a query term. We explore an integration of the n-gram and context approaches by computing context-dependent term weights based on a mixture of unigrams and bigrams. Extensive experiments are performed using the title queries of the Text Retrieval Conference (TREC)-6, TREC-7, TREC-8, and TREC-2005 collections, for RF with relevance judgment of either the top 10 or top 20 documents of an initial retrieval. We identify some crucial elements needed in the use of bigrams in our methods, such as proper inverse document frequency (IDF) weighting of the bigrams and noise reduction by pruning bigrams with large document frequency values. We show that enhancing context-dependent term weights with bigrams is effective in further improving retrieval performance.
URI: http://hdl.handle.net/10397/34840
ISSN: 2330-1635 (print)
2330-1643 (online)
DOI: 10.1002/asi.23024
Appears in Collections:Journal/Magazine Article

Access
View full-text via PolyU eLinks SFX Query
Show full item record

WEB OF SCIENCETM
Citations

3
Last Week
0
Last month
Citations as of Mar 23, 2017

Page view(s)

24
Last Week
1
Last month
Checked on Mar 26, 2017

Google ScholarTM

Check

Altmetric



Items in DSpace are protected by copyright, with all rights reserved, unless otherwise indicated.