Please use this identifier to cite or link to this item:
PIRA download icon_1.1View/Download Full Text
Title: Search for K : assessing five topic-modeling approaches to 120,000 Canadian articles
Authors: Fu, Q
Zhuang, Y
Gu, J
Zhu, Y
Qin, H 
Guo, X 
Issue Date: 2019
Source: 2019 IEEE International Conference on Big Data (Big Data), Los Angeles, CA, USA, 2019, p. 3640-3647
Abstract: Topic modeling has been an important field in natural language processing (NLP) and recently witnessed great methodological advances. Yet, the development of topic modeling is still, if not increasingly, challenged by two critical issues. First, despite intense efforts toward nonparametric/post-training methods, the search for the optimal number of topics K remains a fundamental question in topic modeling and warrants input from domain experts. Second, with the development of more sophisticated models, topic modeling is now ironically been treated as a black box and it becomes increasingly difficult to tell how research findings are informed by data, model specifications, or inference algorithms. Based on about 120,000 newspaper articles retrieved from three major Canadian newspapers (Globe and Mail, Toronto Star, and National Post) since 1977, we employ five methods with different model specifications and inference algorithms (Latent Semantic Analysis, Latent Dirichlet Allocation, Principal Component Analysis, Factor Analysis, Nonnegative Matrix Factorization) to identify discussion topics. The optimal topics are then assessed using three measures: coherence statistics, held-out likelihood (loss), and graph-based dimensionality selection. Mixed findings from this research complement advances in topic modeling and provide insights into the choice of optimal topics in social science research.
Keywords: Topic modeling
Natural language processing
Social science
Optimal number of topics
Publisher: Institute of Electrical and Electronics Engineers
ISBN: 978-1-7281-0858-2 (Electronic)
978-1-7281-0857-5 (USB)
978-1-7281-0859-9 (Print on Demand(PoD))
DOI: 10.1109/BigData47090.2019.9006160
Rights: © 2019 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.
The following publication Q. Fu, Y. Zhuang, J. Gu, Y. Zhu, H. Qin and X. Guo, "Search for K: Assessing Five Topic-Modeling Approaches to 120,000 Canadian Articles," 2019 IEEE International Conference on Big Data (Big Data), Los Angeles, CA, USA, 2019, pp. 3640-3647 is available at
Appears in Collections:Conference Paper

Files in This Item:
File Description SizeFormat 
Fu_Search_for_K.pdfPre-Published version985.22 kBAdobe PDFView/Open
Open Access Information
Status open access
File Version Final Accepted Manuscript
View full-text via PolyU eLinks SFX Query
Show full item record

Page views

Last Week
Last month
Citations as of Sep 24, 2023


Citations as of Sep 24, 2023


Citations as of Sep 21, 2023

Google ScholarTM



Items in DSpace are protected by copyright, with all rights reserved, unless otherwise indicated.