Using Twitter dataset for social listening in Singapore

Abstract

As a highly urbanized nation, Singapore faces unique urban planning challenges due to its geographical attributes and demographics. These include optimizing land and transportation, enhancing quality of life, and preparing for pandemics. Quick responses and understanding of region-specific social voices are essential for effective policy-making and real-time insights into local dynamics. This work delves into analyzing social media data sourced from Twitter within the context of Singapore, forming a crucial component of a broader social listening initiative. Specifically, 96.7 million tweets from 2008 to 2023 were collected using Twitter’s free API, providing a decade’s worth of social data from Singapore. Alongside the Twitter data, we release a list of 10,357 places and property names with geographic coordinates, mapped to 332 subzones and 55 planning areas in Singapore. In this paper, we further present examples of locating methods that enable region-specific analysis of different urban zones, gathering information reflecting the attitudes of citizens associated with each estate. We showcase the practical application of the dataset through two distinct use cases: sentiment analysis on the prevalent issue of COVID-19 and bursty topic detection during the years 2020 and 2021. Deep learning-based methods are employed for the analysis: sentiment analysis using a zero-shot pretrained model and bursty topic analysis based on the biterm topic model. The experimental analysis demonstrates the efficacy of social listening, providing valuable insights for future city planning in other countries and cities. This work offers invaluable resources and methodologies for the research community, highlighting the potential of social media data in enhancing urban planning and policy-making. The data is realised at https://doi.org/10.21979/N9/PALUID .