Feature representation for mining evolution patterns in dynamic data

Abstract

Feature representation is an encoding process that projects the raw data into a discriminative latent space so as to extract the characteristics, properties, attributes, and underlying patterns from the data and embed them as features for supporting effective machine learning. It is the heart of AI, powering it to develop and train intelligent algorithms by supplying the useful information and discriminative features extracted from large quantities of high-quality data. In this thesis, I study an important yet overlooked problem of feature representation in dynamic data for capturing and embedding the evolution patterns, thus effectively facilitating the before-the-fact applications such as a prediction. Dynamic data refers to data that changes over time. It contains historical evolution patterns revealing how the data changed over time. Discovering and embedding these evolution patterns introduces additional and effective information to overcome data insufficiency issues in the before-the-fact applications, thereby leading to better performance. Existing studies either is based on differential equations or employ data-driven methods. The former one suffers from the high sensitivity of noise and uncertainty, while the latter merely focus on synchronous and/or period evolution patterns, overlooking the complexity of dynamics. Therefore, both of them fail to achieve satisfactory prediction performance. I aim to capture and embed the evolution from data with complex dynamics. Multivariate, multi-timescale, and asynchronous dynamics arise naturally in the world. Although these dynamics are very difficult to be fully captured due to the high stochastic and uncertainty, discovering the evolution patterns from them and embedding into the representation can effectively facilitate a prediction. To tackle the challenge of multi-variables, I devised a time-capturing dynamic graph embedding algorithm to learn the synchronous linkage evolution from the dynamic connection changes of every vertex over time. To deal with the challenge of synchronization, I propose a time-aware dynamic graph embedding algorithm to fully capture and embed the asynchronous structural evolutions in which the connections of vertices evolve at different times with variant evolution speed. Extensive experiments show that both algorithms achieved significant performance improvement over the state-of-the-art baselines in various graph mining applications. Lastly, a multi-timescale bag-of-regularity method is devised to extract students' learning regularity patterns from their multi-timescale dynamic learning behaviors, thereby achieving impressively high accuracy in early predicting academic at-risk students. I believe this thesis can serve as a solid step towards advanced knowledge discovery and representation in dynamic data.