From unidimensional sequence prediction to multidimensional spatial prediction with deep neural networks

Abstract

Forecasting future values of sequential data is one of the most widely applied tasks but with long-standing challenges. In prediction, a hypothesized causal relationship between the input variables and the outcome variables is modelled and estimated from large historical data in order to forecast the next term of a sequence. The booming of artificial intelligence (AI) over the past decade makes machine learning / deep learning the focus of research in both industry and academia. Deep neural networks have achieved great successes in many fields, such as computer vision, natural language processing, speech recognition and recommender system, and they are extensively studied and applied for time series prediction as well. In this thesis, I focus on sequence prediction and spatial prediction using the state-of-the-art deep learning approaches. Compared to traditional statistical-based prediction methods, deep neural networks with multi-layer hidden units have been proven to be more effective and efficient in learning the abstract representations of input features and modeling the nonlinear causal relationship between the historical inputs and the forecasted future values. This thesis is comprised of two parts. In the first part, I retrospectively review conventional statistical-based forecasting methods. Then, I summarize recent advances of deep neural networks for making one-dimensional sequence prediction and two-dimensional spatial prediction. In the second part, I investigate how to build practical applications using deep learning models. Concretely, I first work on the multimodal solar irradiance prediction, then extend the experience gained in 1D space to higher dimensions. I study the problems of 2D spatial data prediction for wireless coverage estimation and the 2D spatio-temporal representation learning for audio mixture separation.