Mobile sensing based human stress monitoring for smart health applications

Abstract

In the last decade, the advancement of the Internet of Things (IoT) enables communication and computation to appear anytime and anywhere. Numerous mobile devices and sensing equipment are interconnected that realize continuous data collection and analysis, which we refer to as mobile sensing. Based on the integration of artificial intelligence (AI) and IoT, health-related information could be extracted from data collected and provides real-time intervention automatically that has led to the emergence of smart health (s-Health) applications. However, the fundamental issues posed by the new paradigm of healthcare create barriers against the practical use of the s-Health system. One key challenge is the dynamic nature of human responses under different health conditions. It raises the need for calibration of the system to provide personalized diagnosis, making it difficult to scale for a large population and increases the cost of the initial setup. The other challenge stems from the widely used machine learning techniques that require a vast amount of training data to build the model. To unleash the full potential of s-Health, methods that can sense the general set of health indicators and efficiently predict the influences are necessary. This thesis focuses on the monitoring of stress as one major factor that affects our physical and mental health. We carry out a series of studies to 1) detect and recognize repetitive activities related to health conditions; 2) measure the symptoms and predict the impact of stress, using data collected from mobile devices. This thesis made three main contributions. The first contribution is the general approaches proposed for monitoring repetitive activities such as exercises, heartbeat, and respiration. We consider the repeated physical motion as patterns that occur consecutively in multivariate time series obtained from mobile devices. We proposed a multiple-length successive similar pattern detector (mSIMPAD) to detect repetitive activities using the sensor data. The mSIMPAD has barely any assumption regarding the target pattern and scales linearly, that can naturally adapt to different individuals and efficient enough to deploy on resource-constrained devices. On this basis, we proposed a scalable template extraction method (STEM) to locate and identify repeating patterns from multivariate time series. It substitutes the commonly used sliding-window-based technique and achieved more generic and efficient monitoring of repetitive activity. We also demonstrate the proposed approaches have a wide range of applications including a use case of respiration monitoring using wireless signals. The second contribution is the investigation of stress recognition using mobile devices. There is positive stress (eustress) and negative stress (distress) that affects our mental status and altering our behavior in different aspects. We developed a data collection platform using smartphones, wearable sensors, and computers and conducted an empirical study to examine the feasibility of stress recognition by exploiting the data collected. We found that physical and behavioral data help discriminate eustress from distress defined by its effect on performance. The third contribution is the work on predicting the impact of stress. We proposed a computational continuous stress performance prediction (CCSP) method that leverages domain knowledge to model the interaction between stress and cognitive performance over time. An experiment was designed and conducted in a rigorous laboratory environment. Data includes cognitive performance, physiological signals, and the concentrations of cortisol in saliva were collected during the experiment. Our computational model shows improvement in prediction performance on a small dataset with the aid of domain knowledge. It also shed light on the problem of cognitive performance prediction (increase, decrease, same) by estimating the physical stress symptoms with mobile devices.