Deep learning-based human activity real-time recognition for pedestrian navigation

Abstract

Several pedestrian navigation solutions have been proposed to date, and most of them are based on smartphones. Real-time recognition of pedestrian mode and smartphone posture is a key issue in navigation. Traditional ML (Machine Learning) classification methods have drawbacks, such as insufficient recognition accuracy and poor timing. This paper presents a real-time recognition scheme for comprehensive human activities, and this scheme combines deep learning algorithms and MEMS (Micro-Electro-Mechanical System) sensors' measurements. In this study, we performed four main experiments, namely pedestrian motion mode recognition, smartphone posture recognition, real-time comprehensive pedestrian activity recognition, and pedestrian navigation. In the procedure of recognition, we designed and trained deep learning models using LSTM (Long Short-Term Memory) and CNN (Convolutional Neural Network) networks based on Tensorflow framework. The accuracy of traditional ML classification methods was also used for comparison. Test results show that the accuracy of motion mode recognition was improved from <mml:semantics>89.9%</mml:semantics>, which was the highest accuracy and obtained by SVM (Support Vector Machine), to <mml:semantics>90.74%</mml:semantics> (LSTM) and <mml:semantics>91.92%</mml:semantics> (CNN); the accuracy of smartphone posture recognition was improved from <mml:semantics>81.60%</mml:semantics>, which is the highest accuracy and obtained by NN (Neural Network), to <mml:semantics>93.69%</mml:semantics> (LSTM) and <mml:semantics>95.55%</mml:semantics> (CNN). We give a model transformation procedure based on the trained CNN network model, and then obtain the converted <mml:semantics>.tflite</mml:semantics> model, which can be run in Android devices for real-time recognition. Real-time recognition experiments were performed in multiple scenes, a recognition model trained by the CNN network was deployed in a Huawei Mate20 smartphone, and the five most used pedestrian activities were designed and verified. The overall accuracy was up to <mml:semantics>89.39%</mml:semantics>. Overall, the improvement of recognition capability based on deep learning algorithms was significant. Therefore, the solution was helpful to recognize comprehensive pedestrian activities during navigation. On the basis of the trained model, a navigation test was performed; mean bias was reduced by more than 1.1 m. Accordingly, the positioning accuracy was improved obviously, which is meaningful to apply DL in the area of pedestrian navigation to make improvements.