A real-time ASL recognition system using Leap Motion sensors

Abstract

It is always challenging for deaf and speech-impaired people to communicate with non-sign language users. A real-time sign language recognition system using 3D motion sensors could lower the aforementioned communication barrier. However, most existing gesture recognition systems are adopting a single sensor framework, whose performance is susceptible to occlusions. In this paper, we proposed a real-time multi-sensor recognition system for American sign language (ASL). Data collected from Leap Motion sensors are fused using multiple sensors data fusion (MSDF) and the recognition is performed using hidden Markov models (HMM). Experimental results demonstrate that the proposed system can deliver higher recognition accuracy over single-sensor systems. Due to its low implementation cost and higher accuracy, the proposed system can be widely deployed and bring conveniences to sign language users.