Gesture recognition for engaging spatial experiences in healthcare : co-design of intelligent interactive illuminative textiles

Abstract

The integration of artificial intelligence (AI) into textile design enhances functionality, automation, and user interaction. While gesture recognition has been explored in smart textiles, contactless interactive systems for healthcare remain underdeveloped. This study presents a human-centered co-design approach to the development of an AI-integrated gesture recognition system embedded in illuminative textile wall panels, aimed at enhancing spatial engagement in healthcare environments. The research was conducted in three key stages. First, a co-design workshop was conducted to explore user preferences in textile materials, graphic design, and gesture interaction. Second, intelligent illuminative textiles were developed by knitting polymeric optical fiber into base wool yarns to enable illumination. A camera was embedded and integrated with a computer vision-based deep learning model for detecting landmarks on the hands, shoulders, and head. The recognized gestures and body movements triggered specific pre-programmed color changes on the textile surface through edge-integrated light-emitting diodes. Finally, a prototype was fabricated and installed in a government-established District Health Centre in Hong Kong to support physical activity and rehabilitation for elderly users. Semi-structured interviews with stakeholders – including co-designers, users, and occupational therapists – were conducted to evaluate usability and inform design refinements. Stakeholders reported high levels of satisfaction, emphasizing the system’s ability to enhance community connection, therapeutic engagement, intuitive usability, and compelling visual feedback. These findings suggest that AI-driven interactive textiles present promising opportunities for rehabilitation, therapeutic environments, and the promotion of elderly well-being. Graphical abstract: [Figure not available: see fulltext.]