ElderPlay: Supporting Age-Inclusive Gameplay for Older Adults via Real-Time Gesture-to-Controller Translation

Playing video games can enhance older adults’ well-being and social connections. However, most mainstream games rely on button-based controls that require fine motor skills, limiting accessibility. We present ElderPlay, a real-time game input translation system that enables older adults to play unmodified commercial games using intuitive, motion-based interaction. We first conducted a gesture elicitation study to derive user-defined gestures grounded in everyday experiences, which informed the design of a proof-of-concept system translating gestures into controller inputs. We then evaluated ElderPlay with two commercial Nintendo Switch games. Results show that gesture-based interaction improves enjoyment, perceived physical engagement, and performance. Rather than replacing controllers, our findings highlight the effectiveness of hybrid interaction, where gesture and controller inputs support different gameplay actions. We discuss implications for context-dependent and inclusive game interaction design.