Computer engineering scientist Jarek Nabrzyski and his team are developing Voice4PIMD, a personalized artificial intelligence system designed to interpret nonverbal communication for individuals with profound intellectual and multiple disabilities. By utilizing cameras and sensors, the technology monitors eye gaze, facial expressions, heart rate, and body movements. Unlike generic language models, this system requires caregivers to train the software on an individual’s unique behavioral baseline, ensuring that a specific gesture or sound is translated accurately according to the user's personal history.

The project aims to provide long-term continuity of care, a challenge often faced by families as primary caregivers change over time. Currently, the team is conducting pilot testing with families through LOGAN Community Resources in South Bend, Indiana, while simultaneously evaluating the system's potential efficacy for individuals living with autism or dementia. By focusing on this historically underserved population, Nabrzyski hopes to demonstrate that human-centered AI can provide dignity and agency to those who have long been excluded from traditional communication research.