SIGN4HEALTH – Cross-Modal AI for Inclusive Digital Health

SIGN4HEALTH aims to develop cross-modal AI methods that enable Deaf and signing users to interact with digital health systems through sign language as a native communication modality. The project will study representations connecting sign video, pose and facial cues, text, speech, and health-related context, with a focus on robust learning across users and low-resource sign languages.

The planned research includes sign-to-text and sign-to-intent understanding, generation of sign-language output from health information, multimodal interaction, and uncertainty-aware communication. Possible use cases include telehealth, digital health assistants, patient portals, and patient–clinician communication support.

We are interested in collaboration with partners in digital health, clinical research, HCI and accessibility, sign language and Deaf studies, and trustworthy AI. We are also open to joining an existing Horizon Europe consortium where our expertise in computer vision, multimodal learning, and sign language AI can contribute to a broader digital health project. The project is currently at the concept stage, and we are open to shaping its scope together with potential partners. Use cases, technical work packages, and validation settings will be refined based on complementary expertise and consortium interests.