Multimodal wearables for silent speech interfaces

We are interested in developing multimodal wearable silent-speech interfaces combining surface electromyography and ultrasound sensing.

Silent-speech systems aim to infer intended speech from the physiological activity associated with articulation, without requiring the user to produce an audible acoustic signal. Surface EMG captures activity of superficial speech-related muscles, while ultrasound can provide complementary information about deeper muscle motion and articulatory dynamics that are difficult to observe electrically.

Combining these modalities with modern machine-learning methods could improve robustness across speakers, sessions and speaking conditions, while enabling compact and fully wearable systems.

Potential research directions include:

• decoding silently articulated words or continuous speech

• communication support for people with severe speech impairments

• multimodal EMG–ultrasound sensor fusion

• wearable sensing of facial, neck and articulatory muscle activity

• personalization and adaptation across users

• self-supervised and foundation-model approaches for physiological speech signals

• low-power on-device inference for private, real-time communication

• integration with augmentative and alternative communication systems

Our contribution would focus on wearable sensing hardware, EMG and ultrasound acquisition, embedded systems, multimodal machine learning and edge AI.

We are looking for partners with expertise in speech and language technologies, clinical speech disorders, neurology, rehabilitation, assistive communication, machine learning and human–computer interaction, particularly where the objective is to translate silent-speech research into practical patient-facing systems.

Selected Examples of Our Work:

- Silent speech interfaces, leveraging EMG and embedded intelligence (https://arxiv.org/pdf/2509.21964) (https://arxiv.org/abs/2603.02847)