Beyond Keyboard and Mouse: A Wearable AI-Driven Input System for Hands-Free Human-Digital Interaction

It is an innovative wearable and AI-enabled human-machine interaction platform designed to transform the way people interact with digital environments.

Current digital interaction relies heavily on fixed desktop peripherals, forcing users into static working positions and limiting accessibility for people with reduced mobility. Existing XR ecosystems also lack truly natural, wearable and hands-free interaction methods.

Our proposal introduces a new interaction paradigm that enables users to control digital systems while standing, walking, sitting, reclining or lying down. The platform combines:

• Novel wearable hardware

• Embedded AI and Edge AI capabilities

• Speech-to-Text and Text-to-Speech technologies

• Multimodal interaction methods

• Mouse, keyboard and joystick replacement functionalities

• Multi-device and multi-screen control capabilities

The project contributes to the objectives of HORIZON-CL4-2027-04-HUMAN-01 by developing advanced wearable interaction technologies for immersive digital experiences, accessibility and future XR environments.

Beyond XR applications, the solution has strong societal impact through:

• Increased accessibility for people with motor limitations

• Improved inclusion and digital participation

• Reduced dependence on static workstation environments

• Better ergonomics and healthier digital working habits

• New opportunities for hands-free interaction in industrial and professional contexts

Planned demonstrators include:

• XR and Virtual Worlds interaction

• Accessibility and assisted interaction scenarios

• Ergonomic workplace environments

• Professional and industrial hands-free workflows

Partner Search

We are currently seeking partners interested in joining a consortium for HORIZON-CL4-2027-04-HUMAN-01, including:

• XR / Virtual Worlds developers

• Human-Computer Interaction (HCI) researchers

• SSH experts (human factors, psychology, sociology, digital inclusion)

• Ergonomics and occupational health specialists

• Universities and research centres working on immersive technologies

• Industrial pilot partners

• Hardware, sensor and haptic technology providers

• Digital twin and spatial computing experts

We are open to acting either as a technology leader/work package leader or as a core technology partner within a larger consortium.