Safe Artificial Intelligence for Healthcare

Artificial intelligence is attracting increasing interest in the healthcare sector, where it can play a supportive role across different stages of the clinical pathway—screening and prevention, diagnosis and treatment, and prognosis—by processing large volumes of data, including heterogeneous data (e.g., clinical measurements, images, textual reports, etc.). The critical nature of this domain and emerging AI regulations require the adoption of trustworthy AI approaches, in which both healthcare professionals and patients are placed at the center. This highlights the need to rethink the medical software V&V system to account for the new, AI-specific requirements and its inherently stochastic nature, by defining appropriate safety requirements and corresponding countermeasures.

Our research aims to address these challenges by proposing AI performance monitoring approaches designed to detect and signal the operational limits of AI algorithms, such as unforeseen situations on which the algorithms have never been trained.

To this end, we study and develop new tools capable of profiling the confidence of AI outputs, by combining uncertainty quantification techniques (e.g., conformal prediction) with eXplainable AI methods (e.g., decision rules, counterfactual explanations, etc), thus adhering to the principle of transparency imposed by current legislation.

We look for opportunities to join or build consortia within Cluster 1 calls.