KETs2026 reloaded

24 Sept 2026 | Germany

Project cooperationUpdated on 18 August 2026

Reliable Human Feedback for Frontier Model Alignment

Assistant Professor at University of Southern Denmark

Odense, Denmark

About

The SDU Metaverse Lab at the University of Southern Denmark works on human feedback for AI systems. We develop tools and protocols for collecting, structuring and analysing subjective human judgement on multimodal data, and methods for estimating annotator reliability.

This addresses a known weak point in alignment. Preference data and human oversight signals carry annotator noise, disagreement and bias. We model that variation rather than averaging it away, which matters where reward hacking of human oversight is a stated risk. Our staff developed PAGAN, a platform for continuous annotation, and lead work at HumanFeedback.ai on annotator reliability for multimodal AI.

We also evaluate interactive AI systems with users. For agentic frameworks we design protocols measuring how people interpret, trust and act on model output, including where they fail to notice model error.

We contribute to consortia through:

  • Human-in-the-loop annotation protocols and tooling for post-training data

  • Annotator reliability estimation and disagreement modelling

  • Human oversight protocols for agentic AI systems

  • Experimental design and human-centred evaluation of AI systems

  • Behavioural and affective modelling of user response to model output

We seek a task-level role on human feedback and human oversight, within a consortium led by partners doing frontier model training. We do not offer model training or compression capability.

Contact: davm@mmmi.sdu.dk

Similar opportunities