This project contributes to Grand Challenge 2: Situated AI by advancing human-centered and trustworthy AI agents capable of operating in dynamic environments and collaborating effectively with humans in real time. Positioned within Work Package 3 - Interaction and Cognitive Systems, the project focuses on developing interaction mechanisms that support complex task execution through clear communication, shared understanding, and adaptive planning between (semi-)autonomous agents and people. The central goal is to enable AI systems that do not merely act in an environment, but meaningfully engage with human partners in a safe, empathetic, and cognitively aligned manner.
Within Task 3.3 - Affective and Engaging Personalized Delivery, the project investigates how human involvement in dialogue can be simulated through sonic, visual, and linguistic communication, enhancing human-machine interaction from the human perspective. While situational awareness remains a guiding principle, particular emphasis is placed on clarity, explainability, and emotional attunement in interaction. Prior research has shown that clear communication increases perceived empathy and open-mindedness, while empathetic dialogue can reduce human stress and foster a sense of connectedness. Building on these insights, the project explores how agents can express attentiveness, appreciation, and affirmation to establish strong empathetic bonds with users during collaborative tasks.
From a technical perspective, the project addresses key limitations of current dialogue systems powered by large language models. Although such models enable fluent language generation, they often lack fine-grained control, goal awareness, and reliability, factors that are critical in applied, safety-sensitive environments. The project integrates multimodal signals, including visual cues and paralinguistic features, into empathetically aware response generation, in coordination with dialogue presentation techniques developed in Task 3.4. By doing so, the project aims to advance the state of the art in affective, personalized, and trustworthy AI communication for situated, collaborative environments.