Research Projects
While I am unable to share all of my current projects, the following examples highlight the key areas of research I am actively engaged in.
Automatic Evaluation Framework for Social Conversations with Robots
When deploying social robots in the wild, it is crucial for developers to gain an understanding of how the interactions between the robot and its human conversational partners are progressing. Unlike in traditional task-based settings in which a human and a robot work on a tangible outcome that can serve as a proxy for how well a conversation is going, social settings require a deeper understanding of the underlying interaction dynamics. In our work, we assess potential features of a robot having social conversations in a multi-party, multi-session setting and correlate them with how people rate their interaction. We also work on a framework that combines the features into a model that can automatically assess an ongoing conversation and determine its performance.
Disney Research
Improving a Robot’s Turn-Taking Behavior in Dynamic Multiparty Interactions
Disney Research
We are interested in developing a robust and natural turn-taking behavior for a social agent to engage a dynamically changing group in a conversation. For that, we assess features derived from more than 9,000 dialogue samples and train model that result in better decision policies for our robots to engage in.
The Rapid Dialogue Game - Map
The domain I use for many of my research projects is called Rapid Dialogue Game - Map (RDG-Map). It's a cooperative, two-player game about locating countries on the world map. One player acts as the Director in the game, who sees one country highlighted on the world map. The Director needs to verbally describe the country to the Matcher, who aims to select the same country as fast as possible.
We have developed both an online version of the game that can be played with the unembodied agent Nellie and a version involving our Furhat robot Neil as the game partner.
uppsala university
Several research projects have been based on this domain:
The development of uncanny feelings over time - studying whether initial uncanny feelings towards a robot can be overcome over the course of repeated interactions
Building an artificial game partner - The map-game poses several challenges for an agent to play the game autonomously, especially when it comes to the Natural Language Understanding (NLU) part of the system. Of particular interest is visual reference resolution
Robot personality - understanding how robots can express coherent and consistent personalities that develop naturally over time
Modelling user engagement - The overall goal is to keep the user engaged in the game by adapting the dialogue strategy of the agent when needed.
Previous Work
RoboCup Team Hamburg Bit-Bots
Together with other students from the University of Hamburg, I founded the RoboCup team Hamburg Bit-Bots in 2011. Since 2012, the team competed in the GermanOpen and the world championship every year. While I was still actively engaged in the team, I contributed to the robot behavior, localization on the field and team communication. I've also been involved in the organization of the team, managing the trips to the tournaments and sponsor acquisition.
Mimicry of Artificial Agents
In a collaboration with researchers at the Institut des Systèmes Intelligents et de Robotique, Université Pierre et Marie Curie, we investigate what influences people's mimicry of an artificial agent with a specific focus on potential uncanny feelings towards the artificial agent. One aim of the project was to research if mimicry can be used as a potential future assessment criteria for uncanny feelings towards a robots and 2D virtual characters.
Designing for the Uncanny
By analyzing the interaction strategies of a street performer with his dragon puppet, we developed design guidelines for what creates the feeling of uncanny and how to overcome this feeling using interaction techniques.
Gender and the Uncanny
To broaden the knowledge of the uncanny valley effect, we analyzed the influence of a perceptual mismatch in the category of gender. We used the Furhat mixed-embodied robot platform with a female and male appearance and voice. By altering different cues and thereby creating a perceptual mismatch, we aimed to understand what triggers the feeling of uncanny in different age groups.
Eve – a Dialogue Agent Playing an Image Matching Game
As part of the NSF founded project "Incremental Speech Processing for Rapid Dialogue" we developed a dialogue agent that can engage in a two-player image matching game. In the game, a human acts as a Director and describes one out of eight images they see on the screen. The automated matcher dialogue system processes the speech of the director and selects the most likely image candidate as soon as possible. The dialogue strategy of the agent is optimized to maximize the points the team scores per second.