Publication Details

Download

Teodora Mitrevska, Sven Mayer
Physiological Signals as Implicit Multimodal Input in Human-AI Interactions During Creative Tasks
GenAICHI: CHI 2024 Workshop on Generative AI and HCI, 2024-01-01 (bib)

Recent experiments have explored AI's role in creative collaboration, producing promising outcomes. While AI excels in generating text and visual art, human-AI collaboration lacks rich feedback compared to human-human interaction. So far, textual feedback is the dominant communication between systems and humans, which is in strong contrast to feedback in human-human conversations with many nonverbal cues. For creative individuals, their mental state, especially emotions, play a big role in creating art. They project their feelings on canvas or translate them into music. However, not being able to express them verbally puts a strain on the feedback and impacts the output of the collaboration. Using methods for emotion recognition can provide insights into humans' emotional states, which can be used to improve human-AI collaborations in creative scenarios. In this work, we present the concept of factoring in human physiological signals in co-creative scenarios with Generative AI as a multimodal input. We underscore this vision by presenting three applications utilizing an emotional-aware input.