Publikations-Information

Download

Thomas Kosch, Mariam Hassib, Robin Reutter, Florian Alt
Emotions on the Go: Mobile Emotion Assessment in Real-Time Using Facial Expressions
In AVI '20: Proceedings of the International Conference on Advanced Visual Interfaces. June 01, 2020. Association for Computing Machinery, New York, NY, USA. (bib)

Exploiting emotions for user interface evaluation became an increasingly important research objective in Human-Computer Interaction. Emotions are usually assessed through surveys that do not allow information to be collected in real-time. In our work, we suggest the use of smartphones for mobile emotion assessment. We use the front-facing smartphone camera as a tool for emotion detection based on facial expressions. Such information can be used to reflect on emotional states or provide emotion-aware user interface adaptation. We collected facial expressions along with app usage data in a two-week field study consisting of a one-week training phase and a one-week testing phase. We built and evaluated a person-dependent classifier, yielding an average classification improvement of 33% compared to classifying facial expressions only. Furthermore, we correlate the estimated emotions with concurrent app usage to draw insights into changes in mood. Our work is complemented by a discussion of the feasibility of probing emotions on-the-go and potential use cases for future emotion-aware applications.