Manipulating and Evaluating Levels of Personality Perceptions of Voice Assistants through Enactment-Based Dialogue Design

Sarah Theres Völkel, Samantha Meindl, and Heinrich Hussmann

In Proceedings of the 3rd Conference on Conversational User Interfaces (CUI '21)

Abstract. We present an enactment-based dialogue design approach to imbue voice assistants with different levels of personality. In two focus groups, we asked amateur actors to animate different voice assistant personalities by writing and enacting dialogues between a voice assistant and a user. We realised the resulting dialogues using Amazon Alexa and presented them to N=156 participants in an online survey to investigate whether the personality levels were successfully synthesised and if user personality influences the preference for specific personality levels. Our results indicate that participants ranked the personality levels as intended but that high personality levels were perceived as less pronounced than expected. Furthermore, we found a small relationship between extraversion and a preference for a voice assistant that is Social-entertaining whilst conscientious participants tended to reject a Confrontational voice assistant. We discuss implications for researchers and practitioners on how to manipulate voice assistant personalities and adapt them to users.

Research Design

To imbue voice assistants with different levels of a personality dimension, we follow a similar approach to the design of commercial voice assistants, namely using focus groups with amateur actresses and actors, which we refer to as Enactment-based Dialogue Design. Following the resulting scripts and discussion from the focus groups, we realised the dialogues using Amazon Alexa and recorded a conversation between the voice assistant and a user. In a third step, we evaluated these recordings in an online survey with N=156 participants.

Dialogues

We modelled the dialogues for our online survey after participants' results in the focus groups, whilst slightly revising them based on the group's feedback at the end of each session and literature on conversational user interfaces, for example providing the user with options to choose from.

We then implemented the dialogues with the help of Amazon Alexa and created audio recordings of a dialogue between a user and the voice assistant for each scenario. In an online survey, we presented participants with the resulting dialogues.

here

the first author

Study Material

Please find our original study material below for the focus groups with amateur actors (part 1) and the online survey (part 2). Please note that the studies were conducted in German. If you require a translation, please contact the first author.

Focus Groups

We conducted two focus groups with six amateur actors each to develop the dialogues for the voice assistants, one focus groups for each dimension. We first briefed participants on the personality dimension they were supposed to synthesise in the dialogues. The main part of the focus group then comprised three steps: First, actresses and actors were randomly assigned one scenario, which was followed by asking all participants to draft the voice assistant part for their scenario, writing different versions which represent the three levels of the personality dimension: low, rather high, and high. Second, the two participants who worked on the same scenario got together for discussing, merging, and optimising their dialogues and practising them subsequently. Third, each scenario team enacted their dialogues in front of the group, followed by a discussion and feedback.

Online Survey

We conducted an online survey with 156 participants to investigate (1) if our approach was successful in imbuing voice assistants with different levels of Social-Entertaining and Confrontational and (2) whether there is a relationship between user personality and their preference for a certain personality level. To do this, we presented participants with the audio recordings of our developed dialogues (cf. )

Material

Focus group guideline Focus group information Online survey questionnaire

Takeaways

Enactment-based dialogue design is suitable to synthesise different levels of personality. Higher levels were perceived as less pronounced.

Preference for voice assistant personality is to some extent influenced by user personality.

Users preferred more pronounced personality levels in low-stake scenarios.

Need for an adequate personality evaluation instrument to fully compare different levels.