Recognizing Compound Facial Expressions of Virtual Characters in Augmented Reality

Sammanfattning: How is it possible to design virtual characters that can express different emotions such as compound emotions that are a mix of basic emotion expressions? Augmented reality (AR) can create engaging experiences for participants, and in recent years, virtual faces and virtual characters have become increasingly realistic and expressive, for example, when reducing costs using therapeutic applications. The validity of virtual expression has been shown in studies on desktop computers but less so in AR. In this paper, the basic emotions and the compound emotions of virtual characters were studied and the character was designed to work with Microsoft HoloLens in AR. The process of creating basic emotion blends to a human virtual character was created and the animations were modified using Unity3D game engine. The participants (n = 24) experienced the virtual character in a job interview context wearing HoloLens mixed reality glasses. The virtual character made basic and compound facial expressions and the participants were asked to label them. The result show that all participants successfully recognized seven basic emotions and seven compound emotions from a virtual character in AR using HoloLens; however, disgust was confused with sad, and angry was sometimes confused with disgust. Also, the fearfully surprised was often mistaken with awed. The result show that the compound emotions were recognized quite well and the results indicate that the perceived valence changes depending on the facial expressions. The study provides insights into how blended emotive expressions for virtual characters are created and perceived.