The Application of Facial Expression and Micro-expression in Virtual Character Emotional Expressions
Draft of Literature Review
Introduction:
Emotion can be an important part of performance animation because it can build up the relationships between characters and the audience. Traditionally, performance animations are based on references and regular patterns, while ignoring the behavioral and psychological exploration of the relationship among emotions and facial expressions. Such method may weaken the expression of emotional details. Based on subjects such as psychology and evolutionary-biology, scientists believe that facial expressions and micro-expressions have a certain connection to human emotions, and this connection can be decoded in a scientific way. Their research provides basis for virtual characters realistic emotional expressions. This paper will base on recent research on facial expressions, micro-expressions and animation techniques, analyzing real cases in film, animation and practices, and try to find an artistic-scientific blended way to build character emotions in performance animations.
Literature review:
Key words: Facial expressions, Emotions, Animation.
Traditional performance animation:
According to Norman McLaren, “Animation is not the art of drawings that move but the art of movements that are drawn”. 2d and 3d animation can affect humans and their emotions significantly. Traditionally, animators animate performance animation based on references or their experience of observing human behaviors. Jason Osipa shares a large number of computer facial animation techniques in his book Stop Starring. The book covers variable kinds of facial expression techniques that animators used in their productions, which also covers techniques on emotion expressions (Osipa, 2010). However, these principles mostly base on observations but ignored the research on scientific level. Moreover, in certain emotional situations like hiding true emotions or deception, it is difficult to find patterns of people’s emotional changes or even represent them through traditional methods. However, some scientific or psychological research can provide materials of these specific emotional expressions for animators. Therefore, human psychology and other science research can be a guidance to help animators to develop correct animations for their audiences to connect with.
The scientific analysis development of facial expression and micro-expressions:
The research on the relationship among emotions, facial expressions and micro expressions has lasted for decades. Several scholars’ developments have been considered as the foundation of this area, and part of them has already been applied into film and animation industry.
Facial expression:
The research on facial expressions started by Charles Darwin and then refined by other scholars. Neuropsychological studies point out the asymmetry of facial expressions, which means that the two sides of human face are not performed symmetrically when emotional expressions occurs. Additionally, scientists realized that emotions can be recognized more easily on the left part of human face. They mentions that socially appropriate signals are clearly visible on the right face, while personalized signals are visible on the left face (Mandal and Awasthi, 2015, p. 274). Other scientists, for example, Ekman and Friesen, developed the universality thesis of facial expressions, which refers to accurate recognition of facial expressions across cultures at better-than-chance levels (Ekman et al. 1987). Ekman also proposed the idea of six basic emotional expressions, which has been accepted by psychologists: happiness, sadness, anger, fear, surprise and disgust (Russell and José Miguel Fernández-Dols, 2002, p. 11). However, it is still being criticized by some cross-cultural studies on emotional facial expressions (Russell, 1994).
Some of scientists then developed several automated system for the recognition of facial expressions. For example, electromyography and electroencephalogram (Mandal and Awasthi, 2015, p. 9). These systems fit the requirement of collecting experiment data and scientific research. However, they are not good guidance for animators. With the development of anatomically based coding systems, however, animators start to have more resources for analyzing facial expressions. Hjortsjö’s Mimic language can be considered as one of the earliest explorations of facial muscular activities systematization. The mimicry covers the additional expressive movements of gestures and postures, which are the characteristic manifestations of emotional states (Hjortsjö, 1970). Hjortsjö also described the facial expressions which are related to twenty four emotions, and divided these expressions into eight categories (Hjortsjö, 1970).
Another important development of anatomically based coding systems is Facial Action Coding System (FACS). FACS is developed by Paul Ekman and Wallace Friesen in 1978. They breaks down facial actions into small parts, which is called action units (AUs). Each of them can be considered as the basic elements of facial expressions. With the combination of different AUs, people can make different kinds of facial expressions based on muscle movements. At first facial action coding system is designed for motion records (Ekman, Friesen and Hager, 2002). However, it has now been widely applied into film industry and computer animation for decades (Parke and Waters, 2020, p. 33).
Micro-expressions:
Micro-expression is considered as a kind of more typical facial expression of emotion. The definition of micro-expression is still not clear. Mark G. Frank and Elena Svetieva tend to define micro-expression as any expression of emotion that is shown at 0.5 s or less because previous research had suggested that most of spontaneous expressions of emotion lasts between 0.5 and 4 (or 5) seconds (Mandal and Awasthi, 2015, p. 229). A distinctive feature of micro-expressions is that they reveal true emotional states in the form of facial expression for a very short period of time (0.5s or less), which is then quickly disguised or suppressed by another one.
The relationship between micro-expression and deception has been analyzed by some scholars. Some of the research focusing on the difference between real smile and fake smile. Scientists found that the main difference between real smile and fake smile is the muscle movements around eye area (Duchenne, 1990). Although muscles on the mouth corners are pulled up in both of these kinds of smiles, only the real smile will trigger the movements of orbicularis oculi muscle around the eyes (Duchenne, 1990). Additionally, according to DePaulo’s research, the facial pleasantness of liars is much lower than normal level. Liars will have more chin raises, more lip pressing, and look more nervous (DePaulo et al. 2003). However, other facial expressions like smiling, eyebrow lowering or raising have not shown consistently significant effect sizes (DePaulo et al. 2003).
References:
Carl-Herman Hjortsjö (1970). Man’s face and mimic language. Studentlitteratur.
DePaulo, B. M., Lindsay, J. J., Malone, B. E., Muhlenbruck, L., Charlton, K., & Cooper, H. (2003). “Cues to deception”, Psychological Bulletin, pp. 74–118.
Duchenne, G. B. (1990). The Mechanism of Human Facial Expression. Edited by R. A. Cuthbertson. Cambridge: Cambridge University Press (Studies in Emotion and Social Interaction). doi: 10.1017/CBO9780511752841.
Ekman, P., Friesen, W.V. and Hager, J.C. (2002). Facial action coding system. Salt Lake City: Research Nexus.
Ekman, P., Friesen, W. V., O’Sullivan, M., Chan, A., Diacoyanni-Tarlatzis, I., Heider, K., et al. (1987). “Universals and cultural differences in the judgments of facial expressions of emotion”, Journal of Personality and Social Psychology, 53(4), pp. 712–717.
Mandal, M.K. and Awasthi, A. (2015). Understanding Facial Expressions in Communication: Cross-cultural and Multidisciplinary Perspectives. New Delhi: Springer India.
Osipa, J. (2010). Stop staring facial modeling and animation done right. Indianapolis, Ind Sybex.
Parke, F.I. and Waters, K. (2020). Computer facial animation. Boca Raton: Crc Press.
Russell, J. A. (1994). “Is there universal recognition of emotion from facial expression? A review of the cross-cultural studies”, Psychological Bulletin, pp. 102–141.
Russell, J.A. and José Miguel Fernández-Dols (2002). The psychology of facial expression. Cambridge: Cambridge University Press.