Month: June 2022

The Application of Facial Expression and Micro-expression in Virtual Character Emotional Expressions Thesis Proposal

1.Introduction:

Emotion can be an important part of performance animation because it can build up the relationships between characters and the audience. Traditionally, performance animations are based on references and regular patterns, while ignoring the behavioral and psychological exploration of the relationship among emotions and facial expressions. Such method may weaken the expression of emotional details. Based on subjects such as psychology and evolutionary-biology, scientists believe that facial expressions and micro-expressions have a certain connection to human emotions, and this connection can be decoded in a scientific way. Their research provides basis for virtual characters realistic emotional expressions. This paper will base on recent research on facial expressions, micro-expressions and animation techniques, analyzing real cases in film, animation and practices, and try to find an artistic-scientific blended way to build character emotions in performance animations.

Key words: Facial expressions, Emotions, Animation.

2.Background Significance:

Traditional 3d animation has already set up a form of facial expression animation techniques based on observations and regular patterns. Animators summarize several facial expression characteristics of human basic emotions, and form these shapes on faces by using customized controllers on character’s rig. However, recent techniques and rules cannot manage to achieve some facial expressions like micro-expressions or expressions in specific situations like lying, disguising real emotions, etc. At present, the research on facial expressions in these areas is mainly in the field of psychology, biology and sociology. Multidisciplinary research on facial expressions and micro-expressions provides a great number of practical materials, which can be valuable references for 3D animations production. Therefore, based on existing research content in other areas, combined with the theory and techniques of rigging and facial animation, this paper will explore and summarize micro-expressions and facial animation methods in some specific situations.

Despite of the close relationship between facial expressions and body language, emotion-related body language research may be covered in part of this paper as supplementary information. This paper will not delve into the core principles of facial expressions and micro-expressions in biology and psychology. It will focus on summarizing and filtering the content of existing research, and try to find an artistic-scientific blended way to build character emotions in performance animations.

3.Literature review:

Traditional performance animation:

According to Norman McLaren, “Animation is not the art of drawings that move but the art of movements that are drawn”. 2d and 3d animation can affect humans and their emotions significantly. Traditionally, animators animate performance animation based on references or their experience of observing human behaviors. Jason Osipa shares a large number of computer facial animation techniques in his book Stop Starring. The book covers variable kinds of facial expression techniques that animators used in their productions, which also covers techniques on emotion expressions (Osipa, 2010). However, these principles mostly base on observations but ignored the research on scientific level. Moreover, in certain emotional situations like hiding true emotions or deception, it is difficult to find patterns of people’s emotional changes or even represent them through traditional methods. However, some scientific or psychological research can provide materials of these specific emotional expressions for animators. Therefore, human psychology and other science research can be a guidance to help animators to develop correct animations for their audiences to connect with.

The scientific analysis development of facial expression and micro-expressions:

The research on the relationship among emotions, facial expressions and micro expressions has lasted for decades. Several scholars’ developments have been considered as the foundation of this area, and part of them has already been applied into film and animation industry.

Facial expression:

The research on facial expressions started by Charles Darwin and then refined by other scholars. Neuropsychological studies point out the asymmetry of facial expressions, which means that the two sides of human face are not performed symmetrically when emotional expressions occurs. Additionally, scientists realized that emotions can be recognized more easily on the left part of human face. They mentions that socially appropriate signals are clearly visible on the right face, while personalized signals are visible on the left face (Mandal and Awasthi, 2015, p. 274). Other scientists, for example, Ekman and Friesen, developed the universality thesis of facial expressions, which refers to accurate recognition of facial expressions across cultures at better-than-chance levels (Ekman et al. 1987). Ekman also proposed the idea of six basic emotional expressions, which has been accepted by psychologists: happiness, sadness, anger, fear, surprise and disgust (Russell and José Miguel Fernández-Dols, 2002, p. 11). However, it is still being criticized by some cross-cultural studies on emotional facial expressions (Russell, 1994).

Some of scientists then developed several automated system for the recognition of facial expressions. For example, electromyography and electroencephalogram (Mandal and Awasthi, 2015, p. 9). These systems fit the requirement of collecting experiment data and scientific research. However, they are not good guidance for animators. With the development of anatomically based coding systems, however, animators start to have more resources for analyzing facial expressions. Hjortsjö’s Mimic language can be considered as one of the earliest explorations of facial muscular activities systematization. The mimicry covers the additional expressive movements of gestures and postures, which are the characteristic manifestations of emotional states (Hjortsjö, 1970). Hjortsjö also described the facial expressions which are related to twenty four emotions, and divided these expressions into eight categories (Hjortsjö, 1970).

Another important development of anatomically based coding systems is Facial Action Coding System (FACS). FACS is developed by Paul Ekman and Wallace Friesen in 1978. They breaks down facial actions into small parts, which is called action units (AUs). Each of them can be considered as the basic elements of facial expressions. With the combination of different AUs, people can make different kinds of facial expressions based on muscle movements. At first facial action coding system is designed for motion records (Ekman, Friesen and Hager, 2002). However, it has now been widely applied into film industry and computer animation for decades (Parke and Waters, 2020, p. 33).

Micro-expressions:

Micro-expression is considered as a kind of more typical facial expression of emotion. The definition of micro-expression is still not clear. Mark G. Frank and Elena Svetieva tend to define micro-expression as any expression of emotion that is shown at 0.5 s or less because previous research had suggested that most of spontaneous expressions of emotion lasts between 0.5 and 4 (or 5) seconds (Mandal and Awasthi, 2015, p. 229). A distinctive feature of micro-expressions is that they reveal true emotional states in the form of facial expression for a very short period of time (0.5s or less), which is then quickly disguised or suppressed by another one.

The relationship between micro-expression and deception has been analyzed by some scholars. Some of the research focusing on the difference between real smile and fake smile. Scientists found that the main difference between real smile and fake smile is the muscle movements around eye area (Duchenne, 1990). Although muscles on the mouth corners are pulled up in both of these kinds of smiles, only the real smile will trigger the movements of orbicularis oculi muscle around the eyes (Duchenne, 1990). Additionally, according to DePaulo’s research, the facial pleasantness of liars is much lower than normal level. Liars will have more chin raises, more lip pressing, and look more nervous (DePaulo et al. 2003). However, other facial expressions like smiling, eyebrow lowering or raising have not shown consistently significant effect sizes (DePaulo et al. 2003).

4.Research Design, method, and schedule:

Based on quantitative research method, combined with the existing research data on facial expressions of other scholars in biology, psychology and other subjects, this paper will summarize micro-expressions and facial expressions of non-basic emotions, and conduct a more detailed analysis of the theory of facial expression animation. This paper will base on the theory of FACS (Facial Action Coding System), research data from scholars like Bella M. DePaulo, Paul Ekman and Carl-Herman Hjortsjö, and films and animations focusing on facial expression animation applications, classify and design the system of micro-expression and non-basic emotional facial animations in practical applications.

The research will firstly review the data collected by other scholars and visual materials in animation industry, and classify the form of micro-expression and non-basic facial expressions. The research will then focusing on building a system for the practical application of micro-expression and non-basic facial expressions theory in 3D animation. To test the system and methods of micro-expression and non-basic expression animations, the research will run experiments to collect data, test technical feasibility and improve the design. These experiments will focusing on some topics like the recognition of micro-expression in facial animation or the readability of deception in facial animation.

The difficulty of supplementing facial animation theory is how to clearly classify supplementary content, which requires detailed cross-research on various types of data. In addition, how to effectively realize these micro-expressions and specific emotions in animations is also one of the difficulties, which requires the design of new facial animation system base on recent facial animation and rigging technology.

5.Supposition and Implication:

This research can provide basic directions and data for future research on micro-expression and complex expression animation, and provide animation and some rigging solutions for these types of expression animation. The research on micro-expression and non-basic facial expression animation from the perspectives of psychology and biology can provide new ideas for 3d animation, and explore the possibility of making more convincing facial expression or performance animations from both artistic and scientific perspectives. Recently, animation techniques and rules cannot manage to achieve some facial expressions like micro-expressions or expressions in specific situations like lying, disguising real emotions, etc. Based on 3d facial animation techniques and the research of facial expression and micro-expression in other scientific areas, this paper will provide solutions to some of the problems above to a certain extent. The research content of facial expression animation in this paper can be used in film or film-based game animation areas. The theory can also be applied to animation refinement after motion capture to increase more performance details.

6.Conclusion:

In conclusion, the multidisciplinary based research on micro-expression and facial expression animations provides usable materials for facial animations which focusing on non-basic emotions in 3D animation. It can be foreseen that the solution of micro-expression and non-basic facial expression is different with the solution of basic emotion in 3D animation. The research on micro-expression and facial expression animation of specific emotions effectively adds more details to facial animation of virtual characters. The feasibility and practicability of the new system have been confirmed.

There are two areas where the work in this paper could be extended. First of all, the relationship among micro-expressions, facial animations and body language in character animations. Body language, or gesture expressions, can be considered as one of the most important parts of character performance animations, which can also be explored and researched further by combining traditional animation techniques with other disciplines such as psychology, biology, and sociology. Due to space limitations, the research did not delve into the body language in animation and its connection with micro-expression and facial expression animation. Additionally, the relationship between audio and facial expressions. Human emotions affect both the voice and behaviour. Under the premise of the same emotion, different levels of emotional expression will significantly affect the intonation and tone. Therefore, it is possible that animators can manipulate facial details and magnitude to show expressions in different levels based on the intonation and tone.

7.Bibliography:

Carl-Herman Hjortsjö (1970). Man’s face and mimic language. Studentlitteratur.

DePaulo, B. M., Lindsay, J. J., Malone, B. E., Muhlenbruck, L., Charlton, K., & Cooper, H. (2003). “Cues to deception”, Psychological Bulletin, pp. 74–118.

Duchenne, G. B. (1990). The Mechanism of Human Facial Expression. Edited by R. A. Cuthbertson. Cambridge: Cambridge University Press (Studies in Emotion and Social Interaction). doi: 10.1017/CBO9780511752841.

Ekman, P., Friesen, W.V. and Hager, J.C. (2002). Facial action coding system. Salt Lake City: Research Nexus.

Ekman, P., Friesen, W. V., O’Sullivan, M., Chan, A., Diacoyanni-Tarlatzis, I., Heider, K., et al. (1987). “Universals and cultural differences in the judgments of facial expressions of emotion”, Journal of Personality and Social Psychology, 53(4), pp. 712–717.

Mandal, M.K. and Awasthi, A. (2015). Understanding Facial Expressions in Communication: Cross-cultural and Multidisciplinary Perspectives. New Delhi: Springer India.

Osipa, J. (2010). Stop staring facial modeling and animation done right. Indianapolis, Ind Sybex.

Parke, F.I. and Waters, K. (2020). Computer facial animation. Boca Raton: Crc Press.

Russell, J. A. (1994). “Is there universal recognition of emotion from facial expression? A review of the cross-cultural studies”, Psychological Bulletin, pp. 102–141.‌

Russell, J.A. and José Miguel Fernández-Dols (2002). The psychology of facial expression. Cambridge: Cambridge University Press.