Category: Final Major Project & Thesis

Just like last week I’m still focusing on making animations. As I have already made several animations, the speed of animating becomes faster this time.

I also added several shots into the storyboard this week since I found that the translation between some shots are not quite natural. I also added more reaction shots to show the facial expressions of the character. This change made the rhythm looks better, but also increased a huge amount of works.

There is nothing fancy this week. I just keep making facial animations and body animations. Animation is a huge task, which is also something that takes time and patience. I am not an expert on animating, but I will try my best to make it as good as possible.

This week I started to make animations. The animation contains camera animation, props animation and character animation.

Camera animations and prop animations are quite easy. I have already set up the rig and controlers on these props so that I just need to do some simple animations. However, character animations can be a big task, which not only has body animations but also facial performance. I tried to finish about 2-3 shots everyday so that I can have enough time to do VFX parts and compositions.

Based on my research on facial animations and expressions, I know exactly how to make facial expressions to express feelings. However, some of the shots still took me a lot of time to finish. I need to consider the rhythm of the whole shot and the layers of expressions. For example, I need to animate the face in several ways to show different levels of fear. When the scene becomes stranger and stranger, the emotion of character should keep changing so that audience can realize the situation becomes worse.

After I finished all the scene models and textures, I started to combine them together to form the real scene in the animation.

I firstly used the reference system in Maya to reference these objects into a new scene project so that these original projects can be protected. I layout several scenes like working room, livingroom, office, corridor, printing room and dead scene. This can be the most easy part for me. I just need to move things here and there.

After I layout all the scenes I started to do lighting. The lighting section didn’t take to much time. What I need to do is to create a sense of cinematic and horror in these shots. I created light layers in redshift so that after I finished rendering, I can change these lights in other softwares like Nuke. Therefore, the set up of light is quite easy to me.

Below is some scenes layout that I finished:

This week I finished the other part of texturing. And mostly focusing on riggings.

I textured some props and covered them with blood, since some scenes need to be blooded to create a sense of horror and disgust.

Fortunately, some props in the scene don’t need to be textured in detail. I just need to unfold the UV carefully and put textures into their materials, which saved some time and energy for me.

After I finished texturing, I started to set up rigs. What I need to rig in this project is some props, the character’s face and body.

I firstly rigged the most difficult part: character whole body rig. I started with the body, which is not a big task for me because I have done this several time in the former projects. It took me several hours to set up controllers and paint skin weights for the body. After that I started to rig the face. What I want is a character which can show complex facial expressions. Therefore, based on the research in my FMP thesis, I set up a facial rig based on FACS (Facial Action Coding System). Since the time and my technique are limited, I cannot set up a cinematic level rig, so I did a easier version. It took me days to make the action unit blendshapes and paint skin weights, which can be really struggled. Finally, I finished it, and the result looks good.

After I finshed the character rig, I turned to rig some props in my scenes.

The rig of printer took me some time to finish, since it is an important object in this project.

I also rigged something like laptop, vending machine, doors, clock, etc.

In week 4, I still need to finish the other part of the scene first.After that I will start to make textures for the whole scene, which can be a big task.

Just like last week, I prepared the corridor and the printing room. I add more props into these scenes. For eample, some books and laptops.

After I finished all the scenes props that the model list. I started to make textures.

I use Substance Painter to make textures. Since Quixel has a lot of great and high resolution textures, I grabed some textures on this app and use them to shade my models. It took me some time to finish part of these textures.

I started modeling and texturing scenes in my animation. Totally I need four scenes: working room, livingroom, corridor and printing room.

Since these rooms requires a lot of furnitures and props. I modeled part of it, others are found on the internet, all the models are free. After that I textured them in Substance Painter. I did two versions of the livingroom. One is for the first part of the story, and the second version is for the suicide scene. I firstly layout the scene and build the pose of dead body, after that I export the character and the scene into Substance Painter to draw the blood. It took me several days to finish shading and texturing.

Below is some photo of scenes

I started to design my character after finished the script. The character is a man who works as an architect/designer in a company. The character should wear like an office stuff. Therefore, I draw an outfit design for the character. The design is quite normal. The man is wearing blue shirt, jeans and sneakers.

Below is the design of the character:

After that I modeled the character. I firstly find a free male base character on website and reshape his head in Zbrush to get the result that I want. I then find our clothes in character creator 4, which is quite suitable for my character.

I then used Substance Painter and Mari to paint skin textures of this character. I spent most of the time making diffuse texture, normal and SSS textures because these textures will significantly affect the quality of character rendering. Especially the SSS ( sub-surface scattering) texture, which judges the texture of human bone and skin.

After that I also used the base model to generate the human-like creature for this animation. The creature has a giant and weird mouth which is full of blood. It also has long fingers, which looks creepy. I also want the creature looks like a dead body and has some blood on it’s skin. Therefore, I used Substance Painter to make another version of skin texture. This skin texture has more green and blue colors on it to mimic the dead body texture. I used some physical brush to paint blood on the model so that it can form quite realistic blood flow effects on the body. These brushs are really handy, which can also be used in the texturing of environment.

Story

This week I started to prepare for my FMP. What I would like to do in my FMP is a short horror animation which is based on some urban legend artworks such as SCP, Backrooms, Mandela Catalogue, Game Control , etc.

I firstly did some research on urban legend stories and artworks, trying to find out which kind of style I want. Urban legends usually base on some background like night city, village, secret laboratories and folk stories. I finally decided to build up a story based on abnormal space and abnormal products.

After that I came with several ideas like secret investigations, laboratory accidents and other things. I finally choose to write a weird experience of a normal person working at midnight. I want the protagonist to get into a space that looks normal at first, but as the story goes the space will become weird and become aggressive.

Here is the breif of this story:

CHANNEL-7: the protagonist who works overtime until midnight experiences hallucinations after watching a non-existent Channel-7 program on TV, which triggers his abnormal behavior and leads to tragedy.

I decided to split this story into three parts:

• Part 1, the protagonist is working normally and accidentally get into the strange space (in his mind).
• Part 2, the protagonist exploring in this space and then find out weird things in this space. Finally the protagonist was trapped in this space and lost.
• Part 3, back to the reality, the protagonist has been found dead and committed susuicide.

Storyboard

Based on the story, I started to draw the storyboard, which is about 40-50 shots. I added more shots during the production, and finally the storyboard covers about 50-60 shots.

Below is the final storyboard :

The Application of Facial Expression and Micro-expression in Virtual Character Emotional Expressions Thesis Proposal

1.Introduction:

Emotion can be an important part of performance animation because it can build up the relationships between characters and the audience. Traditionally, performance animations are based on references and regular patterns, while ignoring the behavioral and psychological exploration of the relationship among emotions and facial expressions. Such method may weaken the expression of emotional details. Based on subjects such as psychology and evolutionary-biology, scientists believe that facial expressions and micro-expressions have a certain connection to human emotions, and this connection can be decoded in a scientific way. Their research provides basis for virtual characters realistic emotional expressions. This paper will base on recent research on facial expressions, micro-expressions and animation techniques, analyzing real cases in film, animation and practices, and try to find an artistic-scientific blended way to build character emotions in performance animations.

Key words: Facial expressions, Emotions, Animation.

2.Background Significance:

Traditional 3d animation has already set up a form of facial expression animation techniques based on observations and regular patterns. Animators summarize several facial expression characteristics of human basic emotions, and form these shapes on faces by using customized controllers on character’s rig. However, recent techniques and rules cannot manage to achieve some facial expressions like micro-expressions or expressions in specific situations like lying, disguising real emotions, etc. At present, the research on facial expressions in these areas is mainly in the field of psychology, biology and sociology. Multidisciplinary research on facial expressions and micro-expressions provides a great number of practical materials, which can be valuable references for 3D animations production. Therefore, based on existing research content in other areas, combined with the theory and techniques of rigging and facial animation, this paper will explore and summarize micro-expressions and facial animation methods in some specific situations.

Despite of the close relationship between facial expressions and body language, emotion-related body language research may be covered in part of this paper as supplementary information. This paper will not delve into the core principles of facial expressions and micro-expressions in biology and psychology. It will focus on summarizing and filtering the content of existing research, and try to find an artistic-scientific blended way to build character emotions in performance animations.

3.Literature review:

Traditional performance animation:

According to Norman McLaren, “Animation is not the art of drawings that move but the art of movements that are drawn”. 2d and 3d animation can affect humans and their emotions significantly. Traditionally, animators animate performance animation based on references or their experience of observing human behaviors. Jason Osipa shares a large number of computer facial animation techniques in his book Stop Starring. The book covers variable kinds of facial expression techniques that animators used in their productions, which also covers techniques on emotion expressions (Osipa, 2010). However, these principles mostly base on observations but ignored the research on scientific level. Moreover, in certain emotional situations like hiding true emotions or deception, it is difficult to find patterns of people’s emotional changes or even represent them through traditional methods. However, some scientific or psychological research can provide materials of these specific emotional expressions for animators. Therefore, human psychology and other science research can be a guidance to help animators to develop correct animations for their audiences to connect with.

The scientific analysis development of facial expression and micro-expressions:

The research on the relationship among emotions, facial expressions and micro expressions has lasted for decades. Several scholars’ developments have been considered as the foundation of this area, and part of them has already been applied into film and animation industry.

Facial expression:

The research on facial expressions started by Charles Darwin and then refined by other scholars. Neuropsychological studies point out the asymmetry of facial expressions, which means that the two sides of human face are not performed symmetrically when emotional expressions occurs. Additionally, scientists realized that emotions can be recognized more easily on the left part of human face. They mentions that socially appropriate signals are clearly visible on the right face, while personalized signals are visible on the left face (Mandal and Awasthi, 2015, p. 274). Other scientists, for example, Ekman and Friesen, developed the universality thesis of facial expressions, which refers to accurate recognition of facial expressions across cultures at better-than-chance levels (Ekman et al. 1987). Ekman also proposed the idea of six basic emotional expressions, which has been accepted by psychologists: happiness, sadness, anger, fear, surprise and disgust (Russell and José Miguel Fernández-Dols, 2002, p. 11). However, it is still being criticized by some cross-cultural studies on emotional facial expressions (Russell, 1994).

Some of scientists then developed several automated system for the recognition of facial expressions. For example, electromyography and electroencephalogram (Mandal and Awasthi, 2015, p. 9). These systems fit the requirement of collecting experiment data and scientific research. However, they are not good guidance for animators. With the development of anatomically based coding systems, however, animators start to have more resources for analyzing facial expressions. Hjortsjö’s Mimic language can be considered as one of the earliest explorations of facial muscular activities systematization. The mimicry covers the additional expressive movements of gestures and postures, which are the characteristic manifestations of emotional states (Hjortsjö, 1970). Hjortsjö also described the facial expressions which are related to twenty four emotions, and divided these expressions into eight categories (Hjortsjö, 1970).

Another important development of anatomically based coding systems is Facial Action Coding System (FACS). FACS is developed by Paul Ekman and Wallace Friesen in 1978. They breaks down facial actions into small parts, which is called action units (AUs). Each of them can be considered as the basic elements of facial expressions. With the combination of different AUs, people can make different kinds of facial expressions based on muscle movements. At first facial action coding system is designed for motion records (Ekman, Friesen and Hager, 2002). However, it has now been widely applied into film industry and computer animation for decades (Parke and Waters, 2020, p. 33).

Micro-expressions:

Micro-expression is considered as a kind of more typical facial expression of emotion. The definition of micro-expression is still not clear. Mark G. Frank and Elena Svetieva tend to define micro-expression as any expression of emotion that is shown at 0.5 s or less because previous research had suggested that most of spontaneous expressions of emotion lasts between 0.5 and 4 (or 5) seconds (Mandal and Awasthi, 2015, p. 229). A distinctive feature of micro-expressions is that they reveal true emotional states in the form of facial expression for a very short period of time (0.5s or less), which is then quickly disguised or suppressed by another one.

The relationship between micro-expression and deception has been analyzed by some scholars. Some of the research focusing on the difference between real smile and fake smile. Scientists found that the main difference between real smile and fake smile is the muscle movements around eye area (Duchenne, 1990). Although muscles on the mouth corners are pulled up in both of these kinds of smiles, only the real smile will trigger the movements of orbicularis oculi muscle around the eyes (Duchenne, 1990). Additionally, according to DePaulo’s research, the facial pleasantness of liars is much lower than normal level. Liars will have more chin raises, more lip pressing, and look more nervous (DePaulo et al. 2003). However, other facial expressions like smiling, eyebrow lowering or raising have not shown consistently significant effect sizes (DePaulo et al. 2003).

4.Research Design, method, and schedule:

Based on quantitative research method, combined with the existing research data on facial expressions of other scholars in biology, psychology and other subjects, this paper will summarize micro-expressions and facial expressions of non-basic emotions, and conduct a more detailed analysis of the theory of facial expression animation. This paper will base on the theory of FACS (Facial Action Coding System), research data from scholars like Bella M. DePaulo, Paul Ekman and Carl-Herman Hjortsjö, and films and animations focusing on facial expression animation applications, classify and design the system of micro-expression and non-basic emotional facial animations in practical applications.

The research will firstly review the data collected by other scholars and visual materials in animation industry, and classify the form of micro-expression and non-basic facial expressions. The research will then focusing on building a system for the practical application of micro-expression and non-basic facial expressions theory in 3D animation. To test the system and methods of micro-expression and non-basic expression animations, the research will run experiments to collect data, test technical feasibility and improve the design. These experiments will focusing on some topics like the recognition of micro-expression in facial animation or the readability of deception in facial animation.

The difficulty of supplementing facial animation theory is how to clearly classify supplementary content, which requires detailed cross-research on various types of data. In addition, how to effectively realize these micro-expressions and specific emotions in animations is also one of the difficulties, which requires the design of new facial animation system base on recent facial animation and rigging technology.

5.Supposition and Implication:

This research can provide basic directions and data for future research on micro-expression and complex expression animation, and provide animation and some rigging solutions for these types of expression animation. The research on micro-expression and non-basic facial expression animation from the perspectives of psychology and biology can provide new ideas for 3d animation, and explore the possibility of making more convincing facial expression or performance animations from both artistic and scientific perspectives. Recently, animation techniques and rules cannot manage to achieve some facial expressions like micro-expressions or expressions in specific situations like lying, disguising real emotions, etc. Based on 3d facial animation techniques and the research of facial expression and micro-expression in other scientific areas, this paper will provide solutions to some of the problems above to a certain extent. The research content of facial expression animation in this paper can be used in film or film-based game animation areas. The theory can also be applied to animation refinement after motion capture to increase more performance details.

6.Conclusion:

In conclusion, the multidisciplinary based research on micro-expression and facial expression animations provides usable materials for facial animations which focusing on non-basic emotions in 3D animation. It can be foreseen that the solution of micro-expression and non-basic facial expression is different with the solution of basic emotion in 3D animation. The research on micro-expression and facial expression animation of specific emotions effectively adds more details to facial animation of virtual characters. The feasibility and practicability of the new system have been confirmed.

There are two areas where the work in this paper could be extended. First of all, the relationship among micro-expressions, facial animations and body language in character animations. Body language, or gesture expressions, can be considered as one of the most important parts of character performance animations, which can also be explored and researched further by combining traditional animation techniques with other disciplines such as psychology, biology, and sociology. Due to space limitations, the research did not delve into the body language in animation and its connection with micro-expression and facial expression animation. Additionally, the relationship between audio and facial expressions. Human emotions affect both the voice and behaviour. Under the premise of the same emotion, different levels of emotional expression will significantly affect the intonation and tone. Therefore, it is possible that animators can manipulate facial details and magnitude to show expressions in different levels based on the intonation and tone.

7.Bibliography:

Carl-Herman Hjortsjö (1970). Man’s face and mimic language. Studentlitteratur.

DePaulo, B. M., Lindsay, J. J., Malone, B. E., Muhlenbruck, L., Charlton, K., & Cooper, H. (2003). “Cues to deception”, Psychological Bulletin, pp. 74–118.

Duchenne, G. B. (1990). The Mechanism of Human Facial Expression. Edited by R. A. Cuthbertson. Cambridge: Cambridge University Press (Studies in Emotion and Social Interaction). doi: 10.1017/CBO9780511752841.

Ekman, P., Friesen, W.V. and Hager, J.C. (2002). Facial action coding system. Salt Lake City: Research Nexus.

Ekman, P., Friesen, W. V., O’Sullivan, M., Chan, A., Diacoyanni-Tarlatzis, I., Heider, K., et al. (1987). “Universals and cultural differences in the judgments of facial expressions of emotion”, Journal of Personality and Social Psychology, 53(4), pp. 712–717.

Mandal, M.K. and Awasthi, A. (2015). Understanding Facial Expressions in Communication: Cross-cultural and Multidisciplinary Perspectives. New Delhi: Springer India.

Osipa, J. (2010). Stop staring facial modeling and animation done right. Indianapolis, Ind Sybex.

Parke, F.I. and Waters, K. (2020). Computer facial animation. Boca Raton: Crc Press.

Russell, J. A. (1994). “Is there universal recognition of emotion from facial expression? A review of the cross-cultural studies”, Psychological Bulletin, pp. 102–141.‌

Russell, J.A. and José Miguel Fernández-Dols (2002). The psychology of facial expression. Cambridge: Cambridge University Press.