Facial Animation and Modeling

Computer Graphics Group, MPI Informatik, Saarbrücken, Germany

Introduction

Animation of human faces has been an active topic of research in Computer Graphics since the 1970's. With the increasing computational power and graphics capabilities of modern hardware, it becomes more and more feasible to use complex simulation techniques in an interactive environment. Our research aims mainly at exploration of the possibilities for anatomically correct modeling and physics-based simulation in such a setting.

Medusa

Medusa is a system for Facial Modeling and Animation. It includes tools and techniques for a variety of sub-tasks that have to be handled in order to generate photorealistic facial animations in real-time on standard PC hardware. Medusa is used both for research and education purposes. The initiators of Medusa, Jörg Haber and Kolja Kähler, have been awarded the Heinz-Billing-Preis 2001 of the Max-Planck-Society and the SaarLB Science Award 2001 of the Saarland Ministry of Education, Culture, and Sciences.

Research Topics

Data acquisition: To obtain the facial geometry of real individuals, we use a range scanning device. Our experiences have shown that post-processing of the resulting set of surface samples takes significant time due to missing data and noise artifacts. It is thus necessary to automate this procedure as much as possible.
Muscle models: The appearance of the human face in static poses and in motion is determined to a great deal by the underlying muscle layer. Facial expressions are formed by orchestrated contraction of muscles. Built from strands of individual fibers, facial muscles appear in various shapes and form a complex network, sliding across and merging into each other. Accurate modeling and simulation of these motivators of facial expression is a challenging task.
Simulation of skin: The skin is composed of several layers of tissue, each of these having specific biomechanical properties, such as elasticity and viscosity. Physics-based animation systems approximate these properties to mimic the important characteristics of facial skin, such as bulging and formation of expressive wrinkles and furrows. The computational complexity of this task enforces a trade-off between accuracy, stability and speed of simulation in an interactive system.
Textures and rendering: For fast real-time rendering, rather coarse triangle meshes have to be used. Visual appearance can be improved by using not only textures from photographs, but exploiting the capabilities of current graphics boards for simulation of skin structure and reflection properties.
Multiresolution surfaces: For accurate representation of geometric detail, displaced subdivision surfaces can be employed. Exploiting the multiresolution hierarchy, skin/tissue simulation can be done quickly on a coarse base mesh. The finally rendered surface is then reconstructed on that deformed mesh.
Speech synchronization: An important topic in facial animation is the generation of lip movements corresponding to recorded speech. In general, phonemes are mapped to muscle activation values which produce the desired mouth shapes (visemes). Due to the complex interactions between consecutive phonemes in human speech (coarticulation), blending between visemes in a convincing manner is non-trivial. Also, the emotional state of the speaker should be reflected in the animation.
Anthropometric modeling: Deformation of head models according to anthropometric data enables us to simulate human head growth based on statistical measurements that have been collected over decades for populations varying in age, sex, and ethnicity. In comparison to standard geometric operators, modeling by modification of proportions and facial attributes promises greatly eased creation of realistic virtual heads.

Publications

V. Blanz, I. Albrecht, J. Haber, H.-P. Seidel: Creating Face Models from Vague Mental Images, Computer Graphics Forum (Proc. Eurographics 2006), 4-8 September 2006, to appear. PDF version
I. Albrecht, M. Schröder, J. Haber, H.-P. Seidel: Mixed feelings -- Expression of non-basic emotions in a muscle-based talking head, Virtual Reality (Special Issue "Language, Speech and Gesture for VR"), August 2005.
H. Yamauchi, H. Lensch, J. Haber, H.-P. Seidel: Textures Revisited, The Visual Computer 21(4), pp. 217-241, May 2005. PDF version
M. Koster, J. Haber, H.-P. Seidel: Real-Time Rendering of Human Hair using Programmable Graphics Hardware, Proceedings of Computer Graphics International 2004 (CGI 2004), June 2004, pp. 248-256. PDF version
T. Fuchs, J. Haber, H.-P. Seidel: MIMIC -- A Language for Specifying Facial Animations, Proceedings of WSCG 2004, 2-6 Feb 2004, pp. 71-78. PDF version
K. Kähler, J. Haber, H.-P. Seidel: Reanimating the Dead: Reconstruction of Expressive Faces from Skull Data, ACM Transactions on Graphics (SIGGRAPH 2003 Conference Proceedings), 22(3), July 2003, pp. 554-561. PDF version
K. Kähler, J. Haber, H.-P. Seidel: Dynamically refining animated triangle meshes for rendering, The Visual Computer, 19(5), pp. 310-318, August 2003. PDF version
I. Albrecht, J. Haber, K. Kähler, M. Schröder, H.-P. Seidel: "May I talk to you? :-)" -- Facial Animation from Text, Proceedings Pacific Graphics 2002, 9-11 Oct 2002, pp. 77-86. PDF version
K. Kähler, J. Haber, H. Yamauchi, H.-P. Seidel: Head shop: Generating animated head models with anatomical structure, Proceedings ACM SIGGRAPH Symposium on Computer Animation (SCA) 2002, 21-22 July 2002, pp. 55-64. PDF version
I. Albrecht, J. Haber, H.-P. Seidel: Automatic Generation of Non-Verbal Facial Expressions from Speech, Proceedings Computer Graphics International (CGI) 2002, 3-5 July 2002, pp. 283-293. PDF version
M. Tarini, H. Yamauchi, J. Haber, H.-P. Seidel: Texturing Faces, Proceedings Graphics Interface 2002, 27-29 May 2002, pp. 89-98. PDF version
W.-K Jeong, K. Kähler, J. Haber, H.-P. Seidel: Automatic Generation of Subdivision Surface Head Models from Point Cloud Data, Proceedings Graphics Interface 2002, 27-29 May 2002, pp. 181-188. PDF version
I. Albrecht, J. Haber, H.-P. Seidel: Speech Synchronization for Physics-based Facial Animation, Proceedings WSCG 2002, 4-8 Feb 2002, pp. 9-16. PDF version
J. Haber, K. Kähler, I. Albrecht, H. Yamauchi, H.-P. Seidel: Face to Face: From Real Humans to Realistic Facial Animation, Proceedings Israel-Korea Binational Conference on Geometrical Modeling and Computer Graphics 2001, 11-12 Oct 2001, pp. 73-82. PDF version
K. Kähler, J. Haber, H.-P. Seidel: Geometry-based Muscle Modeling for Facial Animation, Proceedings Graphics Interface 2001, 7-9 June 2001, pp. 37-46. PDF version
K. Kähler, J. Haber, H.-P. Seidel: Dynamic Refinement of Deformable Triangle Meshes for Rendering, Proceedings Computer Graphics International (CGI) 2001, 3-6 July 2001, pp. 285-290. PDF version

Snapshots / Movies

Please follow this link for more information and demo material regarding our facial animation system.

Contact:

Jörg Haber	Volker Blanz	Kolja Kähler	Hitoshi Yamauchi	Irene Albrecht
haberj_at_mpi-sb.mpg.de	blanz_at_mpi-sb.mpg.de	kaehler_at_mpi-sb.mpg.de	hitoshi_at_mpi-sb.mpg.de	albrecht_at_mpi-sb.mpg.de

Max-Planck-Institut für Informatik Stuhlsatzenhausweg 85 66123 Saarbrücken Germany

[AG4] [MPI homepage]