Colecturers: Kiran Varanasi, Kwang In Kim
Since the advent of photography, scientists and engineers have been studying ways to capture the real world in a richer manner that permits new applications. This endeavour has been revolutionized by the development of computers. The twin fields of computer vision and computer graphics have discovered various techniques to capture the real world into rich mathematical models, that allow the user to not only revisualize the scene but also to interact with it. This has enabled a variety of groundbreaking applications, in particular in computer graphics or animation, where photorealistic virtual versions of static and dynamic realworld scenes can be displayed. To reach this goal, several aspects of the real world scene have to be captured, including geometry, reflectance and motion. In this course we will review fundamental methods from computer vision and graphics to capture such aspects of real world scenes. We will enrich the disucssion of basic methods by providing examples of the research in our own group on the topics of static and dynamic scene capture.
The course is given as a class room lecture. There will be 90 miuntes of lecture every week. The following is needed to pass the course:
This is an advaned specialized lecture in visual computing that is mainly geared towards senior undrgraduate students and graduate students. The medium of instruction and communication is English. We expect that the students have taken courses in graphcis and / or vision. Basic knowledge in linear algebra, computer graphics and image processing are required. We briefly revise the fundamentals in these areas in the course, but a more thorough introduction is not possible. An aptitude for programming and good skills in the C++ programming language are absolutely necessary. We shall provide the students with the ennvironment for project developement : Kinect cameras, powerful computer workstations with Microsoft Kinect SDK, and tutorials on how to use the system. A few tutorials are also provided on key ideas of the course. But the students are expected to be proactive and make up for the shortcomings of their backgrounds by themselves.
A more detailed introduction to the course will be given in a preparatory meeting on October 18, 6 pm in room MPI 024.
In case you want to take the course, please register by sending an email to Kiran Varanasi  varanasi (at) mpii.de. This should be done at the latest by October 27.
Attendance is required in all the lectures and tutorials.
Course Instructors : Christian Theobalt, Kiran Varanasi, KwangIn Kim
Teaching Assistants : Miguel Granados, Chenglei Wu, Ahmed Elhayek
Meeting Hours : Kiran (Tuesdays 17h) Office 223, MPI Building
In case you have further questions about the course, please contact Kiran Varanasi  varanasi (at) mpiinf.mpg.de .
Once the course becomes operational, all the students should subscribe to the following mailing list, where all major announcements regarding the course are made : https://lists.mpiinf.mpg.de/listinfo/crcourse
An additional mailing list is provided for more informal discussions between the students about the programming projects. All the students are suggested to register as well to : https://lists.mpiinf.mpg.de/listinfo/crcourseproject
Lectures :  Thursdays 14:0016:00 

Tutorials :  Tuesdays 18:0020:00 
First meeting :  October 18, 2011 at 18:00 hrs 
Room:  Building E1.4 (MaxPlanckInstitut), Room 024 
Registration:  open 
Date  Topic  Presenter  Description  Slides 

October 18 
Introductory Meeting Lab tour & Tutorial : Microsoft Kinect 
Christian Theobalt Michal Richter 
The contents and objectives of the course are outlined. Formalities about using the lab are given. A tutorial on using the Microsoft Kinect SDK is presented. 

October 20 
Maths Intro 1 
KwangIn Kim 
Fundamental topics of linear matrix algebra are revised. 

October 27 
Physics of the camera 
Kiran Varanasi 
The physical process of imaging by a digital camera is explained. The fundamental aesthetics of photography are explained from a mathematical point of view. Assignment 1 on "mathematical background" is handed out. 

November 3 
Camera geometry & calibration 
Kiran Varanasi 
The mathematical model of a pinhole camera is explained via projective geometry. Image projection from a perspective camera is described as a matrix operation. The process of calibrating internal and external parameters of a camera is described. 

November 10 
Maths Intro 2 
KwangIn Kim 
Basic numerical methods for optimization and regression are revised. 

November 15 
Tutorial : Project Proposals 
Each group gives a short presentation about their project proposal. Questions related to each project are discussed. 


November 17 
Structure from motion 
Kiran Varanasi 
The geometrical relationships between multiple cameras imaging the same scene are described. The method behind "matchmoving", i.e, automatic estimation of camera pose for visual effects in films, is described. Assignment 1 is handed in. Assignment 2 on "camera geometry" is handed out. 

November 22 
Tutorial : Mathematical background 
Miguel Granados 
Solutions to assignment 1 are presented. Questions on mathematical background are answered. 

November 24 ** Shifted to Room 019 ** 
Light transport 
Christian Theobalt 
Various models for surface reflectance of light are described. The plenoptic function for light transport in a medium, and its use in computer graphics, is explained. 

December 1 
2D image features & matching 
KwangIn Kim 
Fundamentals of digital image processing  edge & corner detection are revised. 2D image features based on photometric gradients are described. Modeling spatial relationship between pixels in an image is described. Methods for matching features consistently across images are reviewed. Assignment 2 is handed in. Assignment 3 on "feature correspondence" is handed out. 

December 6 
Tutorial : Camera geometry 
Chenglei Wu 
Solutions to assignment 2 are presented. Questions on projective geometry & camera models are answered. 
Solution Sheet : Assignment 2 
December 8 
Optic flow & scene flow 
KwangIn Kim 
The notion of optical flow is introduced for the analysis of a videostream of data. Variational methods for the computation of the flow are described. 

December 15 
3D scanning 
Christian Theobalt 
Alternative means of capture through 3D scanners are introduced. Techniques for the analysis and registration of 3D point data are described. Assignment 3 is handed in. 

December 20 
Tutorial : Feature correspondence 
Ahmed Elhayek 
Solutions to assignment 3 are presented. Questions on using OpenCV, matching features and computing optic flow are answered. 

December 22 
Shape from silhouette 
Kiran Varanasi 
3D reconstruction from image silhouettes is described through the notion of the visual hull. The practical aspects governing a multicamera studio, that facilitate such capture, are described. 

January 10 
Tutorial : Interim Project Status Report 
Each group gives a short presentation about the status of their project. Questions related to how certain problems faced within can be solved are discussed. 


January 12 
Shape from stereo 
Kiran Varanasi 
Practical algorithms that reconstruct dense 3D shapes via multicamera stereo are described. 

January 19 
Skeletal motion capture 
Christian Theobalt 
The field of motion capture for animation is reviewed. Motion parametrization through a skeletal hierarchy is described. The various phases in markerbased motion capture are studied. 

January 26 
Shape from Shading 
Kiran Varanasi 
Shading is a fundamental cue for reconstructing 3D shape from an image. The concepts of shape from shading and photometric stereo are described. The construction of a light stage for imaging a dynamic subject under different lighting conditions is described. The application of relighting the shape under new lighting conditions is studied. 

February 2 
Markerless Motion Capture 
Christian Theobalt 
Visionbased techniques for motion capture (mocap) without the use of markers are described. The statistical modeling of human shape and pose variations, to be used as priors for mocap, is reviewed. Realtime methods for efficient and accurate mocap are described. 

February 9 
Performance Capture 
Christian Theobalt 
Performance capture means a much denser description of structure and motion than is possible by skeletal pose. For example, subtle muscle deformations and dynamic folds in cloth can be accurately captured as 3D meshes. The underlying theory of mesh deformation and visual analysis for performance capture is described. The challenges in modeling arbitrary 3D scenes are reviewed and addressed. 

February 16 
Demo sessions 
 
Each group gives a live demo of their project. 

February 23 
Final Exam 

Each student will be tested over the material covered in the lectures by an oral examination. 
