max planck institut informatik
mpii logo Minerva of the Max Planck Society

Real-time Reflective and Refractive Novel-view Synthesis, Vision, Modeling, and Visualization 2014

Interactive By-example Design of Artistic Packing Layouts

Gerrit Lochmann1     Bernhard Reinert2     Tobias Ritschel2, 3     Stefan Müller1     Hans-Peter Seidel2

Universität Koblenz-Landau1, MPI Informatik2, Saarland Univerity3

Novel-view images containing planar and curved reflections and refractions that are correctly warped with our algorithm in real-time. Bottom left: The visual flow between the initial- and the novel-view on the diffuse layer.


We extend novel-view image synthesis from the common diffuse and opaque image formation model to the reflective and refractive case. Our approach uses a ray tree of RGBZ images, where each node contains one RGB light path which is to be warped differently depending on the depth Z and the type of path. Core of our approach are two efficient procedures for reflective and refractive warping. Different from the diffuse and opaque case, no simple direct solution exists for general geometry. Instead, a per-pixel optimization in combination with informed initial guesses warps an HD image with reflections and refractions in 18 ms on a current mobile GPU. The key application is latency avoidance in remote rendering in particular for head-mounted displays. Other applications are single-pass stereo or multi-view, motion blur and depth-of-field rendering as well as their combinations.



Paper Full Quality (15 MB) or Compressed (2.1 MB).
Video (49 MB)


Gerrit Lochmann, Bernhard Reinert, Tobias Ritschel, Stefan Müller, Hans-Peter Seidel
Real-time Reflective and Refractive Novel-view Synthesis
Vision, Modeling, and Visualization (2014).

	author = {Gerrit Lochmann and Bernhard Reinert and Tobias Ritschel and Stefan Müller and Hans-Peter Seidel },
	title = {{Real-time Reflective and Refractive Novel-view Synthesis}},
	pages = {9-16},
	URL = {},
	DOI = {10.2312/vmv.20141270},
	editor = {Jan Bender and Arjan Kuijper and Tatiana von Landesberger and Holger Theisel and Philipp Urban},
	title = {VMV 2014: Vision, Modeling & Visualization},
	year = {2014},
	isbn = {978-3-905674-74-3},
	issn = {-},
	address = {Darmstadt, Germany},
	publisher = {Eurographics Association}

© EUROGRAPHICS 2014. This is the author's version of the work. It is posted here for your personal use. Not for redistribution. The definitive Version of Record was published in EUROGRAPHICS European Associaction for Computer Graphics - VMV 2014: Vision, Modeling & Visualization - Inproceedings.