GazeStereo3D: Seamless Disparity Manipulations, SIGGRAPH 2016

GazeStereo3D: Seamless Disparity Manipulations

Petr Kellnhofer^1,2     Piotr Didyk^2,3     Karol Myszkowski²     Mohamed M. Hefeeda⁴     Hans-Peter Seidel²     Wojciech Matusik<sup>1

1</sup> MIT CSAIL     ² MPI Informatik     ³ Saarland University, MMCI     ⁴ Qatar Computing Research Institute

Beyond common disparity mapping (a) our approach adapts to attended regions such as the Armadillo (b) or Bunny (d) and modifies disparity in that region to enhance depth and reduce discomfort from the accomodation-vergence conflict (c,e). To that end we build non-linear remapping curves and use our novel perceptual model to ensure a seamless transition between them (f).

Abstract

Producing a high quality stereoscopic impression on current displays is a challenging task. The content has to be carefully prepared in order to maintain visual comfort, which typically affects the quality of depth reproduction. In this work, we show that this problem can be significantly alleviated when the eye fixation regions can be roughly estimated. We propose a new method for stereoscopic depth adjustment that utilizes eye tracking or other gaze prediction information. The key idea that distinguishes our approach from the previous work is to apply gradual depth adjustments at the eye fixation stage, so that they remain unnoticeable. To this end, we measure the limits imposed on the speed of disparity changes in various depth adjustment scenarios, and formulate a new model that can guide such seamless stereoscopic content processing. Based on this model, we propose a real-time controller that applies local manipulations to stereoscopic content to find the optimum between depth reproduction and visual comfort. We show that the controller is mostly immune to the limitations of low-cost eye tracking solutions. We also demonstrate benefits of our model in off-line applications, such as stereoscopic movie production, where skillful directors can reliably guide and predict viewers' attention or where attended image regions are identified during eye tracking sessions. We validate both our model and the controller in a series of user experiments. They show significant improvements in depth perception without sacrificing the visual quality when our techniques are applied.

Video

Side-by-side version (requires Flash YouTube player - more info)

Downloads

Citation

Petr Kellnhofer, Piotr Didyk, Karol Myszkowski, Mohamed M. Hefeeda, Hans-Peter Seidel, Wojciech Matusik
GazeStereo3D: Seamless Disparity Manipulations
ACM Transactions on Graphics 35(4) (Proc. SIGGRAPH 2016, Anaheim, California, USA)

@article{Kellnhofer2016,
  author = { 
	Petr Kellnhofer and
	Piotr Didyk and 
	Karol Myszkowski and 
	Mohamed M. Hefeeda and 
	Hans-Peter Seidel and 
	Wojciech Matusik},
  title = {GazeStereo3D: Seamless Disparity Manipulations},
  journal = {ACM Transactions on Graphics (Proc. SIGGRAPH)},
  year = {2016},
  volume = {35},
  number = {4},
  doi = {10.1145/2897824.2925866},
  url = {http://dx.doi.org/10.1145/2897824.2925866}
}

© 2016 The Authors. This is the author's version of the work. It is posted here for your personal use. Not for redistribution.
The definitive version was published in ACM Transactions on Graphics 35(4) (Proc. SIGGRAPH 2016). http://dx.doi.org/10.1145/2897824.2925866

Acknowledgements

We would like to thank Junaid Ali, Thomas Leimkühler, Alexandre Kaspar, Krzysztof Templin, Louise van den Heuvel, Tobias Ritschel, and the anonymous subjects who took part in our perceptual studies. This work was partially supported by the Fraunhofer and Max Planck cooperation program within the framework of the German pact for research and innovation (PFI).