What makes 2D-to-3D stereo conversion perceptually plausible?

Petr Kellnhofer Thomas Leimkühler Tobias Ritschel Karol Myszkowski Hans-Peter Seidel
MPI Informatik

Proceedings of ACM Symposium on Applied Perception 2015, Tübingen, 13-14th September 2015

Different from classic reconstruction of physical depth in computer vision, depth for 2D-to-3D stereo conversion is assigned by humans using semi-automatic painting interfaces and, consequently, is often dramatically wrong. Here we seek to better understand why it still does not fail to convey a sensation of depth. To this end, four typical disparity distortions resulting from manual 2D-to-3D stereo conversion are analyzed: i) smooth remapping, ii) spatial smoothness, iii) motion-compensated, temporal smoothness, and iv) completeness. A perceptual experiment is conducted to quantify the impact of each distortion on the plausibility of the 3D impression relative to a reference without distortion. Close-to-natural videos with known depth were distorted in one of the four above-mentioned aspects and subjects had to indicate if the distortion still allows for a plausible 3D effect. The smallest amounts of distortion that result in a significant rejection suggests a conservative upper bound on the quality requirement of 2D-to-3D conversion.

We intentionally introduce the depth distortions typically produced by 2D-to-3D conversion into close-to-natural computer generated images (a, top) such as the one from the MPI Sintel dataset [Butler2012] where ground truth depth is available (a, bottom). User response to stereo images (b--e, top) showing typical disparity distortions (b--e, bottom) gives an indication whether a certain amount of distortion results in functional equivalence for natural images or not. According to numerical measures such as PSNR or perceptual disparity metrics, the depth is considered very different (inequality sign, bottom), whereas it is functionally equivalent (equivalence sign, top).


You can view our stimuli online and we also offer our stimuli as a compressed zip package (88 MiB).


Paper (4.1 MiB).
SAP 2015 presentation slides (19.2 MiB).
Supplemental as zip (88 MiB).


Petr Kellnhofer, Thomas Leimkühler, Tobias Ritschel, Karol Myszkowski, Hans-Peter Seidel
What makes 2D-to-3D stereo conversion perceptually plausible?
ACM Symposium on Applied Perception 2015.

 author = {Kellnhofer, Petr and Leimk\"{u}hler, Thomas and Ritschel, Tobias and Myszkowski, Karol and Seidel, Hans-Peter},
 title = {What Makes 2D-to-3D Stereo Conversion Perceptually Plausible?},
 booktitle = {Proceedings of the ACM SIGGRAPH Symposium on Applied Perception},
 series = {SAP '15},
 year = {2015},
 isbn = {978-1-4503-3812-7},
 location = {T\"ubingen, Germany},
 pages = {59--66},
 numpages = {8},
 url = {http://doi.acm.org/10.1145/2804408.2804409},
 doi = {10.1145/2804408.2804409},
 acmid = {2804409},
 publisher = {ACM},
 address = {New York, NY, USA},

© The Authors, 2015. This is the author's version of the work. It is posted here for your personal use. Not for redistribution. The definitive version was published in the Proceedings of the ACM Symposium on Applied Perception 2015 (SAP 2015),

Copyright is held by the owner/author(s). Publication rights licensed to ACM. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from Permissions@acm.org.