A Transformation-Aware Perceptual Image Metric
IS&T/SPIE Electronic Imaging 2015,
extended version in
IS&T/SPIE Journal of Electronic Imaging 2016
A Transformation-Aware Perceptual Image Metric
Petr Kellnhofer
Tobias Ritschel
Karol Myszkowski
Hans-Peter Seidel
MPI Informatik
Abstract
Predicting human visual perception has several applications such as compression, rendering, editing and retargeting. Current approaches however, ignore the fact that the human visual system compensates for geometric transformations, e. g., we see that an image and a rotated copy are identical. Instead, they will report a large, false-positive difference. At the same time, if the transformations become too strong or too spatially incoherent, comparing two images indeed gets increasingly difficult. Between these two extrema, we propose a system to quantify the effect of transformations, not only on the perception of image differences, but also on saliency and motion parallax. To this end, we first fit local homographies to a given optical flow field and then convert this field into a field of elementary transformations such as translation, rotation, scaling, and perspective. We conduct a perceptual experiment quantifying the increase of difficulty when compensating for elementary transformations. Transformation entropy is proposed as a novel measure of complexity in a flow field. This representation is then used for applications, such as comparison of non-aligned images, where transformations cause threshold elevation, detection of salient transformations, and a model of perceived motion parallax. Novel applications of our approach are a perceptual level-of-detail for real-time rendering and viewpoint selection based on perceived motion parallax.
Downloads
|
Citation
Extended JEI 2016 Paper
Petr Kellnhofer, Tobias Ritschel, Karol Myszkowski, Hans-Peter Seidel
A Transformation-Aware Perceptual Image Metric
Journal of Electronic Imaging, 2016.
@proceeding{Kellnhofer2016jei,
author = {Kellnhofer, Petr and Ritschel, Tobias and Myszkowski, Karol and Seidel, Hans-Peter},
title = {Transformation-aware perceptual image metric},
journal = {Journal of Electronic Imaging},
volume = {25},
number = {5},
pages = {053014},
abstract = {
Predicting human visual perception has several applications such as compression, rendering,
editing, and retargeting. Current approaches, however, ignore the fact that the human visual
system compensates for geometric transformations, e.g., we see that an image and a rotated
copy are identical. Instead, they will report a large, false-positive difference.
At the same time, if the transformations become too strong or too spatially incoherent,
comparing two images gets increasingly difficult. Between these two extrema, we propose
a system to quantify the effect of transformations, not only on the perception of image
differences but also on saliency and motion parallax. To this end, we first fit local
homographies to a given optical flow field, and then convert this field into a field
of elementary transformations, such as translation, rotation, scaling, and perspective.
We conduct a perceptual experiment quantifying the increase of difficulty when compensating
for elementary transformations. Transformation entropy is proposed as a measure of complexity
in a flow field. This representation is then used for applications, such as comparison
of nonaligned images, where transformations cause threshold elevation, detection of salient
ransformations, and a model of perceived motion parallax. Applications of our approach are
a perceptual level-of-detail for real-time rendering and viewpoint selection based
on perceived motion parallax.
},
year = {2016},
isbn = {1017-9909},
doi = {10.1117/1.JEI.25.5.053014},
URL = { http://dx.doi.org/10.1117/1.JEI.25.5.053014},
eprint = {}
}
Conference IS&T/SPIE EI 2015 Paper
Petr Kellnhofer, Tobias Ritschel, Karol Myszkowski, Hans-Peter Seidel
A Transformation-Aware Perceptual Image Metric
IS&T/SPIE Electronic Imaging, Human Vision and Electronic Imaging XX, February 2015.
@proceeding{Kellnhofer2015SPIE,
author = {Petr Kellnhofer and Tobias Ritschel and Karol Myszkowski and Hans-Peter Seidel},
title = {A Transformation-Aware Perceptual Image Metric},
journal = {Proc. SPIE},
volume = {9394},
number = {},
pages = {939408-939408-14},
abstract = {
Predicting human visual perception of image differences has several applications such as compression,
rendering, editing and retargeting. Current approaches however, ignore the fact that the human visual
system compensates for geometric transformations, e.g. we see that an image and a rotated copy are
identical. Instead, they will report a large, false-positive difference. At the same time, if the
transformations become too strong or too spatially incoherent, comparing two images indeed gets
increasingly difficult. Between these two extremes, we propose a system to quantify the effect
of transformations, not only on the perception of image differences, but also on saliency and motion
parallax. To this end, we first fit local homographies to a given optical flow field and then convert
this field into a field of elementary transformations such as translation, rotation, scaling, and
perspective. We conduct a perceptual experiment quantifying the increase of difficulty when compensating
for elementary transformations. Transformation entropy is proposed as a novel measure of complexity
in a flow field. This representation is then used for applications, such as comparison of non-aligned
images, where transformations cause threshold elevation, and detection of salient transformations.
},
year = {2015},
doi = {10.1117/12.2076754},
URL = { http://dx.doi.org/10.1117/12.2076754},
eprint = {}
}
Petr Kellnhofer, Tobias Ritschel, Karol Myszkowski, Hans-Peter Seidel,
"A Transformation-Aware Perceptual Image Metric"
IS&T/SPIE Journal of Electronic Imaging (2016).
Petr Kellnhofer, Tobias Ritschel, Karol Myszkowski, Hans-Peter Seidel,
"A Transformation-Aware Perceptual Image Metric"
IS&T/SPIE Electronic Imaging, Editors, Volume (Issue) 9394, Page 14, (2015).
Copyright 2015 nad 2016 Society of Photo Optical Instrumentation Engineers. One print or electronic copy may be made for personal use only. Systematic reproduction and distribution, duplication of any material in this paper for a fee or for commercial purposes, or modification of the content of the paper are prohibited.
Link to SPIE's official online version: http://proceedings.spiedigitallibrary.org/proceeding.aspx?articleid=2209714 and http://proceedings.spiedigitallibrary.org/proceeding.aspx?articleid=2209714