On Evaluation of Video Quality Metrics: an HDR Dataset for Computer Graphics Applications

Martin Čadík, mcadik@mpi-inf.mpg.de
Tunc Ozan Aydin, tunc@mpi-inf.mpg.de
Karol Myszkowski, karol@mpi-inf.mpg.de
Hans-Peter Seidel, hpseidel@mpi-inf.mpg.de


In this paper we propose a new dataset for evaluation of image/video quality metrics with emphasis on applications in computer graphics. The proposed dataset includes LDR-LDR, HDR-HDR, and HDR-LDR reference-test video pairs with various types of distortions. We also present an example evaluation of recent image and video quality metrics that were applied in the field of computer graphics. In this evaluation all video sequences were shown on an HDR display, and subjects were asked to mark the regions where they saw differences between test and reference videos. As a result, we capture not only the magnitude of distortions, but also their spatial distribution. This has two advantages: on one hand the local quality information is valuable for computer graphics applications, on the other hand the subjectively obtained distortion maps are easily comparable to the maps predicted by quality metrics.


[Paper (pdf)]
[Presentation slides (pdf)]
[bibTeX entry (bib)]
[HDR Dataset (complete dataset, 905MB zip file)]
[Browse HDR Dataset]

Acknowledgements and Credits: the presented dataset should not be used for commercial purposes without our explicit permission. Please acknowledge the use of the dataset by citing the publication.