ROC and correlation analysis

The plots below illustrate the performance of the metrics for all scenes analyzed together and for each scene separately.

auc@25, auc@50, auc@75 is the area under the ROC assuming that more than 25%, 50% or 75% of observers mark the region.

rho is Spearman's non-parametric coefficient of correlation. The correlation is computed only for the regions that were selected by more than 5% and less than 95% of observers, as explained in the paper.

Hover the mouse over the test image to see the reference image.

Hover the mouse over the result of a metric to see the visualization of false positives and false negatives. False positives are marked with red, false negatives with green and true positives with blue. The threshold was selected so that for all scenes the true-postitive rate is equal 75% for the case where at least 50% of observers mark the distortion.

Group: all

With-reference

Metric auc@25 auc@50 auc@75 rho AD 0.828 0.817 0.835 0.255 SSIM 0.798 0.782 0.792 0.17 MS-SSIM 0.736 0.723 0.738 0.077 HDR-VDP-2 0.774 0.751 0.738 0.128 sCIE-Lab 0.808 0.793 0.8 0.186 sCorrel 0.872 0.886 0.918 0.452

No-reference

Metric auc@25 auc@50 auc@75 rho AD 0.789 0.801 0.832 0.302 SSIM 0.754 0.744 0.832 0.264 MS-SSIM 0.686 0.647 0.787 0.177 HDR-VDP-2 0.741 0.753 0.841 0.252 sCIE-Lab 0.756 0.728 0.831 0.256 sCorrel 0.843 0.86 0.871 0.43

appartment_II - test image	With-reference Metric auc@25 auc@50 auc@75 rho AD 0.992 0.977 0.949 0.556 SSIM 0.987 0.97 0.937 0.452 MS-SSIM 0.977 0.961 0.933 0.397 HDR-VDP-2 0.963 0.938 0.934 0.187 sCIE-Lab 0.993 0.98 0.955 0.611 sCorrel 0.992 0.982 0.954 0.651	No-reference Metric auc@25 auc@50 auc@75 rho AD 0.945 0.972 0.983 0.582 SSIM 0.938 0.96 0.98 0.523 MS-SSIM 0.941 0.972 0.988 0.514 HDR-VDP-2 0.953 0.956 0.956 0.599 sCIE-Lab 0.942 0.971 0.988 0.574 sCorrel 0.938 0.949 0.974 0.575	Mean observer data - with reference experiment AD - distortion map SSIM - distortion map MS-SSIM - distortion map HDR-VDP-2 - distortion map sCIE-Lab - distortion map sCorrel - distortion map
atrium - test image	With-reference Metric auc@25 auc@50 auc@75 rho AD 0.954 0.956 0.944 0.755 SSIM 0.932 0.933 0.951 0.697 MS-SSIM 0.894 0.894 0.883 0.593 HDR-VDP-2 0.94 0.928 0.907 0.653 sCIE-Lab 0.958 0.951 0.94 0.719 sCorrel 0.936 0.931 0.876 0.618	No-reference Metric auc@25 auc@50 auc@75 rho AD 0.955 NaN NaN 0.706 SSIM 0.916 NaN NaN 0.646 MS-SSIM 0.913 NaN NaN 0.654 HDR-VDP-2 0.907 NaN NaN 0.609 sCIE-Lab 0.952 NaN NaN 0.692 sCorrel 0.934 NaN NaN 0.581	Mean observer data - with reference experiment AD - distortion map SSIM - distortion map MS-SSIM - distortion map HDR-VDP-2 - distortion map sCIE-Lab - distortion map sCorrel - distortion map
bathroom - test image	With-reference Metric auc@25 auc@50 auc@75 rho AD 0.979 0.93 0.865 0.699 SSIM 0.957 0.906 0.871 0.674 MS-SSIM 0.97 0.932 0.87 0.683 HDR-VDP-2 0.957 0.858 0.796 0.527 sCIE-Lab 0.983 0.933 0.866 0.686 sCorrel 0.978 0.946 0.913 0.759	No-reference Metric auc@25 auc@50 auc@75 rho AD 0.923 0.849 0.884 0.662 SSIM 0.897 0.79 0.798 0.581 MS-SSIM 0.927 0.743 0.721 0.607 HDR-VDP-2 0.863 0.895 0.951 0.547 sCIE-Lab 0.918 0.735 0.724 0.563 sCorrel 0.929 0.737 0.782 0.616	Mean observer data - with reference experiment AD - distortion map SSIM - distortion map MS-SSIM - distortion map HDR-VDP-2 - distortion map sCIE-Lab - distortion map sCorrel - distortion map
buddha - test image	With-reference Metric auc@25 auc@50 auc@75 rho AD 0.901 0.822 NaN 0.325 SSIM 0.892 0.782 NaN 0.312 MS-SSIM 0.885 0.743 NaN 0.339 HDR-VDP-2 0.838 0.794 NaN -0.074 sCIE-Lab 0.9 0.819 NaN 0.304 sCorrel 0.852 0.825 NaN 0.207	No-reference Metric auc@25 auc@50 auc@75 rho AD 0.768 NaN NaN 0.078 SSIM 0.739 NaN NaN 0.037 MS-SSIM 0.736 NaN NaN 0.122 HDR-VDP-2 0.789 NaN NaN 0.02 sCIE-Lab 0.806 NaN NaN 0.093 sCorrel 0.792 NaN NaN 0.155	Mean observer data - with reference experiment AD - distortion map SSIM - distortion map MS-SSIM - distortion map HDR-VDP-2 - distortion map sCIE-Lab - distortion map sCorrel - distortion map
chairs - test image	With-reference Metric auc@25 auc@50 auc@75 rho AD 0.961 0.975 0.952 0.653 SSIM 0.936 0.936 0.946 0.577 MS-SSIM 0.96 0.965 0.959 0.606 HDR-VDP-2 0.929 0.924 0.918 0.377 sCIE-Lab 0.963 0.965 0.943 0.571 sCorrel 0.957 0.964 0.954 0.531	No-reference Metric auc@25 auc@50 auc@75 rho AD 0.969 0.932 NaN 0.727 SSIM 0.925 0.875 NaN 0.612 MS-SSIM 0.959 0.889 NaN 0.646 HDR-VDP-2 0.913 0.815 NaN 0.512 sCIE-Lab 0.958 0.901 NaN 0.655 sCorrel 0.963 0.916 NaN 0.648	Mean observer data - with reference experiment AD - distortion map SSIM - distortion map MS-SSIM - distortion map HDR-VDP-2 - distortion map sCIE-Lab - distortion map sCorrel - distortion map
city-d - test image	With-reference Metric auc@25 auc@50 auc@75 rho AD 0.922 NaN NaN 0.09 SSIM 0.895 NaN NaN 0.067 MS-SSIM 0.817 NaN NaN 0.059 HDR-VDP-2 0.792 NaN NaN 0.041 sCIE-Lab 0.892 NaN NaN 0.057 sCorrel 0.929 NaN NaN 0.079	No-reference Metric auc@25 auc@50 auc@75 rho AD 0.831 NaN NaN -0.061 SSIM 0.853 NaN NaN -0.146 MS-SSIM 0.82 NaN NaN -0.265 HDR-VDP-2 0.791 NaN NaN -0.252 sCIE-Lab 0.814 NaN NaN -0.156 sCorrel 0.858 NaN NaN -0.043	Mean observer data - with reference experiment AD - distortion map SSIM - distortion map MS-SSIM - distortion map HDR-VDP-2 - distortion map sCIE-Lab - distortion map sCorrel - distortion map
city-u - test image	With-reference Metric auc@25 auc@50 auc@75 rho AD 0.942 0.911 NaN 0.738 SSIM 0.9 0.851 NaN 0.613 MS-SSIM 0.917 0.852 NaN 0.631 HDR-VDP-2 0.856 0.803 NaN 0.472 sCIE-Lab 0.907 0.839 NaN 0.603 sCorrel 0.937 0.891 NaN 0.705	No-reference Metric auc@25 auc@50 auc@75 rho AD 0.871 0.891 0.824 0.705 SSIM 0.832 0.768 0.749 0.609 MS-SSIM 0.848 0.765 0.707 0.634 HDR-VDP-2 0.775 0.689 0.676 0.497 sCIE-Lab 0.84 0.757 0.715 0.625 sCorrel 0.863 0.826 0.82 0.67	Mean observer data - with reference experiment AD - distortion map SSIM - distortion map MS-SSIM - distortion map HDR-VDP-2 - distortion map sCIE-Lab - distortion map sCorrel - distortion map
cornell - test image	With-reference Metric auc@25 auc@50 auc@75 rho AD 0.829 0.82 0.845 0.471 SSIM 0.864 0.846 0.874 0.529 MS-SSIM 0.86 0.833 0.834 0.493 HDR-VDP-2 0.899 0.879 0.95 0.56 sCIE-Lab 0.852 0.818 0.828 0.466 sCorrel 0.814 0.782 0.767 0.44	No-reference Metric auc@25 auc@50 auc@75 rho AD 0.785 0.81 0.894 0.236 SSIM 0.828 0.874 0.918 0.305 MS-SSIM 0.812 0.827 0.844 0.266 HDR-VDP-2 0.906 0.954 0.966 0.499 sCIE-Lab 0.79 0.807 0.87 0.201 sCorrel 0.768 0.752 0.784 0.21	Mean observer data - with reference experiment AD - distortion map SSIM - distortion map MS-SSIM - distortion map HDR-VDP-2 - distortion map sCIE-Lab - distortion map sCorrel - distortion map
dragons - test image	With-reference Metric auc@25 auc@50 auc@75 rho AD 0.739 0.77 0.86 0.302 SSIM 0.667 0.677 0.754 0.012 MS-SSIM 0.55 0.541 0.625 -0.361 HDR-VDP-2 0.599 0.596 0.655 -0.153 sCIE-Lab 0.678 0.676 0.741 -0.03 sCorrel 0.787 0.84 0.878 0.43	No-reference Metric auc@25 auc@50 auc@75 rho AD 0.763 0.85 0.936 0.432 SSIM 0.711 0.74 0.893 0.17 MS-SSIM 0.615 0.613 0.781 -0.204 HDR-VDP-2 0.612 0.643 0.881 -0.044 sCIE-Lab 0.733 0.739 0.841 0.125 sCorrel 0.815 0.877 0.779 0.521	Mean observer data - with reference experiment AD - distortion map SSIM - distortion map MS-SSIM - distortion map HDR-VDP-2 - distortion map sCIE-Lab - distortion map sCorrel - distortion map
hall - test image	With-reference Metric auc@25 auc@50 auc@75 rho AD NaN NaN NaN 0.119 SSIM NaN NaN NaN 0.063 MS-SSIM NaN NaN NaN -0.187 HDR-VDP-2 NaN NaN NaN 0.121 sCIE-Lab NaN NaN NaN 0.115 sCorrel NaN NaN NaN 0.123	No-reference Metric auc@25 auc@50 auc@75 rho AD 0.586 NaN NaN 0.1 SSIM 0.652 NaN NaN 0.202 MS-SSIM 0.468 NaN NaN 0.044 HDR-VDP-2 0.518 NaN NaN 0.026 sCIE-Lab 0.618 NaN NaN 0.147 sCorrel 0.403 NaN NaN -0.079	Mean observer data - with reference experiment AD - distortion map SSIM - distortion map MS-SSIM - distortion map HDR-VDP-2 - distortion map sCIE-Lab - distortion map sCorrel - distortion map
icido - test image	With-reference Metric auc@25 auc@50 auc@75 rho AD 0.914 0.951 0.963 0.307 SSIM 0.934 0.961 0.967 0.367 MS-SSIM 0.885 0.894 0.873 0.098 HDR-VDP-2 0.969 0.972 0.948 0.549 sCIE-Lab 0.934 0.953 0.966 0.323 sCorrel 0.913 0.941 0.89 0.282	No-reference Metric auc@25 auc@50 auc@75 rho AD 0.877 0.946 0.96 0.475 SSIM 0.909 0.959 0.965 0.506 MS-SSIM 0.882 0.886 0.873 0.43 HDR-VDP-2 0.938 0.956 0.948 0.503 sCIE-Lab 0.921 0.95 0.963 0.504 sCorrel 0.872 0.925 0.89 0.403	Mean observer data - with reference experiment AD - distortion map SSIM - distortion map MS-SSIM - distortion map HDR-VDP-2 - distortion map sCIE-Lab - distortion map sCorrel - distortion map
kitchen_II - test image	With-reference Metric auc@25 auc@50 auc@75 rho AD 0.919 0.921 0.943 0.469 SSIM 0.881 0.884 0.922 0.372 MS-SSIM 0.896 0.896 0.925 0.395 HDR-VDP-2 0.808 0.813 0.827 0.181 sCIE-Lab 0.932 0.936 0.96 0.537 sCorrel 0.886 0.882 0.912 0.387	No-reference Metric auc@25 auc@50 auc@75 rho AD 0.924 0.931 0.917 0.403 SSIM 0.868 0.896 0.89 0.337 MS-SSIM 0.89 0.897 0.864 0.329 HDR-VDP-2 0.776 0.797 0.768 0.094 sCIE-Lab 0.938 0.942 0.914 0.429 sCorrel 0.881 0.873 0.739 0.252	Mean observer data - with reference experiment AD - distortion map SSIM - distortion map MS-SSIM - distortion map HDR-VDP-2 - distortion map sCIE-Lab - distortion map sCorrel - distortion map
livingroom - test image	With-reference Metric auc@25 auc@50 auc@75 rho AD 0.929 0.967 0.952 0.806 SSIM 0.93 0.983 0.956 0.795 MS-SSIM 0.898 0.979 0.953 0.728 HDR-VDP-2 0.911 0.979 0.954 0.711 sCIE-Lab 0.915 0.943 0.918 0.724 sCorrel 0.916 0.818 0.838 0.392	No-reference Metric auc@25 auc@50 auc@75 rho AD 0.962 0.897 NaN 0.671 SSIM 0.981 0.87 NaN 0.713 MS-SSIM 0.979 0.88 NaN 0.695 HDR-VDP-2 0.98 0.884 NaN 0.71 sCIE-Lab 0.941 0.841 NaN 0.624 sCorrel 0.811 0.89 NaN 0.497	Mean observer data - with reference experiment AD - distortion map SSIM - distortion map MS-SSIM - distortion map HDR-VDP-2 - distortion map sCIE-Lab - distortion map sCorrel - distortion map
mpii - test image	With-reference Metric auc@25 auc@50 auc@75 rho AD 0.985 0.983 0.985 0.601 SSIM 0.939 0.968 0.99 0.515 MS-SSIM 0.991 0.985 0.988 0.602 HDR-VDP-2 0.977 0.974 0.959 0.383 sCIE-Lab 0.991 0.984 0.987 0.62 sCorrel 0.953 0.967 0.982 0.33	No-reference Metric auc@25 auc@50 auc@75 rho AD 0.937 0.976 0.988 0.544 SSIM 0.883 0.892 0.99 0.448 MS-SSIM 0.905 0.97 0.989 0.401 HDR-VDP-2 0.89 0.933 0.956 0.329 sCIE-Lab 0.927 0.973 0.988 0.48 sCorrel 0.898 0.965 0.981 0.47	Mean observer data - with reference experiment AD - distortion map SSIM - distortion map MS-SSIM - distortion map HDR-VDP-2 - distortion map sCIE-Lab - distortion map sCorrel - distortion map
plants-d - test image	With-reference Metric auc@25 auc@50 auc@75 rho AD 0.923 0.901 NaN 0.669 SSIM 0.876 0.895 NaN 0.632 MS-SSIM 0.915 0.879 NaN 0.682 HDR-VDP-2 0.846 0.839 NaN 0.62 sCIE-Lab 0.917 0.892 NaN 0.661 sCorrel 0.898 0.887 NaN 0.661	No-reference Metric auc@25 auc@50 auc@75 rho AD 0.881 NaN NaN 0.318 SSIM 0.873 NaN NaN 0.315 MS-SSIM 0.834 NaN NaN 0.256 HDR-VDP-2 0.842 NaN NaN 0.253 sCIE-Lab 0.87 NaN NaN 0.316 sCorrel 0.871 NaN NaN 0.265	Mean observer data - with reference experiment AD - distortion map SSIM - distortion map MS-SSIM - distortion map HDR-VDP-2 - distortion map sCIE-Lab - distortion map sCorrel - distortion map
plants-u - test image	With-reference Metric auc@25 auc@50 auc@75 rho AD 0.938 0.941 0.864 0.607 SSIM 0.893 0.888 0.967 0.642 MS-SSIM 0.945 0.947 0.958 0.748 HDR-VDP-2 0.919 0.903 0.94 0.67 sCIE-Lab 0.947 0.944 0.955 0.707 sCorrel 0.925 0.924 0.898 0.625	No-reference Metric auc@25 auc@50 auc@75 rho AD 0.954 0.869 NaN 0.65 SSIM 0.896 0.969 NaN 0.615 MS-SSIM 0.953 0.957 NaN 0.709 HDR-VDP-2 0.907 0.947 NaN 0.551 sCIE-Lab 0.949 0.955 NaN 0.664 sCorrel 0.933 0.921 NaN 0.642	Mean observer data - with reference experiment AD - distortion map SSIM - distortion map MS-SSIM - distortion map HDR-VDP-2 - distortion map sCIE-Lab - distortion map sCorrel - distortion map
room - test image	With-reference Metric auc@25 auc@50 auc@75 rho AD 0.827 0.77 0.766 0.184 SSIM 0.84 0.787 0.781 0.224 MS-SSIM 0.82 0.727 0.682 0.095 HDR-VDP-2 0.852 0.806 0.791 0.292 sCIE-Lab 0.857 0.792 0.775 0.2 sCorrel 0.773 0.704 0.639 -0.182	No-reference Metric auc@25 auc@50 auc@75 rho AD 0.514 0.75 NaN -0.15 SSIM 0.515 0.744 NaN -0.127 MS-SSIM 0.432 0.662 NaN -0.293 HDR-VDP-2 0.505 0.766 NaN -0.127 sCIE-Lab 0.497 0.757 NaN -0.17 sCorrel 0.466 0.635 NaN -0.243	Mean observer data - with reference experiment AD - distortion map SSIM - distortion map MS-SSIM - distortion map HDR-VDP-2 - distortion map sCIE-Lab - distortion map sCorrel - distortion map
sala - test image	With-reference Metric auc@25 auc@50 auc@75 rho AD 0.434 0.39 0.399 -0.215 SSIM 0.493 0.45 0.46 -0.127 MS-SSIM 0.437 0.435 0.455 -0.247 HDR-VDP-2 0.488 0.293 0.257 -0.029 sCIE-Lab 0.649 0.757 0.763 0.201 sCorrel 0.818 0.896 0.917 0.556	No-reference Metric auc@25 auc@50 auc@75 rho AD 0.4 0.779 NaN -0.282 SSIM 0.482 0.854 NaN -0.161 MS-SSIM 0.448 0.846 NaN -0.253 HDR-VDP-2 0.372 0.374 NaN -0.176 sCIE-Lab 0.697 0.94 NaN 0.269 sCorrel 0.852 0.889 NaN 0.608	Mean observer data - with reference experiment AD - distortion map SSIM - distortion map MS-SSIM - distortion map HDR-VDP-2 - distortion map sCIE-Lab - distortion map sCorrel - distortion map
sanmiguel - test image	With-reference Metric auc@25 auc@50 auc@75 rho AD 0.961 NaN NaN 0.393 SSIM 0.915 NaN NaN 0.235 MS-SSIM 0.931 NaN NaN 0.217 HDR-VDP-2 0.84 NaN NaN -0.159 sCIE-Lab 0.94 NaN NaN 0.236 sCorrel 0.966 NaN NaN 0.478	No-reference Metric auc@25 auc@50 auc@75 rho AD 0.948 NaN NaN 0.316 SSIM 0.867 NaN NaN 0.2 MS-SSIM 0.878 NaN NaN 0.072 HDR-VDP-2 0.804 NaN NaN 0.016 sCIE-Lab 0.906 NaN NaN 0.122 sCorrel 0.975 NaN NaN 0.267	Mean observer data - with reference experiment AD - distortion map SSIM - distortion map MS-SSIM - distortion map HDR-VDP-2 - distortion map sCIE-Lab - distortion map sCorrel - distortion map
sanmiguel_cam3 - test image	With-reference Metric auc@25 auc@50 auc@75 rho AD 0.975 0.977 0.98 0.661 SSIM 0.901 0.894 0.937 0.517 MS-SSIM 0.989 0.987 0.979 0.72 HDR-VDP-2 0.907 0.921 0.924 0.189 sCIE-Lab 0.987 0.986 0.986 0.701 sCorrel 0.981 0.983 0.98 0.625	No-reference Metric auc@25 auc@50 auc@75 rho AD 0.935 NaN NaN 0.204 SSIM 0.847 NaN NaN 0.17 MS-SSIM 0.934 NaN NaN 0.101 HDR-VDP-2 0.863 NaN NaN 0.116 sCIE-Lab 0.94 NaN NaN 0.153 sCorrel 0.944 NaN NaN 0.16	Mean observer data - with reference experiment AD - distortion map SSIM - distortion map MS-SSIM - distortion map HDR-VDP-2 - distortion map sCIE-Lab - distortion map sCorrel - distortion map
sanmiguel_cam4 - test image	With-reference Metric auc@25 auc@50 auc@75 rho AD 0.781 0.774 0.794 0.179 SSIM 0.759 0.742 0.759 0.121 MS-SSIM 0.751 0.729 0.745 0.112 HDR-VDP-2 0.685 0.653 0.665 -0.035 sCIE-Lab 0.779 0.77 0.791 0.175 sCorrel 0.909 0.946 0.968 0.589	No-reference Metric auc@25 auc@50 auc@75 rho AD 0.773 0.77 0.728 0.181 SSIM 0.744 0.754 0.736 0.155 MS-SSIM 0.735 0.721 0.678 0.087 HDR-VDP-2 0.662 0.656 0.694 0.023 sCIE-Lab 0.771 0.775 0.759 0.198 sCorrel 0.935 0.951 0.941 0.608	Mean observer data - with reference experiment AD - distortion map SSIM - distortion map MS-SSIM - distortion map HDR-VDP-2 - distortion map sCIE-Lab - distortion map sCorrel - distortion map
sibenik - test image	With-reference Metric auc@25 auc@50 auc@75 rho AD 0.939 NaN NaN -0.101 SSIM 0.986 NaN NaN -0.078 MS-SSIM 0.97 NaN NaN -0.203 HDR-VDP-2 0.88 NaN NaN 0.089 sCIE-Lab 0.969 NaN NaN -0.103 sCorrel 0.638 NaN NaN 0.115	No-reference Metric auc@25 auc@50 auc@75 rho AD NaN NaN NaN 0.109 SSIM NaN NaN NaN 0.067 MS-SSIM NaN NaN NaN 0.11 HDR-VDP-2 NaN NaN NaN -0.276 sCIE-Lab NaN NaN NaN 0.093 sCorrel NaN NaN NaN -0.142	Mean observer data - with reference experiment AD - distortion map SSIM - distortion map MS-SSIM - distortion map HDR-VDP-2 - distortion map sCIE-Lab - distortion map sCorrel - distortion map
sponza - test image	With-reference Metric auc@25 auc@50 auc@75 rho AD 0.868 0.964 0.988 0.522 SSIM 0.962 0.994 0.996 0.687 MS-SSIM 0.965 0.988 0.992 0.67 HDR-VDP-2 0.901 0.93 0.941 0.506 sCIE-Lab 0.948 0.989 0.995 0.667 sCorrel 0.775 0.886 0.925 0.383	No-reference Metric auc@25 auc@50 auc@75 rho AD 0.821 NaN NaN 0.133 SSIM 0.707 NaN NaN 0.153 MS-SSIM 0.675 NaN NaN 0.103 HDR-VDP-2 0.726 NaN NaN 0.138 sCIE-Lab 0.716 NaN NaN 0.185 sCorrel 0.658 NaN NaN 0.054	Mean observer data - with reference experiment AD - distortion map SSIM - distortion map MS-SSIM - distortion map HDR-VDP-2 - distortion map sCIE-Lab - distortion map sCorrel - distortion map
tt - test image	With-reference Metric auc@25 auc@50 auc@75 rho AD 0.91 0.928 0.944 0.44 SSIM 0.856 0.94 0.951 0.276 MS-SSIM 0.689 0.933 0.947 0.102 HDR-VDP-2 0.803 0.866 0.863 0.179 sCIE-Lab 0.869 0.939 0.951 0.295 sCorrel 0.915 0.816 0.787 0.32	No-reference Metric auc@25 auc@50 auc@75 rho AD 0.892 0.935 NaN 0.418 SSIM 0.821 0.94 NaN 0.272 MS-SSIM 0.711 0.93 NaN 0.154 HDR-VDP-2 0.782 0.861 NaN 0.172 sCIE-Lab 0.85 0.938 NaN 0.306 sCorrel 0.894 0.817 NaN 0.332	Mean observer data - with reference experiment AD - distortion map SSIM - distortion map MS-SSIM - distortion map HDR-VDP-2 - distortion map sCIE-Lab - distortion map sCorrel - distortion map
villa_cam1 - test image	With-reference Metric auc@25 auc@50 auc@75 rho AD 0.967 0.952 0.918 0.5 SSIM 0.943 0.926 0.91 0.413 MS-SSIM 0.852 0.844 0.833 0.267 HDR-VDP-2 0.882 0.873 0.876 0.226 sCIE-Lab 0.954 0.935 0.908 0.407 sCorrel 0.941 0.953 0.978 0.619	No-reference Metric auc@25 auc@50 auc@75 rho AD 0.944 0.926 0.918 0.578 SSIM 0.936 0.924 0.931 0.533 MS-SSIM 0.854 0.887 0.869 0.308 HDR-VDP-2 0.883 0.876 0.892 0.335 sCIE-Lab 0.938 0.931 0.928 0.521 sCorrel 0.942 0.95 0.972 0.624	Mean observer data - with reference experiment AD - distortion map SSIM - distortion map MS-SSIM - distortion map HDR-VDP-2 - distortion map sCIE-Lab - distortion map sCorrel - distortion map
villa_cam2 - test image	With-reference Metric auc@25 auc@50 auc@75 rho AD 0.611 0.576 0.515 0.097 SSIM 0.645 0.635 0.551 0.181 MS-SSIM 0.587 0.586 0.451 0.123 HDR-VDP-2 0.667 0.608 0.634 0.032 sCIE-Lab 0.635 0.573 0.51 0.096 sCorrel 0.756 0.804 0.84 0.489	No-reference Metric auc@25 auc@50 auc@75 rho AD 0.547 0.546 0.552 -0 SSIM 0.614 0.571 0.614 0.077 MS-SSIM 0.571 0.443 0.392 0.106 HDR-VDP-2 0.581 0.654 0.755 -0.041 sCIE-Lab 0.546 0.541 0.596 -0.017 sCorrel 0.811 0.828 0.877 0.423	Mean observer data - with reference experiment AD - distortion map SSIM - distortion map MS-SSIM - distortion map HDR-VDP-2 - distortion map sCIE-Lab - distortion map sCorrel - distortion map
villa_cam3 - test image	With-reference Metric auc@25 auc@50 auc@75 rho AD 0.946 0.938 0.945 0.362 SSIM 0.954 0.937 0.951 0.4 MS-SSIM 0.946 0.928 0.945 0.364 HDR-VDP-2 0.894 0.875 0.847 0.099 sCIE-Lab 0.954 0.939 0.947 0.369 sCorrel 0.941 0.947 0.926 0.431	No-reference Metric auc@25 auc@50 auc@75 rho AD 0.896 0.932 0.999 0.451 SSIM 0.892 0.937 0.998 0.458 MS-SSIM 0.876 0.931 0.997 0.454 HDR-VDP-2 0.842 0.868 0.972 0.383 sCIE-Lab 0.889 0.932 0.999 0.434 sCorrel 0.89 0.896 0.998 0.458	Mean observer data - with reference experiment AD - distortion map SSIM - distortion map MS-SSIM - distortion map HDR-VDP-2 - distortion map sCIE-Lab - distortion map sCorrel - distortion map