Inferential Statistics - the Effect of Computer Graphics Expertise

The images below show results of Kruskal-Wallis statistical significance test calculated on per-pixel basis separately for each scene.

Left column: hover the mouse over the test image to see the reference image.

Right colums: the images show p-values for a particular test, p-values smaller than 0.05 (i.e. statistically significant differences) are visualized in green (when experts average is higher than non-experts) and red (when experts average is lower than non-experts), hover the mouse over the result to see the visualization of p-values smaller than 0.01. The number S shows the overall significance (per scene) and was obtained as the fraction of the number of statistically significant pixels (i.e. p-values<0.05) over the number of pixels in regions marked by at least 5% of observers (i.e. the number of pixels with probabilities of detection >0.05).

appartment_II - test image	With-ref experiment: Effect of observer experience: S=0.0057675, Kruskal-Wallis p < 0.05	With-ref experiment: mean observer data of experts	No-ref experiment: Effect of observer experience: S=0.05435, Kruskal-Wallis p < 0.05	No-ref experiment: mean observer data of experts
atrium - test image	With-ref experiment: Effect of observer experience: S=0.052395, Kruskal-Wallis p < 0.05	With-ref experiment: mean observer data of experts	No-ref experiment: Effect of observer experience: S=0.086186, Kruskal-Wallis p < 0.05	No-ref experiment: mean observer data of experts
bathroom - test image	With-ref experiment: Effect of observer experience: S=0.20089, Kruskal-Wallis p < 0.05	With-ref experiment: mean observer data of experts	No-ref experiment: Effect of observer experience: S=0.42804, Kruskal-Wallis p < 0.05	No-ref experiment: mean observer data of experts
buddha - test image	With-ref experiment: Effect of observer experience: S=0.04905, Kruskal-Wallis p < 0.05	With-ref experiment: mean observer data of experts	No-ref experiment: Effect of observer experience: S=0.10322, Kruskal-Wallis p < 0.05	No-ref experiment: mean observer data of experts
chairs - test image	With-ref experiment: Effect of observer experience: S=0.17289, Kruskal-Wallis p < 0.05	With-ref experiment: mean observer data of experts	No-ref experiment: Effect of observer experience: S=0.21455, Kruskal-Wallis p < 0.05	No-ref experiment: mean observer data of experts
city-d - test image	With-ref experiment: Effect of observer experience: S=0, Kruskal-Wallis p < 0.05	With-ref experiment: mean observer data of experts	No-ref experiment: Effect of observer experience: S=0, Kruskal-Wallis p < 0.05	No-ref experiment: mean observer data of experts
city-u - test image	With-ref experiment: Effect of observer experience: S=0.06427, Kruskal-Wallis p < 0.05	With-ref experiment: mean observer data of experts	No-ref experiment: Effect of observer experience: S=0.016194, Kruskal-Wallis p < 0.05	No-ref experiment: mean observer data of experts
cornell - test image	With-ref experiment: Effect of observer experience: S=0.13777, Kruskal-Wallis p < 0.05	With-ref experiment: mean observer data of experts	No-ref experiment: Effect of observer experience: S=0.098045, Kruskal-Wallis p < 0.05	No-ref experiment: mean observer data of experts
dragons - test image	With-ref experiment: Effect of observer experience: S=0.078262, Kruskal-Wallis p < 0.05	With-ref experiment: mean observer data of experts	No-ref experiment: Effect of observer experience: S=0.34445, Kruskal-Wallis p < 0.05	No-ref experiment: mean observer data of experts
hall - test image	With-ref experiment: Effect of observer experience: S=0, Kruskal-Wallis p < 0.05	With-ref experiment: mean observer data of experts	No-ref experiment: Effect of observer experience: S=0.11079, Kruskal-Wallis p < 0.05	No-ref experiment: mean observer data of experts
icido - test image	With-ref experiment: Effect of observer experience: S=0.09307, Kruskal-Wallis p < 0.05	With-ref experiment: mean observer data of experts	No-ref experiment: Effect of observer experience: S=0.14156, Kruskal-Wallis p < 0.05	No-ref experiment: mean observer data of experts
kitchen_II - test image	With-ref experiment: Effect of observer experience: S=0.0062557, Kruskal-Wallis p < 0.05	With-ref experiment: mean observer data of experts	No-ref experiment: Effect of observer experience: S=0.028573, Kruskal-Wallis p < 0.05	No-ref experiment: mean observer data of experts
livingroom - test image	With-ref experiment: Effect of observer experience: S=0.03411, Kruskal-Wallis p < 0.05	With-ref experiment: mean observer data of experts	No-ref experiment: Effect of observer experience: S=0.050304, Kruskal-Wallis p < 0.05	No-ref experiment: mean observer data of experts
mpii - test image	With-ref experiment: Effect of observer experience: S=0.10527, Kruskal-Wallis p < 0.05	With-ref experiment: mean observer data of experts	No-ref experiment: Effect of observer experience: S=0.059571, Kruskal-Wallis p < 0.05	No-ref experiment: mean observer data of experts
plants-d - test image	With-ref experiment: Effect of observer experience: S=0.15907, Kruskal-Wallis p < 0.05	With-ref experiment: mean observer data of experts	No-ref experiment: Effect of observer experience: S=0.016505, Kruskal-Wallis p < 0.05	No-ref experiment: mean observer data of experts
plants-u - test image	With-ref experiment: Effect of observer experience: S=0.071961, Kruskal-Wallis p < 0.05	With-ref experiment: mean observer data of experts	No-ref experiment: Effect of observer experience: S=0.14305, Kruskal-Wallis p < 0.05	No-ref experiment: mean observer data of experts
room - test image	With-ref experiment: Effect of observer experience: S=0.0077641, Kruskal-Wallis p < 0.05	With-ref experiment: mean observer data of experts	No-ref experiment: Effect of observer experience: S=0.051778, Kruskal-Wallis p < 0.05	No-ref experiment: mean observer data of experts
sala - test image	With-ref experiment: Effect of observer experience: S=0.04468, Kruskal-Wallis p < 0.05	With-ref experiment: mean observer data of experts	No-ref experiment: Effect of observer experience: S=0.35035, Kruskal-Wallis p < 0.05	No-ref experiment: mean observer data of experts
sanmiguel - test image	With-ref experiment: Effect of observer experience: S=0.029025, Kruskal-Wallis p < 0.05	With-ref experiment: mean observer data of experts	No-ref experiment: Effect of observer experience: S=0, Kruskal-Wallis p < 0.05	No-ref experiment: mean observer data of experts
sanmiguel_cam3 - test image	With-ref experiment: Effect of observer experience: S=0.029104, Kruskal-Wallis p < 0.05	With-ref experiment: mean observer data of experts	No-ref experiment: Effect of observer experience: S=0.017187, Kruskal-Wallis p < 0.05	No-ref experiment: mean observer data of experts
sanmiguel_cam4 - test image	With-ref experiment: Effect of observer experience: S=0.015803, Kruskal-Wallis p < 0.05	With-ref experiment: mean observer data of experts	No-ref experiment: Effect of observer experience: S=0.0090041, Kruskal-Wallis p < 0.05	No-ref experiment: mean observer data of experts
sibenik - test image	With-ref experiment: Effect of observer experience: S=0.026071, Kruskal-Wallis p < 0.05	With-ref experiment: mean observer data of experts	No-ref experiment: Effect of observer experience: S=0, Kruskal-Wallis p < 0.05	No-ref experiment: mean observer data of experts
sponza - test image	With-ref experiment: Effect of observer experience: S=0.04401, Kruskal-Wallis p < 0.05	With-ref experiment: mean observer data of experts	No-ref experiment: Effect of observer experience: S=0.0012116, Kruskal-Wallis p < 0.05	No-ref experiment: mean observer data of experts
tt - test image	With-ref experiment: Effect of observer experience: S=0.17285, Kruskal-Wallis p < 0.05	With-ref experiment: mean observer data of experts	No-ref experiment: Effect of observer experience: S=0.037988, Kruskal-Wallis p < 0.05	No-ref experiment: mean observer data of experts
villa_cam1 - test image	With-ref experiment: Effect of observer experience: S=0.021526, Kruskal-Wallis p < 0.05	With-ref experiment: mean observer data of experts	No-ref experiment: Effect of observer experience: S=0.02907, Kruskal-Wallis p < 0.05	No-ref experiment: mean observer data of experts
villa_cam2 - test image	With-ref experiment: Effect of observer experience: S=0.021434, Kruskal-Wallis p < 0.05	With-ref experiment: mean observer data of experts	No-ref experiment: Effect of observer experience: S=0.030157, Kruskal-Wallis p < 0.05	No-ref experiment: mean observer data of experts
villa_cam3 - test image	With-ref experiment: Effect of observer experience: S=0.05314, Kruskal-Wallis p < 0.05	With-ref experiment: mean observer data of experts	No-ref experiment: Effect of observer experience: S=0.082254, Kruskal-Wallis p < 0.05	No-ref experiment: mean observer data of experts