Inferential Statistics - the Effect of Experiment

The images below show results of Kruskal-Wallis significance test calculated on per-pixel basis separately for each scene.

Left column: hover the mouse over the test image to see the reference image.

Right colums: the images show p-values for a particular test, p-values smaller than 0.05 (i.e. statistically significant differences) are visualized in green (when experts average is higher than non-experts) and red (when experts average is lower than non-experts), hover the mouse over the result to see the visualization of p-values smaller than 0.01. The number S shows the overall significance (per scene) and was obtained as the fraction of the number of statistically significant pixels (i.e. p-values<0.05) over the number of pixels in regions marked by at least 5% of observers (i.e. the number of pixels with probabilities of detection >0.05).

apartment - test image	Effect of experimental setup (with-ref vs. no-ref): S=0.031697, Kruskal-Wallis p<0.05	Mean observer in with-reference experiment
cg figures - test image	Effect of experimental setup (with-ref vs. no-ref): S=0.096889, Kruskal-Wallis p<0.05	Mean observer in with-reference experiment
disney - test image	Effect of experimental setup (with-ref vs. no-ref): S=0.0089249, Kruskal-Wallis p<0.05	Mean observer in with-reference experiment
kitchen - test image	Effect of experimental setup (with-ref vs. no-ref): S=0.018857, Kruskal-Wallis p<0.05	Mean observer in with-reference experiment
red kitchen - test image	Effect of experimental setup (with-ref vs. no-ref): S=0.019203, Kruskal-Wallis p<0.05	Mean observer in with-reference experiment
sponza above tree - test image	Effect of experimental setup (with-ref vs. no-ref): S=0.0084697, Kruskal-Wallis p<0.05	Mean observer in with-reference experiment
sponza arches - test image	Effect of experimental setup (with-ref vs. no-ref): S=0.032583, Kruskal-Wallis p<0.05	Mean observer in with-reference experiment
sponza atrium - test image	Effect of experimental setup (with-ref vs. no-ref): S=0.026113, Kruskal-Wallis p<0.05	Mean observer in with-reference experiment
sponza tree shadows - test image	Effect of experimental setup (with-ref vs. no-ref): S=0.053164, Kruskal-Wallis p<0.05	Mean observer in with-reference experiment
sponza trees - test image	Effect of experimental setup (with-ref vs. no-ref): S=0.047095, Kruskal-Wallis p<0.05	Mean observer in with-reference experiment