The images below show results of Kruskal-Wallis statistical significance test calculated on per-pixel basis separately for each scene.

Left column: hover the mouse over the test image to see the reference image.

Right colums: the images show p-values for a particular test, p-values smaller than 0.05 (i.e. statistically significant differences) are visualized in green (when experts average is higher than non-experts) and red (when experts average is lower than non-experts), hover the mouse over the result to see the visualization of p-values smaller than 0.01. The number S shows the overall significance (per scene) and was obtained as the fraction of the number of statistically significant pixels (i.e. p-values<0.05) over the number of pixels in regions marked by at least 5% of observers (i.e. the number of pixels with probabilities of detection >0.05).

appartment_II - test image | With-ref experiment: Effect of observer experience: S=0.0057675, Kruskal-Wallis p < 0.05 | With-ref experiment: mean observer data of experts | No-ref experiment: Effect of observer experience: S=0.05435, Kruskal-Wallis p < 0.05 | No-ref experiment: mean observer data of experts |

atrium - test image | With-ref experiment: Effect of observer experience: S=0.052395, Kruskal-Wallis p < 0.05 | With-ref experiment: mean observer data of experts | No-ref experiment: Effect of observer experience: S=0.086186, Kruskal-Wallis p < 0.05 | No-ref experiment: mean observer data of experts |

bathroom - test image | With-ref experiment: Effect of observer experience: S=0.20089, Kruskal-Wallis p < 0.05 | With-ref experiment: mean observer data of experts | No-ref experiment: Effect of observer experience: S=0.42804, Kruskal-Wallis p < 0.05 | No-ref experiment: mean observer data of experts |

buddha - test image | With-ref experiment: Effect of observer experience: S=0.04905, Kruskal-Wallis p < 0.05 | With-ref experiment: mean observer data of experts | No-ref experiment: Effect of observer experience: S=0.10322, Kruskal-Wallis p < 0.05 | No-ref experiment: mean observer data of experts |

chairs - test image | With-ref experiment: Effect of observer experience: S=0.17289, Kruskal-Wallis p < 0.05 | With-ref experiment: mean observer data of experts | No-ref experiment: Effect of observer experience: S=0.21455, Kruskal-Wallis p < 0.05 | No-ref experiment: mean observer data of experts |

city-d - test image | With-ref experiment: Effect of observer experience: S=0, Kruskal-Wallis p < 0.05 | With-ref experiment: mean observer data of experts | No-ref experiment: Effect of observer experience: S=0, Kruskal-Wallis p < 0.05 | No-ref experiment: mean observer data of experts |

city-u - test image | With-ref experiment: Effect of observer experience: S=0.06427, Kruskal-Wallis p < 0.05 | With-ref experiment: mean observer data of experts | No-ref experiment: Effect of observer experience: S=0.016194, Kruskal-Wallis p < 0.05 | No-ref experiment: mean observer data of experts |

cornell - test image | With-ref experiment: Effect of observer experience: S=0.13777, Kruskal-Wallis p < 0.05 | With-ref experiment: mean observer data of experts | No-ref experiment: Effect of observer experience: S=0.098045, Kruskal-Wallis p < 0.05 | No-ref experiment: mean observer data of experts |

dragons - test image | With-ref experiment: Effect of observer experience: S=0.078262, Kruskal-Wallis p < 0.05 | With-ref experiment: mean observer data of experts | No-ref experiment: Effect of observer experience: S=0.34445, Kruskal-Wallis p < 0.05 | No-ref experiment: mean observer data of experts |

hall - test image | With-ref experiment: Effect of observer experience: S=0, Kruskal-Wallis p < 0.05 | With-ref experiment: mean observer data of experts | No-ref experiment: Effect of observer experience: S=0.11079, Kruskal-Wallis p < 0.05 | No-ref experiment: mean observer data of experts |

icido - test image | With-ref experiment: Effect of observer experience: S=0.09307, Kruskal-Wallis p < 0.05 | With-ref experiment: mean observer data of experts | No-ref experiment: Effect of observer experience: S=0.14156, Kruskal-Wallis p < 0.05 | No-ref experiment: mean observer data of experts |

kitchen_II - test image | With-ref experiment: Effect of observer experience: S=0.0062557, Kruskal-Wallis p < 0.05 | With-ref experiment: mean observer data of experts | No-ref experiment: Effect of observer experience: S=0.028573, Kruskal-Wallis p < 0.05 | No-ref experiment: mean observer data of experts |

livingroom - test image | With-ref experiment: Effect of observer experience: S=0.03411, Kruskal-Wallis p < 0.05 | With-ref experiment: mean observer data of experts | No-ref experiment: Effect of observer experience: S=0.050304, Kruskal-Wallis p < 0.05 | No-ref experiment: mean observer data of experts |

mpii - test image | With-ref experiment: Effect of observer experience: S=0.10527, Kruskal-Wallis p < 0.05 | With-ref experiment: mean observer data of experts | No-ref experiment: Effect of observer experience: S=0.059571, Kruskal-Wallis p < 0.05 | No-ref experiment: mean observer data of experts |

plants-d - test image | With-ref experiment: Effect of observer experience: S=0.15907, Kruskal-Wallis p < 0.05 | With-ref experiment: mean observer data of experts | No-ref experiment: Effect of observer experience: S=0.016505, Kruskal-Wallis p < 0.05 | No-ref experiment: mean observer data of experts |

plants-u - test image | With-ref experiment: Effect of observer experience: S=0.071961, Kruskal-Wallis p < 0.05 | With-ref experiment: mean observer data of experts | No-ref experiment: Effect of observer experience: S=0.14305, Kruskal-Wallis p < 0.05 | No-ref experiment: mean observer data of experts |

room - test image | With-ref experiment: Effect of observer experience: S=0.0077641, Kruskal-Wallis p < 0.05 | With-ref experiment: mean observer data of experts | No-ref experiment: Effect of observer experience: S=0.051778, Kruskal-Wallis p < 0.05 | No-ref experiment: mean observer data of experts |

sala - test image | With-ref experiment: Effect of observer experience: S=0.04468, Kruskal-Wallis p < 0.05 | With-ref experiment: mean observer data of experts | No-ref experiment: Effect of observer experience: S=0.35035, Kruskal-Wallis p < 0.05 | No-ref experiment: mean observer data of experts |

sanmiguel - test image | With-ref experiment: Effect of observer experience: S=0.029025, Kruskal-Wallis p < 0.05 | With-ref experiment: mean observer data of experts | No-ref experiment: Effect of observer experience: S=0, Kruskal-Wallis p < 0.05 | No-ref experiment: mean observer data of experts |

sanmiguel_cam3 - test image | With-ref experiment: Effect of observer experience: S=0.029104, Kruskal-Wallis p < 0.05 | With-ref experiment: mean observer data of experts | No-ref experiment: Effect of observer experience: S=0.017187, Kruskal-Wallis p < 0.05 | No-ref experiment: mean observer data of experts |

sanmiguel_cam4 - test image | With-ref experiment: Effect of observer experience: S=0.015803, Kruskal-Wallis p < 0.05 | With-ref experiment: mean observer data of experts | No-ref experiment: Effect of observer experience: S=0.0090041, Kruskal-Wallis p < 0.05 | No-ref experiment: mean observer data of experts |

sibenik - test image | With-ref experiment: Effect of observer experience: S=0.026071, Kruskal-Wallis p < 0.05 | With-ref experiment: mean observer data of experts | No-ref experiment: Effect of observer experience: S=0, Kruskal-Wallis p < 0.05 | No-ref experiment: mean observer data of experts |

sponza - test image | With-ref experiment: Effect of observer experience: S=0.04401, Kruskal-Wallis p < 0.05 | With-ref experiment: mean observer data of experts | No-ref experiment: Effect of observer experience: S=0.0012116, Kruskal-Wallis p < 0.05 | No-ref experiment: mean observer data of experts |

tt - test image | With-ref experiment: Effect of observer experience: S=0.17285, Kruskal-Wallis p < 0.05 | With-ref experiment: mean observer data of experts | No-ref experiment: Effect of observer experience: S=0.037988, Kruskal-Wallis p < 0.05 | No-ref experiment: mean observer data of experts |

villa_cam1 - test image | With-ref experiment: Effect of observer experience: S=0.021526, Kruskal-Wallis p < 0.05 | With-ref experiment: mean observer data of experts | No-ref experiment: Effect of observer experience: S=0.02907, Kruskal-Wallis p < 0.05 | No-ref experiment: mean observer data of experts |

villa_cam2 - test image | With-ref experiment: Effect of observer experience: S=0.021434, Kruskal-Wallis p < 0.05 | With-ref experiment: mean observer data of experts | No-ref experiment: Effect of observer experience: S=0.030157, Kruskal-Wallis p < 0.05 | No-ref experiment: mean observer data of experts |

villa_cam3 - test image | With-ref experiment: Effect of observer experience: S=0.05314, Kruskal-Wallis p < 0.05 | With-ref experiment: mean observer data of experts | No-ref experiment: Effect of observer experience: S=0.082254, Kruskal-Wallis p < 0.05 | No-ref experiment: mean observer data of experts |