Initialization Noise in Image Gradients and Saliency Maps
Ann-Christin Wörl, Jan Disselhoff, Michael Wand
CVPR 2023 Vancouver
Abstract
In this paper, we examine gradients of logits of image classification CNNs by input pixel values. We observe that these fluctuate considerably with the initial random initialization of the networks (after training). We extend our study to gradients of intermediate layers, obtained via GradCAM, as well as popular network saliency estimators such as DeepLIFT, SHAP, LIME, Integrated Gradients, and SmoothGrad. While empirical noise levels vary, qualitatively different attributions to image features are still possible with all of these, which comes with implications for interpreting such attributions, in particular when seeking data-driven explanations of the phenomenon generating the data. Finally, we demonstrate that the observed artefacts can be removed by marginalization over the initialization distribution by simple stochastic integration.
Material
Acknowledgements
This work has been supported by the Carl-Zeiss-Stiftung through grants "Big Data in Atmospheric Physics" and "Emergent Algorithmic Intelligence".
