Download PDFOpen PDF in browser

Classifier-Based Evaluation of Image Feature Importance

14 pagesPublished: September 17, 2018


Significant advances in the performance of deep neural networks, such as Convolutional Neural Networks (CNNs) for image classification, have created a drive for understanding how they work. Different techniques have been proposed to determine which features (e.g., image pixels) are most important for a CNN’s classification. However, the important features output by these techniques have typically been judged subjectively by a human to assess whether the important features capture the features relevant to the classification and not whether the features were actually important to classifier itself. We address the need for an objective measure to assess the quality of different feature importance measures. In particular, we propose measuring the ratio of a CNN’s accuracy on the whole image com- pared to an image containing only the important features. We also consider scaling this ratio by the relative size of the important region in order to measure the conciseness. We demonstrate that our measures correlate well with prior subjective comparisons of important features, but importantly do not require their human studies. We also demonstrate that the features on which multiple techniques agree are important have a higher impact on accuracy than those features that only one technique finds.

Keyphrases: deep learning, Experimental Methodology/Replicability, interpretability

In: Daniel Lee, Alexander Steen and Toby Walsh (editors). GCAI-2018. 4th Global Conference on Artificial Intelligence, vol 55, pages 162--175

BibTeX entry
  author    = {Sai Prabhakar Pandi Selvaraj and Manuela Veloso and Stephanie Rosenthal},
  title     = {Classifier-Based Evaluation of Image Feature Importance},
  booktitle = {GCAI-2018. 4th Global Conference on Artificial Intelligence},
  editor    = {Daniel Lee and Alexander Steen and Toby Walsh},
  series    = {EPiC Series in Computing},
  volume    = {55},
  pages     = {162--175},
  year      = {2018},
  publisher = {EasyChair},
  bibsource = {EasyChair,},
  issn      = {2398-7340},
  url       = {},
  doi       = {10.29007/p655}}
Download PDFOpen PDF in browser