1
$\begingroup$

I constructed and trained a convolutional neural network using Keras in R with the TensorFlow backend. I feed the network with multispectral images for a simple image classification.

Is there some way to calculate which of the input bands were most important for the classification task? Ideally, I would like to have a plot with some measure of importance, grouped by bands and image classes.

How can I obtain this information? Would it be necessary / possible to calculate saliency maps for every band and picture, and take the mean or sum of these images per class and band?

Are there also other ways to get the information, which band was most important for the classification of an image?

Edit: With saliency maps I mean these or these visualizations. They provide information on which part of the image led the CNN to the conclusion to which class it identifies. However, I always see only one saliency map for the whole image. Is it possible to make one for each input band of an image? For example if I input RGB data, one for each color channel?

(This is inspired by a visualization of this paper. I saw it but I don't know if it's valid to do and if yes, how to do.)

$\endgroup$
2
  • $\begingroup$ Can you provide some explanation with images? $\endgroup$ Aug 9 at 19:09
  • $\begingroup$ @DevashishPrasad I tried to add more context! I don't know which other images to add, since I did not mange to calculate those. $\endgroup$
    – user122719
    Aug 9 at 20:39
0
$\begingroup$

There are many methods to choose from in terms of interpreting the network, so you might want to look around for methods, their pros and cons.
A good critique and analysis of the interpretation methods was given in this paper.

Regarding the saliency maps, I found the following Keras package.
The catch is that the project is young, so the documentation is not extensive. You might need to contact the developer for some details.

The same developer has another package that can give you the channel-wise saliency by setting keepdims=True.
However, it was not updated for two years, so it might have been discontinued in favor of tf-keras-vis.

In terms of the manual approach, this question might be of help.

$\endgroup$
2
  • $\begingroup$ Thanks a lot! I will take a look at all these links :) $\endgroup$
    – user122719
    Aug 13 at 7:53
  • $\begingroup$ @user122719 Has this information answered your question? $\endgroup$ Sep 3 at 10:39

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy