3
$\begingroup$

I am encoding audios as Mel-spectrograms and using these Mel-spectrograms as input to my deep learning model (Inception-ResNet V2). The input image is of size 256 X 256, made up of a 128 X 64 spectrogram (128 melbands) along with zero padding. Sample input:enter image description here

I am using LIME to visualize the important regions of a Mel-spectrogram. In some Mel-spectrograms, LIME is labelling the padded regions as important, like in this image.

enter image description here

So, is this happening because of poor learning or is this a genuine problem with LIME when it encounters padded regions?

$\endgroup$

1 Answer 1

1
$\begingroup$

Feeding the neural network 75% irrelevant data (zero padding) seems less-than-ideal, both in terms of computational inefficiency and because it makes your model potentially vulnerable to this data.

I would consider a smaller model with 128x128 input input format (ex: MobileNet). You can either upscale the mel-spectrograms you have now from 128x64 -> 128x128 or extract windows that are 128 long (potentially adjusting FFT/hop size to make the window cover same area in time).

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.