Suppose I have a dataset of audio files that I have to use for whale sound classification. I am choosing the strategy of treating it as an image classification problem by using their corresponding spectrogram (frequency vs time plot) images. The image shown below shows an example how the whale calls look like in the Label B(B is a species of whale and C stands for negative samples) of the spectrogram.
Since the audio files will be of varying length, the pre-processing step would involve padding all the shorter length samples with zero to have a fixed length for all files. So the spectrogram images of all those shorter samples will have the whale call in the beginning (or somewhere) with the majority of the frequency-time area as mere noise from the padding. (The above example divided the audio sample into some frames (to divide them into positive and negative classes) and labelled them as B,C.)
If we were to use the spectrogram images as such, this would hinder the generalization of our CNN model to a large extent.
Or if we save the output from the pre-processing to .npy format (binary form), I guess this could go unnoticed (or not?). What will be the consequences of saving the images in .npy format and then using in our model
I am not sure whether I am correct with my reasoning or not. Can anyone help me out?