12
$\begingroup$

I was looking into the possibility to classify sound (for example sounds of animals) using spectrograms. The idea is to use a deep convolutional neural networks to recognize segments in the spectrogram and output one (or many) class labels. This is not a new idea (see for example whale sound classification or music style recognition).

The problem that I'm facing is that I have sound files of different length and therefore spectrograms of different sizes. So far, every approach I have seen uses a fixed size sound sample but I can't do that because my sound file might be 10 seconds or 2 minutes long.

With, for example, a bird sound in the beginning and a frog sound at the end (output should be "Bird, Frog"). My current solution would be to add a temporal component to the neural network (creating more of a recurrent neural network) but I would like to keep it simple for now. Any ideas, links, tutorials, ...?

Improve this question
$\endgroup$
4
  • 1
    $\begingroup$ The simplest way is to use a fixed-length FFT instead of an STFT (spectrogram). That will eliminate your variable-length problem. Why don't you just apply a recurrent neural network? Do you just need a worked example? If so, are you flexible about which software to use? $\endgroup$
    – Emre
    Commented Jan 29, 2016 at 20:29
  • 1
    $\begingroup$ I think I would lose a lot of information with a fixed-length FFT. If I would do that I think I would have to do a segmentation first, where I look for 'interesting' parts. I might do that or go to the recurrent neural networks (an example is nice but not super necessary, I wanted to use Lasagne). The reason I tried to avoid it is that the output of a recurrent neural network is more difficult to deal with (at each time step but I only have the labels for the whole file). So I wanted to start with the simplest model and gradually make it more complex. $\endgroup$
    – user667804
    Commented Jan 30, 2016 at 0:54
  • $\begingroup$ could you please tell what you ended up using and the best approach you found? @user667804 $\endgroup$
    – nia
    Commented Jul 2, 2017 at 6:35
  • $\begingroup$ Check out this paper for a solution: ceur-ws.org/Vol-1609/16090547.pdf Using a CNN on fixed sized chunks of the spectrogram and then averaging the outputs to generate one final prediction (mean of the indidivual outputs seems to work best). $\endgroup$
    – user667804
    Commented Jul 2, 2017 at 11:32

2 Answers 2

Reset to default
5
$\begingroup$

RNNs were not producing good enough results and are also hard to train so I went with CNNs.

Because a specific animal sound is only a few seconds long we can divide the spectrogram into chunks. I used a length of 3 seconds. We then perform classification on each chunk and average the outputs to create a single prediction per audio file. This works really well and is also simple to implement.

A more in-depth explanation can be found here: http://ceur-ws.org/Vol-1609/16090547.pdf

Improve this answer
$\endgroup$
4
$\begingroup$

For automatic speech recognition (ASR), filter bank features perform as good as CNN on spectrograms Table 1. You can train a DBN-DNN system on fbank for classifying animals sounds.

In practice longer speech utterances are divided into shorter utterances since Viterbi decoding doesn't work well for longer utterances. You could do the same.

You can divide the longer utterances into smaller utterances of fixed length. Dividing the longer utterances into smaller is easy. The problem comes in increasing the length the smaller utterances to reach fixed length.

You could warp the frequency axis of the spectrogram for augmenting the smaller utterances. This data augmentation has been shown to improve ASR performance data augumentation.

For a longer utterance with multiple sounds in it, you could use music segmentation algorithms to divide it into multiple utterances. These utterances can be made of fixed length either by division or augmentation.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.