I am building an LSTM to recognize if the person is sad, happy, angry or neutral. This is done by feeding-in his the wave of his voice into the network, as a sequence of bytes (each byte is 0-to-255).

The problem is, my dataset is not large enough, are there efficient ways I could augument my dataset? I am training on short 1.5 second clips and I have 800 of those, which is not enough.

My current augumentation is:

  1. to add variations in volume
  2. to add a bit of white noise, which makes it worse :(

Reversing the sequences doesn't seem to be applicable, after all, my network will be predicting non-reversed speech when it's fully trained.


1 Answer 1


Your problem is to identify whether a person is sad, happy, angry or neutral from his voice which is emotion classification problem. For speech, we use frames of short duration like 10-20 ms, and extract features from the same like MFCC or other frequency domain features. We extract short duration frames because speech is non-stationary over time, and frequency domain features make the features shift-invariant.

I suggest you read some latest research papers on emotion classification to get latest research in emotion classification.

Emotion in speech is captured by variation in pitch and amplitude across time. So capturing short time energy and instantaneous pitch frequency in speech across time are basic features for emotion classification.

Modifying speech by adding variation in volume and white noise will not help at all.

You need to build a emotion classification system with 4 classes: sad, happy, angry or neutral, using the below steps:

  1. Extract frames of 10-30 ms duration
  2. Convert these frames to frequency domain features like STFT or extract instantaneous pitch
  3. Use these STFT features along with pitch to train a LSTM

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.