I am trying to implement (as a toy project) some aspects of speech recognition in Tensorflow. The audio files I want to use as training and test data have different sample rates (16, 20, 44 and 44.1 kHz).
The following list is not exhaustive, just listing a few options I could think of.
- I can resample the whole signal (which is quite un-periodic) using
scipy.signal.resample, probably after padding with zeros to achieve a signal length which is a power of 2, chopping of the equivalent amount afterwards.
- I can try to chop the signal into short, non-overlapping windows, which I modify using
scipy.signal.resamplebefore glueing them back together and applying the rest of the chain.
- I can chop the signal up into overlapping windows,
scipy.signal.resamplethem and stick them together in a weighted fashion.
- I can take the spectrogram tensor obtained using
tf.contrib.signal.stftand try to stretch it, for example using rational function interpolation (reflecting tube models of the vocal tract)
- I can assume that the model will learn to abstract away from sample rates, maybe even helping it (or hindering?) by adding convolution layers before the actual model I am interested in.
In which step of the model should I normalize sampling rates, and how should I do it?