1
$\begingroup$

I am trying to implement (as a toy project) some aspects of speech recognition in Tensorflow. The audio files I want to use as training and test data have different sample rates (16, 20, 44 and 44.1 kHz).

The following list is not exhaustive, just listing a few options I could think of.

  1. I can resample the whole signal (which is quite un-periodic) using scipy.signal.resample, probably after padding with zeros to achieve a signal length which is a power of 2, chopping of the equivalent amount afterwards.
  2. I can try to chop the signal into short, non-overlapping windows, which I modify using scipy.signal.resample before glueing them back together and applying the rest of the chain.
  3. I can chop the signal up into overlapping windows, scipy.signal.resample them and stick them together in a weighted fashion.
  4. I can take the spectrogram tensor obtained using tf.contrib.signal.stft and try to stretch it, for example using rational function interpolation (reflecting tube models of the vocal tract)
  5. I can assume that the model will learn to abstract away from sample rates, maybe even helping it (or hindering?) by adding convolution layers before the actual model I am interested in.

In which step of the model should I normalize sampling rates, and how should I do it?

$\endgroup$
1
$\begingroup$

If you have ffmpeg installed, you can just use tf.contrib.ffmpeg.decode_audio. That function takes a samples_per_second argument and does the resampling for you. The following calculates a log magnitude short-time spectrogram of your data.

waveform = tf.squeeze(
    tf.contrib.ffmpeg.decode_audio(
        tf.read_file(tf.placeholder(tf.string, name="filename")),
        file_format=tf.placeholder(tf.string, name="format"),
        samples_per_second=p["sample_rate"],
        channel_count=1))

log_mag_spectrogram = tf.log(tf.abs(tf.contrib.signal.stft(
    waveform, win_length, hop_length,
    n_fft, pad_end=False) + 1e-8))
| improve this answer | |
$\endgroup$
  • $\begingroup$ That function is now deprecated. $\endgroup$ – Anaphory Dec 5 '18 at 22:50
0
$\begingroup$

Is there a reason you want to do the re-sampling inside the tensorflow computation graph? Otherwise, there are well-tested and established tools to do re-sampling as a separate step, consider running

$ sox unnormalized_file.wav -r 44100 normalized_file.wav

on all your data files.

| improve this answer | |
$\endgroup$
  • $\begingroup$ ffmpeg can do the same, but if you have ffmpeg installed, you can even use it from inside tensorflow, see my other answer. $\endgroup$ – Anaphory Sep 2 '18 at 21:09

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.