I've been reading on Tacotron-2, a text-to-speech system, that generates speech just-like humans (indistinguisahble from humans) using the github https://github.com/Rayhane-mamah/Tacotron-2.

I'm very confused about a simple aspect of text-to-speech even after reading the paper several times. Tacotron-2 generates spectogram frames for a given input-text. During training the dataset is a text sentence and its generated spectogram (it seems at a rate of 12.5 ms per spectogram frame).

  • If the input is provided as a character string, then how many spectogram frames does it predict for each character?
  • How does training supply which frames form the expected output from the dataset? Because the training dataset is simply a thousands of frames for a sentences, how does it know which frames are ideal output for a given character?

This basic aspect seems just not mentioned clearly anywhere and I'm having a hard time figuring this one out.

original paper: https://arxiv.org/pdf/1712.05884.pdf


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.