1
$\begingroup$

I have chosen my synopsis topic for my second last semester.
I want to make a text-to-speech program, that speaks with the voice of a game character.

I have worked with machine learning in my class, but nothing that covers audio or text-to-speech.
I know how big of a project I have taken on my shoulders, but that is just what I need.

I don't know where to start at all, since there is way too much information out there. In other programming areas, I'm usually pretty good at finding information, but I just don't know enough about ML yet to do that here (plus the keywords used online kind of contradict each other).
I'm not sure which tools I need to look at. Where should I begin?

I have 68.871 audio files, which is about 76 hours of audio in total. The files are of different lengths, they start at around under a second and end around 49 mins. I have not heard all the audio myself, but I'm pretty sure all of it is completely clean.

(I know that I probably can't feed any model with 68.871 files, that could all be of different lengths, so I think I have to pick out a smaller sample size, with more equal lengths)

I would like to use these files as audio samples, so a model can learn how to speak like this character.

Any push in the right direction would be deeply appreciated.

Improve this question
$\endgroup$

2 Answers 2

Reset to default
0
$\begingroup$

Try to look into existing text-to-speech/speech synthesis. This link list multiple different models/approaches in this domain. When trying to look at state-of-the-art (SOTA) methods in this domain I find this GitHub page on which a lot of papers can be found on audio and speech. A very good (SOTA) model/architecture seems to be Tacotron 2/WaveNet. Hope this helps you with your project.

Improve this answer
$\endgroup$
1
  • $\begingroup$ thank you for the tips, I will take a look at those. btw, I have been cleaning my dataset, only 19.357 of the files had the voice I needed, so I went from 76 hours to 18.30 hours $\endgroup$
    – Repeller
    Apr 8, 2020 at 20:41
0
$\begingroup$

Sounds like a cool project - perhaps you could look into fastspeech and NVIDIA WaveGlow. If you're going to build something "from scratch" that's quite ambitious IMHO.

You're going to need matching transcriptions of each audio file for your training data - if you don't have these you could get create them fairly quickly with a general Speech-to-Text / transcription service or model.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.