3
$\begingroup$

I would like to analyze speech samples looking for speech-language pathologies. Most of the resources I can find are about speech recognition which is a completely different problem.

  1. I will always know what is the text that is being spoken on the given sample.
  2. I can assume that audio samples will always be of good quality with no background noises.

I would like to construct a neural net(or maybe some different model) that will detect certain anomalies, but I have few questions.

  1. How should I pass the input to the net? The input will have two parts: speech and text being spoken (+ label). Should I make I somehow divide these parts or just concatenate them as one sample?
  2. What can I read to better understand working with audio/speech in ML environment?
$\endgroup$
2
$\begingroup$

I think you are too quick to throw AI at this problem. (As you already noted, keep alternatives to neural networks open: maybe factor analysis or principal component analysis are more suitable, or support vector machines)

First, you have to do more work in the problem domain, to find out that different pathologies may result in different models.

My recommendation would be to start small with one pathology. Use domain knowledge to determine if (and to what extent) the pathology is present in some samples, and how that could be observed from the audio samples, in a limited number of parameters.

For some pathologies, speech recognition would help. You could count differences in the text spoken and the text recognized. The count (and maybe a pattern in those counts) could be an input for the model (not necessary a neural network as you already noted).

Some other pathologies would need audio spectral or temporal observations in the audio samples (unnatural pitch, unnatural pauses).

To give an example: stuttering.

I would look in the audio samples and spectra for differences in the patterns. E.g. a spectral plot of stuttering vs. regular speech could look like: enter image description here

So typically you'd want to detect/measure relative power in some frequency regions as a measure for stuttering and learn your model with such data.

An alternative that might also work is to run a stuttering sample (and a natural sample) through speech recognition, to see if differences can be observed in a countable/measurable way.

| improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.