I've explored text-to-speech evaluation matrices and they seem to used Mean Opinion Score (MOS) to evaluate a particular model. This matrice required humans to help to judge the model based on a scale (Bad, Moderate, Good, Etc.).

Are there other evaluation matrices that algorithmically estimate the TTS system and don't require any human? but it still gives the result that correlated to human evaluation?

  • 1
    $\begingroup$ In most language-related tasks evaluation is based on human annotations, simply because these tasks are meant to produce an output meaningful to humans. The only way to know if something is meaningful to a human is to ask a human, any automatic measure cannot be as reliable. $\endgroup$
    – Erwan
    Oct 4, 2021 at 19:31
  • $\begingroup$ @Erwan, care to upgrade that comment to an answer? $\endgroup$
    – Ben Reiniger
    Apr 9 at 13:35

1 Answer 1


The Mean Opinion Score (MOS) is a numerical expression of the overall performance of such an event or incident as rated by humans. Mean Opinion Scores are now the median of a variety of other human-scored different parameters, and are usually graded on a scale from 1 to 5.

A Mean Opinion Score would be a measurement of the quality of audio and data sessions in telecommunication services. An Objective Measurement Technique that approximates a human ranking generates a MOS. This is frequently used in practise to assess digital approximations of real-world phenomena.

Human involvement might be the most efficient method to evaluate MOS within a reasonable size, but this is not necessarily the most practicable. The final MOS score is calculated by averaging the scores of all individuals, resulting in a wide range of 0-5. A score of 5 indicates a high-quality call, while a score of 0 indicates incomprehensible communication.

The R-Factor (Rating-Factor) seems to be a human-based MOS score estimation. All of those are conversation quality measures that are not even caused by network errors. Clarity, latency, packet loss, jitter, as well as other parameters are measured using real audio signals.

  • $\begingroup$ This doesn't seem to answer the question. $\endgroup$
    – Ben Reiniger
    Apr 9 at 13:36

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.