Background Information:

  • Dynamic Time Warping (DTW):

In time series analysis, dynamic time warping (DTW) is one of the algorithms for measuring similarity between two temporal sequences, which may vary in speed. (Source: Wikipedia)

  • Phoneme Segmentation:

Phoneme segmentation is the ability to break words down into individual sounds. For example, a child may break the word “sand” into its component sounds – /sss/, /aaa/, /nnn/, and /d/. (Source)

The Question(s):

(a) How can we do phoneme segmentation using DTW?

(b) Which type of data do we need to implement this idea? (by asking about the type of the data, we mean which features should be available for each sample in the training dataset).

My try:

Assume that we have some audio files in each the phoneme segments are completely calculated and available. For instance, we know that from $t_1$ to $t_2$, the speaker is just saying /a/, and the other parts also have this kind of label. If we have a new sample which the system has not seen yet, a simple approach would be to calculate the difference between the new sample and each of the training samples. This approach would be like a KNN (K nearest neighbors) algorithm. We can just cast a vote to see which phoneme wins the game.

Another case is when the data is not labeled. In this case, I think we may be able to do some kind of clustering (e.g., K-means) to extract some cluster means, and use them. We could just calculate the distance between the new sample and the means of the clusters (which would be much faster than the previous calculations we had for the other case).

The problem is that these approaches seem too simple and inefficient to me. Is there a better (or smarter) way to tackle this problem of segmenting the phonemes using DTW? Should the samples have any other kind of features? (By other, I mean other than the time segments for each phoneme being specified).

  • $\begingroup$ I'm not expert in signal processing but I suspect that DTW is not necessarily the right approach: it's a distance measure, so it works only for comparing two given segments. The first method you mention looks more like classifying a segment, but this assumes that we already know where the segment starts and ends. I would imagine that segmentation models are trained using specific data annotated with segment start/end position. $\endgroup$
    – Erwan
    Jan 8, 2021 at 23:48
  • $\begingroup$ @Erwan I am not an expert either. However, I saw a course where this question was asked by the professor. So, assume that whatever type of data that we need would be available.(a) What type of data should we have? (b) How should we use DTW to solve the problem? (a) and (b) are my questions. $\endgroup$
    – Sam Kagawa
    Jan 9, 2021 at 0:11
  • 1
    $\begingroup$ phoneme segmentation is different than DTW. DTW is used on already segmented phonemes. So phoneme segmentation requires some pre-processing and breaking down the audio signal in small timeframes up to phonemes. Note that DTW as a recognition technique has been surpassed by statistical methods (like HMMs) $\endgroup$
    – Nikos M.
    Jan 9, 2021 at 9:41
  • 1
    $\begingroup$ here are some links to phoneme extraction to get you started: authot.com/en/2017/11/22/phoneme-detection-speech-recognition, researchgate.net/figure/… $\endgroup$
    – Nikos M.
    Jan 9, 2021 at 9:43


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Browse other questions tagged or ask your own question.