# Identification of inexactly-recurring material in time series stream

I am working on a personal project involving the analysis of a stream of audio data and the identification of (non-verbatim) repeated subsequences.

My research on time series has so far lead me to publications on LSTMs, CTC, wavelet packets, and motif discovery using "shapelets". However, I am unsure as to whether or not these strategies are applicable to my particular situation.

As an undergraduate EE student, none of my classes so far have covered time series or data streams in much detail, if at all.

In case a more specific description of my aims would help, I've included one below. If you'd rather skip to the questions, see "Uncertainties" at the end of this post.

I am not sure which direction to go in and would really appreciate any guidance or suggestions.

Thanks!

# Three-Point Outline

## Input

• Two time series streams (or "streaming time series"), A0 and B0, each consisting of a different set of features computed from a stream of input data.

## Intermediate Stage

In no particular order (or potentially simultaneously):

• Calculate another time series (stream) B1, consisting of the derivative (change) in each of the features B0.
• Calculate another time series (stream) A1, indicating redundant (or loosely redundant), superseded subsequences in A0 (see below). Note that A0 is multivariate while A1 is univariate.

## Output

• Another time series, C, indicating the locations of portions of A0 to remove based on the repetition information (A1) and contour cues from B0 and B1.

# Definition of Redundancy

1. A segment Q is considered redundant if there is an earlier-in-time segment P whose content closely resembles that of Q.

2. "Close resemblance" allows differences in timing, short insertions, short deletions, and brief variation (when surrounded by otherwise consistent material).

3. Ideally, "close resemblance" also permits exchanges in position of sub-sub-sequences of sufficient length. (I.e. "ABCDE" closely resembles "ADBCE" and vice versa.)

4. When this "close resemblance" coincides with appropriate cues in B0 and B1, the earlier segment P is considered to be superseded by the later segment Q, and so P is marked for removal.

5. This designation for removal may be to some degree "fuzzy", if for example the exact start and end points of P cannot be identified with total precision.

6. Note that there may be several non-identical segments P1, P2, P3 (possibly with very short intervening material between them) that are all superseded by Q. (In such a case, P2 likely supersedes P1, and P3 likely supersedes P2.)

7. The lengths of P and Q are not guaranteed to be the same, nor are they defined as parameters.

8. There may be a relatively short segment of intervening material between P and Q.

# Uncertainties

1. How to process an input data stream and also preserve some amount of it for the processing of a subsequent portion or portions

2. How to use the derivative of a feature set time series as an additional feature set time series

3. How to efficiently examine the input for reuse of similar subsequences, and then produce another time series reflecting the locations of these

I have worked on this problem:

See

1. STAMPi (Matrix Profile I: All Pairs Similarity Joins for Time Series: A Unifying View that Includes Motifs, Discords and Shapelets.)

2. Rare Time Series Motif Discovery from Unbounded Streams.

Let me know if you have any questions.

• Thank you for the prompt response! I am about to begin reading the publications you listed, so I don't have questions at the moment but I will probably accumulate them over the course of my reading. I don't know how appropriate it is to carry on a discussion here in the comments, so would it be okay if I sent my questions to your UCR email address instead? There are also Stack Exchange "chat rooms", but I don't know if it's acceptable to create a room for the sole purpose of discussing a single post.
– Stan
Commented Oct 3, 2023 at 14:37