1
$\begingroup$

I am trying write a program that continuously tracks the location a peak. To do that I need a very good peak detection algorithm. It not only has to tell the location of the peak but also the absence of it.

enter image description here

My current approach for peak detection:

  1. Baseline subtracted (account for different baseline signals)

  2. Divided by standard deviation of the smoothed signal (account for different baseline signals)

  3. I superimposed all the candidate peak data from all the spectra (local minima after gaussian smooth) and did a gaussian mixture in one dimension (y-axis; 2 clusters;100 spectra;3000 data points)

enter image description here

Yellow - "peaks" cluster; blue - "noise" cluster.

Trouble region: between 1 and 3 in the Y (cannot distinguish shallow peaks from spectra of with no peak).

How do I approach this problem to make the peak detection sensitive enough but still can tell if the peak is not there? Is there any better way to normalize the data? What unsupervised learning algorithm would be suitable for this(my data is unlabeled)? Please let me know if you have any suggestion. It will be greatly appreciated.

Examples:

Peak shapes and sizes, and noise level vary greatly:

enter image description here Very shallow and broad enter image description here Very narrow and sharp enter image description here Frequency response went outside of the sweep window (need to identify that the peak is absent)

enter image description here shifting baseline sometimes

How the data is obtained:

A peak can be thought of as a response of a harmonic oscillator in a frequency sweep.

One example of a harmonic oscillator is a swing. (or a ball on a spring; a ball in a well). They are all characterized by the periodic motion related to a natural position. In the swing example, the natural position is at the middle when the girl first got on the swing.

Imagine a person started periodically pushing her on the back as she swings, her swinging motion might increase. However, if the person pushes her at the wrong time, like pushing her back when her back is coming towards him, she will slow down and will not swing higher. This is because the frequency of the push (driving frequency) does not match the periodic movement of the swing (natural frequency). Imagine the person gradually increases the frequency of the push from really infrequent to really frequent (frequency sweep), the frequency of the push will at some point match the natural frequency of the swing (coming into resonance). In other words, every push of the person will be at the right time and will push her higher and higher.

Imagine you shine a very bright spot light at the natural position of the swing (the lowest position). The amount of time the girl spends inside of the spot light will depend on how high she swings. So the higher she swings, the less time she will spend in the light. Back to my plot, the signal (y axis) depends on how far the harmonic oscillator swings and the dip of the plot tells me at what driving frequency it comes into resonance.(driving frequency matches with natural frequency so it swings very far and decreases the signal)

A shifting baseline is due to a drift in the spot light brightness itself

I hope that would clarify things a bit.

enter image description here

$\endgroup$
4
  • $\begingroup$ I suspect you're going to need to give us a more precise definition of what counts as a peak for you. There are many things that could potentially be treated as peaks. I suggest reading about changepoint detection and kernel smoothing and doing some searching here and on Stats.SE. See also, e.g., stackoverflow.com/q/3260/781723. $\endgroup$
    – D.W.
    Jun 11, 2018 at 23:05
  • $\begingroup$ A peak can be thought of as a response of a harmonic oscillator in a frequency sweep. The signal indicates the amount of time the oscillator spends at the center (natural position). When the oscillator comes into resonance with the driving frequency, there is a decrease of signal because of the increase of oscillation (spending less time at the middle). As shown in the 3rd plot (shallow and broad), an oscillator response (a dip in the signal) is definitely noticeable to the human eye but more difficult to be identified by a computer. I will digest the info you gave me meanwhile. $\endgroup$
    – Chris
    Jun 11, 2018 at 23:36
  • $\begingroup$ I encourage you to edit the question to explain that in the question, and to develop the model you have in mind. I confess I don't understand what you mean (for instance, I don't know what "in a frequency sweep" means), so it would help to develop the model and explain it in a way that someone who didn't already have that idea would understand it. Perhaps you can explain the model of the harmonic oscillator, and how its state is related to or correlated with the observed value. $\endgroup$
    – D.W.
    Jun 11, 2018 at 23:53
  • $\begingroup$ Sorry I didn't explained it clearly. I added more explanation on the question. Hopefully that will clarify some things. $\endgroup$
    – Chris
    Jun 12, 2018 at 0:38

1 Answer 1

0
$\begingroup$

That is problem commonly called time series anomaly detection / outlier detection. Most systems start with combinations of static and dynamic thresholding. Dynamic thresholding for example can use percentile value as the threshold. One common algorithm is Isolation Forest where the features can including different length moving average windows.

One of the best ways to handle occasionally shifting baseline is to take a Bayesian approach. Generate a posterior estimate which is updated in an online fashion as the data distribution changes.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.