I would like to ask if Savitzky-Golay can be implemented on real-time data.

I have used it on a fixed array size, but would like to extend it to output values for real-time sensor data. Can anyone refer me to appropriate implementation or hint online implementation. Thanks.

  • 1
    $\begingroup$ Why not buffering n data points from your buffer and use that "n" as the window size of the Savitzky-Golay filter that you output? There is then a small lag of "n" data points (or n//2 if consider only the middle point) $\endgroup$ – Wli Feb 14 '17 at 15:45
  • $\begingroup$ Appreciate you comment. But I prefer not to introduce any time lag. Are there other noise filtering techniques for real-time processing? $\endgroup$ – Abdullah Nazir Feb 15 '17 at 15:48
  • $\begingroup$ Well you can use the algorithm on the data points between m-n and m, where m is the last data point index. I mean an moving average does exactly that with a polynomial of order 1, and wikipedia tells me that moving averages are commonly used to smooth time series. $\endgroup$ – Wli Feb 15 '17 at 15:58

The original Savitzky-Golay paper addressed smoothing, meaning that you estimate a value for some point in the past, using values from its past and future. It sounds like what you're interested in is filtering: at time step k, you want to reduce noise and come up with a better estimate of the value at the same time step k, without knowing the future. That's important in many fields such as process control, and some fault diagnosis where delays can't be tolerated.

The SG approach of calculating the constants for least squares fitting for smoothing has also been done for filtering. A curve is fit through n recent points, and instead of estimating the middle point, the end point is estimated by a least squares fit, reduced to simple multiplications by constants. There are tables of the necessary coefficients and an example at https://gregstanleyandassociates.com/whitepapers/FaultDiagnosis/Filtering/LeastSquares-Filter/leastsquares-filter.htm

There is no time delay in the calculations, unlike with smoothing. Also, if the input signal is a ramp, the filter will converge to the correct value once all of the points in the time window reflect that ramp. On the other hand, there are some side effects that may or may not matter to you. Even in the linear case, for a step change in the input, there is overshoot in the filter output. Higher order least-squares filtering can be quirkier, so it is rarely used. But the linear version overall is quite effective, assuming the little bit of overshoot isn't a real problem (for instance, if you don't see step inputs in the data anyway).

The link above is part of a more general "Guide to Filtering", that covers some other common filters as well, such as the exponential filter and moving average filters in particular. (Filters don't have time delays, but most do lag the inputs - meaning smear out changes over time. That's a byproduct of filtering out the noise, essentially cancelling out some new noise by a certain amount of averaging against old noise). It also covers general issues for filtering, such as aliasing.

There are two other filters that are not covered in the Guide that might be considered close competitors to the SG-style filtering: the alpha-beta filter and double exponential smoothing. These are covered in Wikipedia: alpha-beta filtering has its own page, and double exponential smoothing is part of the page on exponential smoothing. The goal of both is to filter/estimate both the values and the derivative, resulting in a filter that will give correct outputs to a ramp input signal. I've long suspected that these two filters are essentially the same, but never taken the time to look into it.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.