I'm trying to figure out a method to perform some kind of average on my data, without smoothing the data. In my case, my data is a waveform, which means amplitude as a function of time. The problem with method such as the moving average, is that the result is a smoothed version of the original data, which means losing important information like sharp edges, which I need to perform my analysis.

Do you have any advice?

  • $\begingroup$ Can you provide some more information on what you're trying to achieve? I think any "averaging" operation will result in information loss, since you're combining multiple points of data into a single point of data. If averaging would destroy some of the information needed for your analysis, why perform averaging at all? $\endgroup$ – zachdj Nov 7 '19 at 18:21
  • $\begingroup$ Yes, I don't really mean averaging. I mean do some kind of operation to still keep the trend of the data, without actually smoothing it. Averaging retains the trend, but kills most of the information. For this reason, if there is a very sudden change of the trend, the moving average won't work. $\endgroup$ – Phys Nov 7 '19 at 18:26
  • 1
    $\begingroup$ I feel that signal processing can help you here. What about something like a fourier transformation/decomposition? I was trying to find a good overview, "anomaly detection fourier" yielded some interesting results. Some good discussion and links here too stats.stackexchange.com/questions/942/… $\endgroup$ – redhqs Nov 8 '19 at 11:03
  • $\begingroup$ I second @redhqs suggestion, and add another possibility: "Bilateral filters" are simple and designed such that they preserve edges better than box- or gaussian filtering. $\endgroup$ – bogovicj Nov 13 '19 at 12:52

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.