0
$\begingroup$

I have used MATLAB code and get the two different row vectors A=1×18 and B=1×350. From both row vectors separately I need to remove the noisy data by using standard deviation. But the problem is that data in both row vectors are NOT normally distributed. Is there any way that I used standard deviation for reducing noise from non normally distributed data. Any guidance will be appreciated. Thanks

$\endgroup$
0
$\begingroup$

First, good practice to raise validity concerns here when removing outliers and/or filtering data. This may have strong affects of validity of results. An intro is here: When to remove outlier in preparing features for machine learning algorithm .

Second, is it possible to address the small dataset problem -- Can you collect more data? Redefine the population to produce more data? Use another data set that is similar in developing the model?

Lastly, this seems to be a filter problem, to get started on a solution, check MATLAB documentation for filters.

If the results are going to be used anywhere, probably a good idea to document all of your decisions and the first two concerns. Absent much more detail, experience says there is a pretty high risk to any conclusions based on this model.

$\endgroup$
1
  • $\begingroup$ @ davmor thanks for your guidance. First I will try what you have mentioned in your answer, then will discuss. thanks $\endgroup$
    – user57546
    Sep 10 '18 at 14:30

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.