How to normalize noisy data

Question

Suppose I have 1-D data which has some outliers, I want to normalize the data to be in the range [0,1]. I tried calculating the maximum value and the minimum value as follows:

q1,q2,q3 = quartiles of the data
max = q3 + (q3-q1)*1.5
min = q1 - (q3-q1)*1.5

I used the above approach because I have read that data above maximum or data less than the minimum (as calculated above is noise).

My question is: whatever I am doing, is it correct or is there any other way to achieve good results?

Thank you for helping.

Answer 1

The question,

whatever I'm doing is it correct or ...

will occur over and over again, until you have a firm understanding of statistics. See if you are looking for short-term learning, then you might pick up some fragments of information here and there, but, for long term learning, I'll suggest that you read a good text on introductory statistics. Try, this book, or this one on Applied Predictive Modeling

Now to answer the question, Yes what you are doing is correct. I've explained it below;

Outliers are observations that fall below the lower limit or above the upper limit. The five number summary of a continuous variable (or numerical variable) is Min, Q1, Q2, Q3, Max. Where the lower and upper limits are defined as, Lower Limit = Q1 - 1.5 x IQR, Upper Limit = Q3 + 1.5 x IQR and IQR is the Inter Quartile Range, which is the difference between the first and third quartile. Q1 is the first quartile, Q2 is the second quartile or the median and Q3 is the third quartile.

Finally, please always plot your data.