Tips on how to preprocess data and outliers for churn analysis

Question

I am doing analysis on telecom churn dataset. I have 4617 observations and 17 variables. I am using Python. I have the following questions,

1) When I skewness and kurtosis for the normality test, two variables are not normally distributed (value exceeds 1). Should I do log transformation for those two variables or else for the entire dataset?

2) When I check the outliers using IQR method, close to 700 observations are outliers. I do not want to remove the outliers. Should I apply log transformation as well to address the outliers? Is it the correct way?

3) I checked google, they say you can cap the outliers by taking percentile values. Is it a good practice to address the outliers?

4) My final aim is to apply all classification algorithms. Can I do scaling after log transformation? Can we need to do scaling after log transformation? Tree models are ok without outliers but want to do for other models. My question is can we build the model after log transformation or we need to do scaling?

Please advise.

Answer 1

1) when the variable follows the non normal distribution you can use different transformation to make it normal, if variable follows the skewed distribution(either positive or negative) you can use log transformation to make it normal.

2) and 3) Instead of removing the outlier always go through outlier capping method, it will help to improve the performance of the model.

4) In machine learning some algorithm need to scale the data before go to modeling. for ex. cluster Analysis, Principal component Analysis(PCA). we can use scaling when the some values of the variable is too high as compare to the values of the other variable. (1,2,3,56,900,100,34,22,9) while using such type of algorithm always go to scaling instead of transformation.