Skip to main content
Data Science

Return to Question

Bumped by Community user
Bumped by Community user
improved title
Link
Leevo
Leevo
  • 6.4k
  • 3
  • 16
  • 52

Can we use log transformation for Tips on how to preprocess data and outliers? for churn analysis

Bumped by Community user
deleted 20 characters in body
Source Link
Jason
Jason
  • 49
  • 4

I am doing analysis on telecom churn dataset. I have 4617 observations and 17 variables. I am using Python. I have the following questions,

  1. When I skewness and kurtosis for the normality test, two variables are not normally distributed (value exceeds 1). Should I do log transformation for those two variables or else for the entire dataset?

  2. When I check the outliers using IQR method, close to 700 observations are outliers. I do not want to remove the outliers. Should I apply log transformation as well to address the outliers? Is it the correct way?

  3. I checked google, they say you can cap the outliers by taking percentile values. Is it a good practice to address the outliers?

  4. My final aim is to apply all classification algorithms. Can I do scaling after log transformation? Can we need to do scaling after log transformation? Tree models are ok without outliers but want to do for other models. My question is can we build the model after log transformation or we need to do scaling?

Please advise.

Regards, Dhilip

I am doing analysis on telecom churn dataset. I have 4617 observations and 17 variables. I am using Python. I have the following questions,

  1. When I skewness and kurtosis for the normality test, two variables are not normally distributed (value exceeds 1). Should I do log transformation for those two variables or else for the entire dataset?

  2. When I check the outliers using IQR method, close to 700 observations are outliers. I do not want to remove the outliers. Should I apply log transformation as well to address the outliers? Is it the correct way?

  3. I checked google, they say you can cap the outliers by taking percentile values. Is it a good practice to address the outliers?

  4. My final aim is to apply all classification algorithms. Can I do scaling after log transformation? Can we need to do scaling after log transformation? Tree models are ok without outliers but want to do for other models. My question is can we build the model after log transformation or we need to do scaling?

Please advise.

Regards, Dhilip

I am doing analysis on telecom churn dataset. I have 4617 observations and 17 variables. I am using Python. I have the following questions,

  1. When I skewness and kurtosis for the normality test, two variables are not normally distributed (value exceeds 1). Should I do log transformation for those two variables or else for the entire dataset?

  2. When I check the outliers using IQR method, close to 700 observations are outliers. I do not want to remove the outliers. Should I apply log transformation as well to address the outliers? Is it the correct way?

  3. I checked google, they say you can cap the outliers by taking percentile values. Is it a good practice to address the outliers?

  4. My final aim is to apply all classification algorithms. Can I do scaling after log transformation? Can we need to do scaling after log transformation? Tree models are ok without outliers but want to do for other models. My question is can we build the model after log transformation or we need to do scaling?

Please advise.

Source Link
Jason
Jason
  • 49
  • 4

Can we use log transformation for outliers?

I am doing analysis on telecom churn dataset. I have 4617 observations and 17 variables. I am using Python. I have the following questions,

  1. When I skewness and kurtosis for the normality test, two variables are not normally distributed (value exceeds 1). Should I do log transformation for those two variables or else for the entire dataset?

  2. When I check the outliers using IQR method, close to 700 observations are outliers. I do not want to remove the outliers. Should I apply log transformation as well to address the outliers? Is it the correct way?

  3. I checked google, they say you can cap the outliers by taking percentile values. Is it a good practice to address the outliers?

  4. My final aim is to apply all classification algorithms. Can I do scaling after log transformation? Can we need to do scaling after log transformation? Tree models are ok without outliers but want to do for other models. My question is can we build the model after log transformation or we need to do scaling?

Please advise.

Regards, Dhilip