2
$\begingroup$

Having some skewed features as shown in the following figure. I am trying to imply log transformation to the feature called vBMD(mgHA/cm3). I run the following codes

Distribution Plots

import numpy as np

import pandas as pd

from sklearn.preprocessing import MinMaxScaler

df=pd.read_csv("Data.csv")

scaler=MinMaxScaler(feature_range=(0,1))

df['vBMD (mgHA/cm3)']=scaler.fit_transform(np.array(df['vBMD (mgHA/cm3)']).reshape(-1,1))

df['vBMD (mgHA/cm3)']=np.log(np.array(df['vBMD (mgHA/cm3)']))

After the transfromation, I have got the following result. After transformation

While I am waiting that the feature will be normalized, its skewness increased. Thus, what am I doing wrong?

$\endgroup$
2
$\begingroup$

Log transformation leads to a normal distribution only for log-normal distributions. Not all distributions are log-normal, meaning they will not become normal after the log transformation.

EDIT:

As you have commented, if you are trying to convert an arbitrary distribution to normal, methods like QuantileTransformer can be used. But note that these transformations make a distribution normal by changing (destroying) some information from the original data.

$\endgroup$
  • 1
    $\begingroup$ Thanks a lot. I figure out the main idea. In my case, to make mentioned feature normalize, can I use QuantileTransformer such as quantile_transformer = QuantileTransformer(output_distribution='normal', random_state=0) df['vBMD (mgHA/cm3)'] = quantile_transformer.fit_transform(np.array(df['vBMD (mgHA/cm3)']).reshape(-1,1)) $\endgroup$ – Ram Mar 6 at 11:57

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.