2
$\begingroup$

The following code gives the target variable Item_Outlet_Sales before transformation and Item_Outlet_Sales_log which is transformed

#treat extreme values in Item_Outlet_Sales
train['Item_Outlet_Sales_log'] = np.log(train.Item_Outlet_Sales)
test['Item_Outlet_Sales_log'] = np.log(test.Item_Outlet_Sales)

plt.figure(1)
plt.subplot(121)
sns.distplot(train.Item_Outlet_Sales)
sns.distplot(test.Item_Outlet_Sales);
plt.subplot(122)
sns.distplot(train.Item_Outlet_Sales_log)
sns.distplot(test.Item_Outlet_Sales_log);

enter image description here

Then use the new target variable (Outlet_Item_Sales):

#creating dummies for the training dataset
X = train.drop('Item_Outlet_Sales', 1) #drop the log target column
y = train.Item_Outlet_Sales_log

X = pd.get_dummies(X)
train = pd.get_dummies(temp_train)
$\endgroup$
3
$\begingroup$

If your objective is to convert non-normal data into something that looks more normal / gaussian - try the Box-Cox Transform here.

It is a family of transforms that looks at your data - and provides the best possible transformation.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.