1
$\begingroup$

I have seen few post related to this question but i am not quite clear about my confusions as mention bellow.

I have some confusion related to fit and fit_transform. suppose, I have X_train and X_test data, and let my scaling function is standard scalar. I am using following code for scaling, sc_X =StandardScaler() X_train = sc_X.fit_transform(X_train) X_test = sc_X.fit_transform`

My question is, if i use same scalar on bot trainin and testing data, wont it creat problem of data leakage?

What if I use the code like below,

sc_X_train =StandardScaler() sc_X_test =StandardScaler() X_train = sc_X_train.fit_transform(X_train) X_test = sc_X_test.fit_transform(X_test)

Does the both code give different results?

Improve this question
$\endgroup$
1
  • $\begingroup$ Your two approaches produce the same results (except that in the second case you have kept both sets of statistics); it is not the correct approach. It doesn't lead to data leakage, but scaling the test set independently isn't great either: especially, what do you expect to happen in production, if you want to make predictions on a single sample? $\endgroup$ – Ben Reiniger Jun 22 '20 at 14:59
0
$\begingroup$

In real life problems you may not not have the actual “test data” to fit on, for example in some time series forecasting problems. My recommendation if you want to keep safe and avoid data leakage is to fit on training data and transform on test.

Improve this answer
$\endgroup$

Not the answer you're looking for? Browse other questions tagged or ask your own question.