0
$\begingroup$

I was wondering if having dummies variable and scaling other variables could joke my model. In particular, I have implemented a Random Forest Regressor by using scikit-learn, but my data model is composed by a set of dummies varibles and 2 numerical variables. I approached in this way:

  1. Convert categorical in dummies variables
  2. Separate the numerical variables
  3. Scale with Standard Scaler from scikit-learn the numerical variables (at point 2)
  4. Join the dummies and numerical
  5. Split train, test
  6. train the model

Would this approach create an inappropriate bias considering the different scale from dummies and the scaler numerical? Or, at least, is it correct?

$\endgroup$

1 Answer 1

0
$\begingroup$

Would this approach create an inappropriate bias considering the different scale from dummies and the scaler numerical? Or, at least, is it correct?

"It is fine" because One-Hot-Encoded features don't have a very large scale. Issue of scale comes when features are at a very different scale e.g. 1 vs 105 etc.
Standardizing them will disturb the sparsity(i.e. log of zeroes). Normalizing them will not have any impact.

What is incorrect here is -
You are standardizing the dataset prior to the split. You must do the other way round.

What is optional here is -
You don't need scaling and OHE for a Tree-based model. Just label encode, split and move.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.