I'm studing machine learning from here and the course uses 'Scikit Learn' for regression - https://www.udemy.com/machinelearning/

I can see that for some training regression algorithms, the author uses feature scaling and for some he doesn't because some 'Scikit Learn' regression algorithms take care of feature scaling by themselves.

  1. If I apply feature scaling to an algorithm which already has feature scaling internally, will it affect in any way ?

  2. Can I always apply feature scaling, no matter whether the algorithm I'm using has a system of feature scaling internally or not ?

  • $\begingroup$ It will help if you give some examples of algorithms that have internal scaling built-in. $\endgroup$ – rnso Dec 6 '18 at 13:30

Short answer: if you apply the same scaling method as what the algorithm does, it won't change anything. If you choose a different scaling method, then you'll end up performing two transforms on the original data, which may not be a bad thing depending on the context.

This implies that you have to know what kind of scaling the algorithm performs.

As a takeaway, it is good practice to scale the data beforehand yourself, so that you are sure what you input the algorithm with. An scaling rarely arms your model results.

As a note (I know this is not your question, but it may help), you have to understand why you scale the data. Taking the simple case of a linear regression, if your input features are house size and number of rooms and say you want to build a model of the house price from those features, number of rooms and size will have very different ranges. Scaling will help to:

  • make the algorithm converge more rapidly
  • allow you to answer questions such as "what is the main contributor to the price" by comparing the coefficients (this would be incorrect w/o scaling beforehand)
| improve this answer | |

I would say that feature scaling wouldn't significantly effect the performance of a model. What I would focus on would be what method of scaling you're using. Standard Scaling is less effected by outliers but has varying ranges, normalization squishes data ranges to 0-1 but is more effected by outliers, etc. Some Algorithms depend on the scaling method you are using, for example, neural networks often expect inputs between 0-1.

Then again, libraries like Scikit-Learn are built by experts that understand the inner workings of the model, so if the model has internal scaling, then I would just leave it be.

| improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.