Data Science Stack Exchange is a question and answer site for Data science professionals, Machine Learning specialists, and those interested in learning more about the field. It only takes a minute to sign up.
Sign up to join this community
When I am cleaning my data, I have some features which contain large numbers and some features that are binary. Should I scale the large features and then add the binary columns or just scale them all together?
My fear is that scaling them all together makes the binary features seem less important than they really are.
Note: I am prepping a neural network for binary classification. I am using a sigmoid output and scaling my features from [0,1]
There are two important concepts that you need to understand.
Intuitively, your neural network will itself learn which feature is important or not by learning weights.
Intuitively, scaling features can make your training faster because your "path to convergence" will probably be shorter if your features are scaled (right picture).
It also makes numerical sense to scale your features because if you have very large and very small values, some of your weights might drop very low and this could cause some numerical issues and hinder the performance of your model.
