When I am cleaning my data, I have some features which contain large numbers and some features that are binary. Should I scale the large features and then add the binary columns or just scale them all together?

My fear is that scaling them all together makes the binary features seem less important than they really are.

Note: I am prepping a neural network for binary classification. I am using a sigmoid output and scaling my features from [0,1]

  • $\begingroup$ Typically binary features are represented as $\{0,1\}$. If you scale such features into 0 to 1 range then they would not be affected. Could you clarify whether you are doing anything different (such as starting with $\{-1,1\}$, or scaling to mean 0, standard deviation 1)? $\endgroup$ – Neil Slater Sep 15 '17 at 6:43

There are two important concepts that you need to understand.

  • Scaling your features won't affect their "importance" in your neural network

Intuitively, your neural network will itself learn which feature is important or not by learning weights.

  • Scaling your features will speed up your convergence and will limit the risks of overshooting or being stuck in a local optimum. It also makes numerical sense.

Intuitively, scaling features can make your training faster because your "path to convergence" will probably be shorter if your features are scaled (right picture).

It also makes numerical sense to scale your features because if you have very large and very small values, some of your weights might drop very low and this could cause some numerical issues and hinder the performance of your model.

enter image description here


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.