1
$\begingroup$

I have some features which are in the thousands, which I scale to the max values of these. This solves the general scaling problems, as well as preserves an important absolute value relationship between these features that would otherwise be lost.

But, I also have some other features which are naturally in [0,1] (since they're derived from ratios) right from the start. Does it make sense to simply skip scaling of these, since dividing them by the same max value as above (e.g. 10,000) kind of ruins them?

Obviously I've tested it out, and found that it works in practice, but are there any sound mathematical arguments against this? I haven't seen or heard any thoughts on this matter.

$\endgroup$
1
$\begingroup$

Yes, these features should not be scaled together. Normally ALL features are scaled individually, so grouped scaling is the exception not the norm. From the Wikipedia on feature scaling:

Since the range of values of raw data varies widely, in some machine learning algorithms, objective functions will not work properly without normalization. For example, the majority of classifiers calculate the distance between two points by the Euclidean distance. If one of the features has a broad range of values, the distance will be governed by this particular feature. Therefore, the range of all features should be normalized so that each feature contributes approximately proportionately to the final distance.

You can scale some features together if you know that they exist on the same scale, i.e. that "Sales January" is scaled together with "Sales February". But you should not scale things together that exist on different scales, i.e. "Sales January" and "Discount Ratio" should be scaled separately.

$\endgroup$
  • 1
    $\begingroup$ "Normally" is not really an argument though. Do you have any sources or empirical evidence? $\endgroup$ – komodovaran_ May 13 at 11:04
  • 1
    $\begingroup$ Added a resource stating why feature scaling is done per feature. $\endgroup$ – Simon Larsson May 13 at 11:16

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.