# Normalization(minmax) gives me worse results than before in KNN, follow up actions?

Hello I'm studying a classification problem with KNN right now.

I have many numeric features that I normalized with MinMaxScaler, I also got some OHE categorical features that not seem to cause the drop, and with this method my accuracy dropped almost 10%.

I used holdout method and cross validation for the results.

What are some follow up actions I could use to try to up my accuracy?

Should I try another normalization technique? Should I try another model ? Anything else?

Look what (I think) is happening:

When you use MinMaxScaler, what you do is relativize (reduce or augment) the distance between individuals in a way it was not before the change.

Let's suppose your best model is defined by the variable $$X_j$$ from the variable set $$X=\{X_1,X_2,...,X_n\}$$. When you change the set to an scalled one: $$S=\{S_1,S_2,...,S_n\}$$, what you do is giving the same importance to every variable, instead of preserving what the variables said before scalling.

$$X_3$$ gives better information on the output than $$S_3$$ (and in $$S$$, all are scalled), $$S_3$$ gives less information because is mixed with all other variables.

Imagine $$X_3$$ as something like "age" (and you are determining probability of having cancer), the greater the age, the greater the probability. This is true for $$S_3$$ also, but when $$S_3$$ is combined with the rest of $$S$$, $$S_3$$ looses importance amongst them. With $$X_3$$, age keeps its relative importance (bigger values amongst lesser values).

This does not happen very often, you could find yourself on the opposite situation: When combined $$S$$ might be a very powerful set.

That is why I think you were simply, unlucky.

• So should I use another normalization technique or that's not gonna work again? – dungeon Apr 22 '19 at 19:00
• It might work, try another normalization technique and observe what happens. Try to use trees too. – Juan Esteban de la Calle Apr 22 '19 at 19:02
• Can you be more specific about what you mean about trees? I'm just learning is this just a different model (decision tree model?) I'm planning to use many models on this problem but I was trying to get better results from knn – dungeon Apr 23 '19 at 16:35
• The trees rely on intervals when working with numeric variables, so for trees is the same to work with $X$ than to work with $S$, because the separations will be the same. If you work with scalled variables there is also no interaction. – Juan Esteban de la Calle Apr 23 '19 at 20:22