Hello I'm studying a classification problem with KNN right now.
I have many numeric features that I normalized with MinMaxScaler, I also got some OHE categorical features that not seem to cause the drop, and with this method my accuracy dropped almost 10%.
I used holdout method and cross validation for the results.
What are some follow up actions I could use to try to up my accuracy?
Should I try another normalization technique? Should I try another model ? Anything else?
Is bad luck.
Look what (I think) is happening:
When you use MinMaxScaler, what you do is relativize (reduce or augment) the distance between individuals in a way it was not before the change.
Let's suppose your best model is defined by the variable $X_j$ from the variable set $X=\{X_1,X_2,...,X_n\}$. When you change the set to an scalled one: $S=\{S_1,S_2,...,S_n\}$, what you do is giving the same importance to every variable, instead of preserving what the variables said before scalling.
$X_3$ gives better information on the output than $S_3$ (and in $S$, all are scalled), $S_3$ gives less information because is mixed with all other variables.
Imagine $X_3$ as something like "age" (and you are determining probability of having cancer), the greater the age, the greater the probability. This is true for $S_3$ also, but when $S_3$ is combined with the rest of $S$, $S_3$ looses importance amongst them. With $X_3$, age keeps its relative importance (bigger values amongst lesser values).
This does not happen very often, you could find yourself on the opposite situation: When combined $S$ might be a very powerful set.
That is why I think you were simply, unlucky.
