I have a naive question about using the K Nearest Neighbor algorithm: is feature selection more important in KNN than in other algorithms?
If a particular feature is not predictive in a neural network, the network will just learn to ignore it. But in KNN, it seems like it could make the prediction worse, right? If I'm predicting height based on weight and age and gender, my model will get worse if I now add house numbers, because people will similar house numbers to me will be closer to me.
In a less extreme example, what if a feature is weakly predictive? Rather than normalize all my features so they have an equal weight, wouldn't I want to make the highly predictive features have more weight than less predictive ones?