For a K nearest neighbors algorithm using a Euclidean distance metric, how does the algorithm compute euclidean distances when one(or all) of the features are categorical? Or does it just go by the most commonly occurring value among the neighbors?

So e.g. if the 2 features of 3 neighbors are age and gender with values: age,gender=[ [20,M], [31,F], [23,M] ], and we need to pick the 2 nearest neighbors for a new observation [20,F], how do we convert the gender to a numeric scale to compute euclidean distances?


1 Answer 1


It doesn't handle categorical features. This is a fundamental weakness of kNN. kNN doesn't work great in general when features are on different scales. This is especially true when one of the 'scales' is a category label. You have to decide how to convert categorical features to a numeric scale, and somehow assign inter-category distances in a way that makes sense with other features (like, age-age distances...but what is an age-category distance?).

If all features are categorical, and inter-category distances are all treated as somehow equal, your job actually gets a little easier since you aren't converting between categorical and scalar scales.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.