KNN with mixed data (feature set)

Question

I have a dataset where the feature set consists of hour of the day (between 0 to 23), day of the week (Monday to Sunday), number of shops (a positive integer) and road category (0 to 8 on an ordinal scale). I wish to predict the pedestrian volume which is a continuous variable. Hence I was looking a regression approaches and was thinking along the lines of using KNN. Will it be appropriate for this task? I am especially worried about the hour and day data as they are cyclic in nature.

df_test[['Day', 'Time','shop_count','road_category','pedestrian_volume']].iloc[0:5]

           Day  Time        shop_count       road_category  pedestrian_volume
  0    Tuesday    20                22                   0             210.0
  1    Tuesday    21                13                   4             196.0
  2    Tuesday    22                39                   4             214.0
  3    Tuesday    23                 2                   8              46.0
  4  Wednesday     0                15                   8              18.0

Answer 1

KNN is a classification algorithm where, typically, continuous variables are used to apply a classification. Your problem is a problem of predicting a continuous value using variables that are not continuous.

Your problem presents several challenges which, to me, would suggest KNN isn’t a good choice.

1. The biggest problem is probably the road_category where you have an ordinal value. Is category 4 four times greater than the difference between category 0 and category 1? Probably not.
2. No matter how deep your data, due to the fact that your predictors are integers or categorical, your neighbors won’t really be neighbors, more like roommates in a warehouse; you will probably have hundreds of observations on the same vector. So, now you have to choose k values from m observations. So, you will need to come up with an algorithm to do that. Then you’ll have to come up with a methodology to get a prediction. An average might work, but depending on your application, Min or max may be more beneficial.

Bottom line, KNN expects the variables to be close to the same order of magnitude otherwise, larger variables tend to determine the neighbors even if they are not necessarily more important. Some limited categorical variables might be accommodated using dummy variables, but I’d be really careful.

Based on your description of the problem, I think you should continue with regression, but use dummy variable encoding for your categorical values.

Answer 2

I would check if weekdays have similar distribution of pedestrian_volume, and do the same check for weekends. If that's true I would just create a binary variable "week day".

For Time, if say an observation with Time 23 is coming in, kNN probably will not be able to find its neighbors with Time 0, but the neighbors with 23 or 22 might be good enough. If not you can convert time to categories, for example: "night", "morning", "lunch time", "afternoon", "evening" and see if this does better.