Good day, I had this question set as optional homework and wanted to ask for some input.
Suppose an individual was to take a data set, divide it in half into training and test data sets and then try out two different classification procedures.
First they use logistic regression and get an error rate of 20% on the training data and 30% on the test data. Next they use 1-nearest neighbours and get an average error rate (average over both test and training data sets) of 18%. According to these numbers, which method would you prefer to use for classification purposes (of new observations)? Why?
I am inclined to say kNN as it is a rather flexible approach and provides a lower error on average. But this doesnt sound formal enough and may likely be a flawed and naive conclusion. Any input?