What is the difference between Hyper-parameter Tuning and k-NN algorithm? Is k-NN also a type of Hyper-parameter tuning?


1 Answer 1


In kNN algorithm, you only try to find a suitable value of parameter k. And some models may have many parameters that can be modified. Normal parameters are optimized by loss functions and Hyperparameter tuning allows you to set various parameters to get the best model. You set them before training. Its 2 methods: GRID METHOD AND RANDOM SAMPLING might work well.

Grid Method: Impose a grid on possible space of a hyperparameter and then go over each cell of grid one by one and evaluate your model against values from that cell. Grid method tends to vast resources in trying out parameter values which would not make sense at all.

Random Sampling Method: In random method, we have a high probability of finding a good set of params quickly. After doing random sampling for a while, we can zoom into the area indicative of the good set of params.

Random sampling allows efficient search in hyperparameter space. But sampling at random does not guarantee uniformity over the range of valid values. Therefore, it is important to pick an appropriate scale.

You can read this: Hyperparameter Tuning of Deep Learning Algorithm

  • $\begingroup$ My model isn't performing well. Though it performs well during training phase. Why? @anmol $\endgroup$
    – Kelly Star
    Apr 5, 2018 at 20:12
  • $\begingroup$ Do not overfit the model. You might add new variables to the model to achieve highly accurate values but at the same time, it causes OVERFITTING. Often called noise the mode does not play well with the unseen data. Try using Cross-Validation. $\endgroup$ Apr 5, 2018 at 20:17
  • $\begingroup$ Cross-validation is a model evaluation method that is better than residuals $\endgroup$ Apr 5, 2018 at 20:46

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.