I thought n_samples is the number of training examples. But when using GridSearchCV, n_samples becomes 32 rather than 50.

Error when using GridSearchCV:

Expected n_neighbors <= n_samples, but n_samples = 32, n_neighbors = 50

Training examples:

print(X_train.shape[0]) => 50

print(len(y_train)) => 50

This works:

neigh = KNeighborsClassifier(n_neighbors=50)
neigh.fit(X_train, y_train) 
result = neigh.predict(X_test)

This fails:

from sklearn.model_selection import GridSearchCV

grid_params = { 
    "n_neighbors" : [50]

g = GridSearchCV(KNeighborsClassifier(), grid_params)
g.fit(X_train, y_train)

I'm confused why n_samples becomes 32 when using GridSearchCV.


The CV stands for CrossValidation, meaning it will split up your training set in a number of folds (in this case 3), train on n-1 of those folds and test on the remaining one. This is why your training is now done on 32 instead of 50 samples. Crossvalidation is useful for estimating how well your model (including specific hyperparameters) does on unseen data.

  • $\begingroup$ Does that mean I don't need to split my data into training and test sets in advance? $\endgroup$ – tim_xyz Feb 7 '18 at 22:14
  • 2
    $\begingroup$ You don't need a validation set anymore no, a hold out or test set could still be useful for other reasons that go pretty deep, but for model selection this is a more stable approach than using one validation set. $\endgroup$ – Jan van der Vegt Feb 7 '18 at 22:18

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.