I did submit my first kaggle kernel, on the avocado dataset kernel link, I treated it like I should predict the avocado price so I splitted the dataset in a train & test set, fitted the model and run an r2 score (accuracy) on the predicted train & test set:
X_train, X_test, y_train, y_test = train_test_split( df2, y, test_size=0.33, random_state=np.random.randint(0,100)) gradModel = GradientBoostingRegressor(n_estimators=100, learning_rate=0.1) gradModel.fit(X_train,y_train) y_pred=gradModel.predict(X_test) (r2_score(y_train, gradModel.predict(X_train)),r2_score(y_test, y_pred))
which gives (0.9325975114210778, 0.8791805732349958) Repeatedly over multiple runs, I believe this is a reasonable score?
When I run a cross validation on the same model (same parameters) I do get a much lower score.
gradModel = GradientBoostingRegressor(n_estimators=100, learning_rate=0.1) from sklearn.model_selection import cross_val_score scores = cross_val_score(gradModel, df2, y, cv=15) print ("Accuracy: %0.2f (+/- %0.2f)" % (scores.mean(), scores.std() * 2)) Accuracy: 0.45 +/- 0.21
Which I believe is a more realistic score, but I can't explain the large differences. Also because the original r2 score is more or less equal over multiple reruns