New answers tagged cross-validation
0
After finding THE best subset doesn't make sense. Your approach is not correct in this way.
If you cannot handle all data you may have to follow one of approaches below:
1- Perform clustering and chose 10% (the more the better, based on what your computer can handle) from each cluster so all 10 subsets may have similar distribution. As a result, the models ...
0
Is the code correct?
Yes, it looks correct to me
How to calculate Errors:
Calculate accuracy for train data
Calculate accuracy for test data
If train error is high (RMSE in your case) - high bias, retrain model with more trees, less learning rate, more data (if possible)
If train error low, but test error high - high variance or overfit: regularization, ...
2
In a short answer, It seems that you used the same hyperparameters for both cross-validation's last training and the simple training process (without cross validation).
In the long answer
In K-fold cross-validation, you tune the hyperparameters according to the mean of K performance in order to obtain the maximum generalizing ability of the model.
Then you ...
0
Answer
In the param_grid dictionary, pass every hyper-parameter as array e.g., in your code above, you missed-out to place 'objective': 'regression' part of param_grid as an array. Even though, you have are using single value place it as array i.e. 'objective': ['regression']
Therefore, you can update below piece of code in your script and re-run. It will ...
1
@Baruch Youssin I am not sure I understand why the variability introduced by the k-fold target mean encoding calculation decreases overfitting.
I understand that by applying k-fold target mean encoding instead of mean encoding you avoid data leakage (i.e. using information about a specific row of Y to calculate a feature X).
But then assuming no data leakage ...
0
Increasing tolerance will result in "higher root mean squared error value" most of the time. Increasing tolerance is telling the model it is okay to stop earlier with higher error and not continue the search for a possible more optimal solution using smaller updates.
1
Kfold will not help you to understand whether your model is getting overfitted or not.
Currently the reason was that you have used max_depth to be as 17 and hence it just highly overfitted on training_set and provided bad performance on the test set.
What You can try is with current model lower that max_depth value and check it changes on the test set.
If ...
1
Overfitting is visible when the performance on the training set is much higher than the test set, so you clearly had overfitting with your first experiment already.
Cross-validation is not a way to see overfitting by itself, it's a way to obtain a more reliable performance value by using several splits (removes chance factor) and using an overall larger test ...
1
That is not a regression problem. Thus those results can not be interpreted.
It should be framed as probabilistic binary classification. The target is binary because there are two outcomes - disease or not. Probabilistic because it on a scale from 0 (not possible) to 1 (completely certain).
1
What I would suggest is to have put the results as a Data frame with
pd.DataFrame(cv.cv_results_)
And then you have the data in a dataframe which is easier to handle.
For the other question (in your comments), once you have the data in a nice dataframe is just about data visualization. How can you put it in a nice visualization?
The easier are either 2d ...
0
There are multiple popular ways to ensemble models. Averaging, majority voting, selecting the one with the highest probability, learn a new model based on these 5 numbers are amongst the many methods available. Check also the Bayes optimal classifier which 'averages' these probabilities in a Bayes way:
https://en.wikipedia.org/wiki/Ensemble_learning#...
1
A standard way to provide the performance of each model would be:
providing, for each split, the value of the chosen metric (accuracy, roc_auc, etc) on the train and test sets (on your case, your one-out sample), something like this (in this case with 2 models):
as a final model performance (for each one of the 5 models), a mean metric value together with ...
Top 50 recent answers are included
