Tag Info

Accepted

How to estimate GridSearchCV computing time?

You could fit your model/pipeline (with default parameters) to your data once and see how long it takes to train. Then you would multiply that by how many times you want to train the model through ...
• 7,758

What is the most efficient method for hyperparameter optimization in scikit-learn?

Optimization isn't my field, but as far as I know, efficient and effective hyper-parameter optimization these days heavily revolves around building a surrogate model. As models increase in complexity, ...
• 371
Accepted

How to plot mean_test score and mean_train score of GridSearchCV

You could visualize them as a heatmap. For example you could use the C values as the rows, the gamma values as the columns and ...
• 311

GridSearch without CV

GridSearchCV is built around cross validation, but if speed is your main concern, you may be able to get better performance using a smaller number of folds. From the docs: class sklearn....
• 578

Log loss vs accuracy for deciding between different learning rates?

According to me, it is not correct to co-relate loss with accuracy. Loss is used to optimize the hypothesis such that we can get best weights whereas accuracy is used to identify how well model ...
• 1,232
Accepted

how to pass parameters over sklearn pipeline's stages?

I figured out how to do that by monkey patching ParameterGrid.__iter__ and GridSearchCV._run_search methods. ParameterGrid.__iter__ iterates over all possible combinations of hyerparameters (dict of ...
Accepted

Can GridSearchCV be used for unsupervised learning?

The goal of GridSearchCV is to iterate over (hence search) all possible combinations (hence grid) of hyper parameters and evaluate a model on a cross-validation (...

When to use BayesianSearchCV and how it works?

As mentioned in that Kaggle notebook, you can use it pretty much as just a drop-in replacement for other search methods (grid or random). Bayesian searches still are random searches over a predefined ...
• 10.8k

What's the default Scorer in Sci-kit learn's GridSearchCV?

From the User Guide: By default, parameter search uses the score function of the estimator to evaluate a parameter setting. These are the ...
• 10.8k

What is the most efficient method for hyperparameter optimization in scikit-learn?

You can take a look at auto-sklearn. That's an automated machine learning toolkit which is a direct extension of scikit-learn.
• 526

Large negative R2 or accuracy scores for random forest with GridSearchCV but not train_test_split

Negative R2 values can be observed when using it in the context of model validation (where we have data that is withheld from the model) because in this context, SST $\ne$ SSE + SSR. That is, this ...
• 1,874

Large negative R2 or accuracy scores for random forest with GridSearchCV but not train_test_split

R2 can be negative if the model is arbitrarily worse according to the sklearn documentation So the very negative train scores were indicative of an extremely bad performance. Why was the test ...
• 151

Default parameters for decision trees give better results than parameters optimised using GridsearchCV

My Suggestion: The intrinsic separation of classes needs more complex model to be captured. I say this, because the difference between default model and your grid search is in max_depth parameter ...
• 6,452
Accepted

How to get mean test scores from GridSearchCV with multiple scorers - scikit-learn

For multi-metric evaluation, the scores for all the scorers are available in the cv_results_ dict at the keys ending with that scorer's name ('_scorer_name'). so use grid.cv_results_['mean_test_(...
• 526
Accepted

Comparison of machine learning approaches for a topic in a scientific paper

If I understand, you could evaluate your approach based on your efforts in using the right preprocessing, picking the right features, building the appropriate architecture (choosing the model that ...
• 886
Accepted

GridSearch mean_test_score vs mean_train_score

When you do k-fold cross-validation, you train k models, each one of them leaving the proportion $1/k$ of the data out. For each of the models, you can compute its train error and validation error. ...
• 5,921

ScikitLearn - RandomForestRegressor score different in and out of grid search

I have found the answer to my question : there is randomness in RandomForestRegressor(). Using the random_state hyper parameter fix the problem : ...
Accepted

sklearn.model_selection: GridSearchCV vs. KFold

Yes, you can replace the cv=5 with cv=KFold(n_splits=5, random_state=None, shuffle=False). Leaving it set to an integer, like 5, ...
• 1,714

GridSearch without CV

By passing a callable for parameter scoring, that uses the model's oob score directly and completely ignores the passed data, you should be able to make the ...
• 10.8k
Accepted

plotting a decision tree based on gridsearchcv

While I don't have a module named graphviz I can still try to help. Reading the documentation for GridSearchCV, I can see that there's a attribute called ...
• 813

Why is cross-validation score so low?

I know this question has been here for two years, however, I was having the same problem when using cross_val_score on my data and I ended up here. The results ...
• 21
Accepted

Why is cross-validation score so low?

In your random forest, this is due to the fact that your final model is overfitting. Sklearn's GridSearchCV has a default argument ...
• 5,921

How to estimate GridSearchCV computing time?

Let the search complete and then you can use cv_results_ attribute to compute the elapsed time as given below. ...

Class weight ineffective in sklearn

This is most likely because sklearn has chosen your majority class for you and you simply need to weigh your classes the other way around with "0" being the majority class.
• 21

Splitting hold-out sample and training sample only once?

In general, it's a good idea to split up your data into three sets: Training Set (60-80% of your data) Cross-Validation Set (10-20% of your data) Test Set (10-20% of your data) When you select a ...
• 573

Log loss vs accuracy for deciding between different learning rates?

Actually, loss is used by the model to decide the probability of the class. So, logloss just indicates how much is your model certain in comparison to the correct labels of the classes in test samples....
• 497
Accepted

How to find optimal number of trees in random forest using Grid search in R?

mtry is the parameter indicating how many of the features are checked in each split decision. http://topepo.github.io/caret/train-models-by-tag.html#random-forest ,...
• 10.8k
Accepted

Am I using GridSearch correctly or do I need to use all data for cross validation?

Your procedure is, from what I can tell, correct. You are correctly splitting your data into train/test, and then using your training data only to find optimal hyper-parameters. Using all of the ...
• 1,874
Accepted

CNNs - Hyperparameter tuning with different training sizes of the same data set

First suggestion: you should first find a CNN architecture that satisfies you, and then stick with it. Second suggestion: be careful with cross validation. CNNs are extremely "heavy" models, they can ...
• 5,897