# Tag Info

13

Well the error message is quite clear. GridSearchCV accepts only lists. Therefore 'random_state': [7]} will solve the issue. However when you have only one value with this parameter, it makes more sense put it directly into the classifier as you did with n_estimators.

10

Yep I figured it out. The answer is that by default GridSearchCV's last act is to expose the API of the estimator object you passed so that you can directly call things like .predict() or .score() on the GridSearchCV object itself. It does this by retraining the estimator against the best parameters it found during cross validation. If you want to skip this ...

6

I would say that you have to remove random_state from the parameter grid. That, or put something like [7, X] which will work but that doesn't make sense I think. If you want to use fixed random_state = 7, you should write it when you instantiate the estimator just as another hyperparameter (next to n_estimators). I can't test it right now but I'd say that's ...

4

We can save the trained model or any other file via Google Colaboratory. How I'm using it? I have mapped my Google Drive with Google Colaboratory notebook and saved trained model as a pickle file in it. You can create a file and save your data in it. How to integrate Google Drive with Google Colaboratory notebook? #Add and execute below mentioned ...

4

The score is based on the scorer defined in the scoring argument. Meaning, the scorer can be any of the default metrics, such as precision, accuracy or F1-score (e.g., this); or a custom scorer. For a scorer (by convention), higher value is better. The value is not necessarily a percentage, but is often normalized between 0 and 1.

3

Yes, you can replace the cv=5 with cv=KFold(n_splits=5, random_state=None, shuffle=False). Leaving it set to an integer, like 5, is the equivalent of setting it to either KFold(n_splits=5) or StratifiedKFold(n_splits=5), depending on the model you pass to the estimator parameter of GridSearchCV()

3

When you use nested estimators with grid search you can scope the parameters with __ as a separator. In this case the LogisticRegression model is stored as an attribute named estimator inside the OneVsRestClassifier model: from sklearn.model_selection import GridSearchCV from sklearn.linear_model import LogisticRegression from sklearn.multiclass import ...

3

For multi-metric evaluation, the scores for all the scorers are available in the cv_results_ dict at the keys ending with that scorer's name ('_scorer_name'). so use grid.cv_results_['mean_test_(scorer_name)'] Ex: grid.cv_results_['mean_test_r2']

3

My Suggestion: The intrinsic separation of classes needs more complex model to be captured. I say this, because the difference between default model and your grid search is in max_depth parameter which is one of complexity indicators in Decision Trees. The default is None so it uses the maximum complexity it can get from max_depth but your parameter values ...

3

Just to add to others here. I guess you simply need to include a early stopping callback in your fit(). Something like: from keras.callbacks import EarlyStopping # Define early stopping early_stopping = EarlyStopping(monitor='val_loss', patience=epochs_to_wait_for_improve) # Add ES into fit history = model.fit(..., callbacks=[early_stopping])

3

First my understanding of your problem. You want to find the best hyperparameters for a Random Forest. For that, you want to first adjust n_estimators parameter and then the rest of parameters in different runs. Before answering to your question, you will only want to do a thorough search of hyperparameters when you want to have an improvement of around 1%...

3

I think that you just need: feature_importances = rf_gridsearch.best_estimator_.feature_importances_ This provides the feature importance for all the attributes in your dataset. For more information on this as well as other options, you may also refer to the Scikit-learn official documentation.

3

By default, GridSearchCV uses the score method of its estimator; see the last paragraph of the scoring parameter on the docs: If None, the estimator’s score method is used. And DecisionTreeRegressor.score (indeed, all/most regressors) uses R^2. In response to your edit: you can specify scoring='neg_mean_squared_error'. But note too that there's a linear ...

3

By default, GridSearchCV provides a score of nan when fitting the model fails. You can change that behavior and raise an error by setting the parameter error_score="raise", or you can try fitting a single model to get the error. You can then use the traceback to help figure out where the problem is. For the LogisticRegression, I can identify the ...

3

The correct way of calling the parameters inside Pipeline is using double underscore like named_step__parameter_name .So the first thing I noticed is in this line: parameters = {'vect__ngram_range': [(1, 1), (1, 2)],'tfidf__use_idf': (True, False),'clf__alpha': (1e-2, 1e-3) } You are calling vect__ngram_range but this should be tfidf__ngram_range Now this ...

2

When doing GridSearchCv, the best model is already scored. You can access it with the attribute best_score_ and get the model with best_estimator_. You do not need to re-score it in a cross validation. Also, yes, the pipeline is entirely fitted when doing each split during the cv.

2

It would be helpful to get the ouput of the program (or at least the error thrown) However, MLPRegressor hidden_layer_sizes is a tuple, please change it to: param_list = {"hidden_layer_sizes": [(1,),(50,)], "activation": ["identity", "logistic", "tanh", "relu"], "solver": ["lbfgs", "sgd", "adam"], "alpha": [0.00005,0.0005]} https://scikit-learn.org/stable/...

2

Negative R2 values can be observed when using it in the context of model validation (where we have data that is withheld from the model) because in this context, SST $\ne$ SSE + SSR. That is, this constraint does not exist due to the data splitting. This is because in the context of model validation, the value of SST is solely calculated using the ...

2

R2 can be negative if the model is arbitrarily worse according to the sklearn documentation So the very negative train scores were indicative of an extremely bad performance. Why was the test performance so colossally bad in GridSearchCV when it was decent in a simple test/train split? The main problem is that train_test_split chooses observations randomly ...

2

First suggestion: you should first find a CNN architecture that satisfies you, and then stick with it. Second suggestion: be careful with cross validation. CNNs are extremely "heavy" models, they can easily take hundreds of thousands or millions of trainable parameters. Running cross validation on those could be computationally impossible. In other words, ...

2

If you would ask for code suggestion please specify your framework in the future. I am assuming you are using Keras I can make you a minimum viable implementation of your case. from sklearn.base import ClassifierMixin, BaseEstimator class CNN_model(ClassifierMixin, BaseEstimator) : def __init__(**model_params) : """ define ...

2

Upgrade your xgboost version. reg:squarederror was added in 0.83 release (In version 0.82 or below, use reg:linear) In general, XGBoost support information is at https://discuss.xgboost.ai/

2

You're mixing up GridSearchCV and cross_val_score; you should only need to run one of them. GridSearchCV will search through your hyperparameter space, for each combination using cross-validation and producing a score. You can access these scores through the attribute cv_results_. cross_val_score has no hyperparameter search; it just scores using cross-...

2

To plot feature importance using gridsearch use: x= X_train_v1.columns,y= rf_grid_search_v1.best_estimator_.feature_importances_

2

First, you are fitting $5 \cdot 3\cdot2\cdot2\cdot2\cdot5=600$ models and n_estimator=500 is quite big. Of course, this depends on your dataset and in your computing power. My first guess will be that you have not enough RAM memory on your laptop(if you are running it there) and that is why it is collapsing. If the error is this one, I recommend sampling ...

2

Edit: oh, now I think I see why @CarlosMougan said no. You said ...start the same GridsearchCV with the same parameter and just change... If you mean use the optimal values for all hyperparameters except n_estimators and now search only on that one hyperparameter, then Carlos is right, and for the right reason. Below, I interpreted your suggestion as ...

2

By other posts and this one seems what you don't have a clear intuition of the n_estimators of the random forest. I am going to assume that you are referring to the n_estimators (from this other question). n_estimators is the number of trees that your 'forest' has. Not the depth of your tree. That is another parameter. If you are referring to max_depth = ...

2

My 2 cents: I'm fan of defying the max_leaf_nodes (in this example 5) and then visualizing it. I suggest starting at 3 and then increasing it slightly (the same applies for your Random Forest). In general, at around 5 I see overfitting. With your large dataset, you might need a bit more (i.e. max_leaf_nodes = 10?). Why? Or the answer to your question... ...

2

This idea of using a small sample to the data set to search for the hyperparameters is called multi-fidelity methods. A good starting point is the book by Frank Hutter, Lars Kotthoff, Joaquin Vanschoren Automated machine learning: methods, systems, challenges which is open access.

2

Some of your hyperparameter values aren't allowed (colsample_bytree and subsample cannot be more than 1), so probably xgboost errors out and sklearn helpfully moves on to the next point, recording the score as NaN. Half of your values for colsample_bytree are disallowed, which supports seeing half of your scores as NaN; and that will happen regardless of the ...

Only top voted, non community-wiki answers of a minimum length are eligible