Questions tagged [grid-search]

In machine learning, grid search refers to multiple runs to find the optimal value of parameter(s)/hyperparameter(s) of a model, e.g. mtry for random-forest or alpha, beta, lambda for glm, or C, kernel and gamma for SVM.

0
votes
1answer
10 views

CNNs - Hyperparameter tuning with different training sizes of the same data set

I would like to compare how much the classification performance (test accuracy) of CNNs changes depending on the size of the data set. For this I would like to use a data set like MNIST or Fashion ...
1
vote
0answers
22 views

How to tune the parameters of ANN in R?

I tried below code where I used method as 'mxnet': classifier = train(form = Survived ~ ., data = training_set_scaled, method = 'mxnet') From this code I got ...
0
votes
1answer
33 views

How to find optimal number of trees in random forest using Grid search in R?

From below code, I am getting optimal number of mtry. What is this mtry ? and How should I find the optimal number of tree that to be assigned to Random forest algorithm so that it will give High ...
1
vote
2answers
43 views

Large negative R2 or accuracy scores for random forest with GridSearchCV but not train_test_split

I'm trying to use GridSearchCV from scikit-learn and look at the difference between train/test metrics. When I do a normal test/train split with RandomForestRegressor, the metrics are comparable. ...
0
votes
1answer
20 views

Am I using GridSearch correctly or do I need to use all data for cross validation?

I'm working with a dataset that has 400 observations, 34 features and quite a few outliers, some of them extreme. Given the nature of my data, these need to be in the model. I started by doing a 75-...
0
votes
0answers
15 views

Reasonable hyperparameters for NuSVR?

I'm looking for reasonable hyperparameters grid for NuSVR. For now I have: ...
1
vote
1answer
35 views

When to use BayesianSearchCV and how it works?

Can somebody highlight when to use BayesianSearchCV and how it works? I have seen the implementation of same on kaggle and wanted to explore it further. Below is the link where the implementation ...
1
vote
1answer
49 views
0
votes
1answer
35 views

Is the role of the validation set in a deep learning network is only for Early Stopping?

In the "deep learning crash course" given by Leo Isikdogan in lecture 4 https://www.youtube.com/watch?v=ms-Ooh9mjiE&list=PLWKotBjTDoLj3rXBL-nEIPRN9V3a9Cx07&index=4 Overfitting, Underfitting, ...
7
votes
2answers
162 views

What is the most efficient method for hyperparameter optimization in scikit-learn?

An overview of the hyperparameter optimization process in scikit-learn is here. Exhaustive grid search will find the optimal set of hyperparameters for a model. The downside is that exhaustive grid ...
1
vote
2answers
84 views

Grid search model isn't recognized as fitted for Graphviz

I find this really weird, and the code is really straight forward. What am I doing wrong ? ...
0
votes
1answer
20 views

How do you search a high dimensional for the global maxima using as few samples as possible?

Suppose the value at any point in the space is defined by Y = f(x1, x2 .. xk). For simplicity, we can assume that x takes only binary values. Which means that we have a total of 2^k possible values. I ...
0
votes
0answers
15 views

Long run time for grid search SARIMA

I am running a grid search for identifying the right set of params for Seasonal ARIMA, for over a 1300 training set and range for all the params being 0,1 and 2. But this process is taking over ...
0
votes
0answers
13 views

Does sklearn's gridsearchCV use the same cross validation train/test splits for evaluating each hyperparameter combination?

I couldn't find the answer on any forum in the interwebs so I hunted down the answer myself. Yes; the same train/test splits are used for each parameter combination. Here Is the relevant sklearn ...
1
vote
1answer
682 views

How to get mean test scores from GridSearchCV with multiple scorers - scikit-learn

I'm trying to get mean test scores from scikit-learn's GridSearchCV with multiple scorers. grid.cv_results_ displays lots of ...
1
vote
1answer
61 views

Voting classifier using grid search for Time Series

I have three models Arima Auto ARIMA Double Exponential Smoothing I want to apply ensemble method - voting method and allow the classifier to learn weights for these three models. I have checked ...
4
votes
4answers
205 views

Default parameters for decision trees give better results than parameters optimised using GridsearchCV

I am using Gridsearch for a DecisionTreeClassifier predicting a binary outcome. When I run fit and predict with default parameters, I get the following results: ...
4
votes
4answers
145 views

Log loss vs accuracy for deciding between different learning rates?

While model tuning using cross validation and grid search I was plotting the graph of different learning rate against log loss and accuracy separately. Log loss When I used log loss as score in ...
5
votes
1answer
430 views

How to plot mean_test score and mean_train score of GridSearchCV

How to plot mean_train_score and mean_test_score values in GridSearchCV for ...
1
vote
1answer
94 views

Optimizing decision threshold on model with oversampled/imbalanced data

I'm working on developing a model with a highly imbalanced dataset (0.7% Minority class). To remedy the imbalance, I was going to oversample using algorithms from imbalanced-learn library. I had a ...
1
vote
2answers
135 views

Scikitlearn grid search random forest using oob as metric?

Have looked at data on oob but would like to use it as a metric in a grid search on a Random Forest classifier (multiclass) but doesn't seem to be a recognised scorer for the scoring parameter . I do ...
1
vote
0answers
134 views

How to perform platt scaling for hyperparameter-optimized model?

I'm using Python and have a best estimator from a grid search. Wanted to be able to calibrate the probability output accordingly, but would like to know more about implementing platt scaling. From ...
1
vote
0answers
286 views

Different values from GridSearch estimation

I am using GridSearchCV in order to find best estimator with best hiperparameters. I also want to check it once against a small portion of data used in cross ...
0
votes
1answer
1k views

GridSearch mean_test_score vs mean_train_score

I am working with scikit learn and GridSearch in order to find the best parameters in my classifiers. I have a map of different hyperparameters and I want to print out GridSearch results, but I do ...
4
votes
1answer
3k views

Why is cross-validation score so low?

I am using Scikit-Learn for this classification problem. The dataset has 3 features and 600 data points with labels. First I used Nearest Neighbor classifier. ...
6
votes
1answer
3k views

How to estimate GridSearchCV computing time?

If I know the time of a given validation with set values, can I estimate the time GridSearchCV will take for n values I want to cross-validate ?
0
votes
1answer
711 views

How to use GridSearchCV with RidgeClassifier

I'm trying to use GridSearchCV with RidgeClassifier, but I'm getting this error: My problem is regression type. IndexError: ...
2
votes
2answers
80 views

Comparison of machine learning approaches for a topic in a scientific paper

As part of my master's thesis, I have made a prediction of data with approaches of machine learning in a topic where are no papers yet. The topic is a regression problem for which several machine ...
2
votes
1answer
167 views

Parameter tuning for machine learning algorithms

When it comes to the topic of tuning parameters, most of the time you read grid search. But if you have 6 parameters, for which you want to test 10 variants, you get to 10^6 = 1000000 runs. Which in ...
1
vote
2answers
3k views

Class weight ineffective in sklearn

I'm dealing with an imbalanced dataset and as usual it's very easy to obtain a high accuracy, but the recall on the less frequent class is very low. I would like to improve on the false negative of ...
2
votes
1answer
1k views

Splitting hold-out sample and training sample only once?

I have a question related to evaluating out-of-sample predictions. For my research I want to tune two parameters related to Support Vector Machines, and use these optimized parameters to predict the ...
3
votes
1answer
533 views

Grid seach is unavailable for Keras in case of multiple outputs?

I do experiments with the following Keras architecture with multiple outputs: ...