Questions tagged [hyperparameter-tuning]

Hyperparameter tuning (also called hyperparameter optimization) refers to the process of finding the optimal set of hyperparameters for a given machine learning algorithm.

Filter by
Sorted by
Tagged with
0
votes
0answers
13 views

Is it possible to fix the validation set when tuning hyperparameters using scikit learn?

I have a question regarding hyperparameter optimization in scikit learn. I am most familiar with tensorflow where you first split your data into three sets: Train, validation and test. Hyperparameters ...
0
votes
0answers
15 views

ValueError: Failed to convert a NumPy array to a Tensor (Unsupported object type int) [migrated]

I am trying to tune the hyperparameters of MLP sequential model but getting an error while performing this task. I have tried degrading/upgrading the scikit-learn version and using ...
0
votes
1answer
31 views

High Recall but too low Precision result in imbalanced data

I was training a model using XGBoost Classifier on heavy imbalanced data base with 232:1 of binary class. Because my training data contains 750k rows and 320 features (after doing many feature ...
1
vote
1answer
17 views

Using Sklearn's predefined split

I am working on a binary classification task using SVM. The dataset is quite large so I don't want to use k-fold CV for parameter tuning, but instead a simple train-validation-test split. I have done ...
0
votes
0answers
16 views

CNN Model Seems To Just Be Guessing

I am working with a binary classification problem, and regardless of what changes I make, the model seems to just be guessing between 0 (Negative) and 1 (Positive). The dataset is imbalanced at a ...
0
votes
0answers
21 views

Method of hyperparameter tuning for regression tree ensembles in Matlab

What regression tree ensemble methods and what parameters does Matlab actually consider in hyperparameter tuning? The documentation appears to be sparse, to say the least. See https://se.mathworks.com/...
0
votes
1answer
17 views

validation after hyperparameter tuning

I tuned my hyperparameter with random search and i used cv=5. Is it important to validated the hyperparameter with model and testdata or is it okay to use the given ...
0
votes
0answers
12 views

Results Random Forest Regression Optimization [duplicate]

I'm going to optimize the hyperparameters of a Random Forest Regressor using scikit-learn and GridSearchCV, with the following code: ...
0
votes
1answer
28 views

Hypertune xgboost to dealing with imbalanced dataset

My training data has extremely class imbalanced {0:872525,1:3335} with 100 features. I use xgboost to build classification model with bayessian optimisation to hypertune the model in range ...
0
votes
1answer
80 views

Classification Threshold Tuning with GridSearchCV

In Scikit-learn, GridSearchCV can be used to validate a model against a grid of parameters. A short example for grid-search cv against some of DecisionTreeClassifier parameters is given as follows: <...
1
vote
1answer
30 views

What is the best practice for tuning hyperparameters using validation data?

I'm building a binary classifier, using task-transfer from resnet and a total training set of 300 images. Initially I put aside 100 images as validation, and tuned the hyperparameters, each time ...
0
votes
1answer
27 views

GridSearch on imbalanced datasets

Im trying to use gridsearch to find the best parameter for my model. Knowing that I have to implement nearmiss undersampling method while doing cross validation, should I fit my gridsearch on my ...
1
vote
0answers
11 views

Can I dynamically change the hyper-parameters of a model?

Question Can I apply different hyper-parameters for different training sets? I can see the point of using the shared parameters but I cannot see the point of using shared hyper-parameters. The ...
1
vote
1answer
26 views

Getting very low/ wrong accuracy from RandomizedSearchCV

I am currently using RandomizedSearchCV to optimize my hyper-parameters. However the reported scores of each iteration is very low. When I then evaluate the highest scoring candidate I get very high ...
0
votes
1answer
28 views

Hyperparameter tuning with Bayesian-Optimization

I'm using LightGBM for the regression problem and here is my code. ...
0
votes
0answers
21 views

Understanding min_samples_split and min_samples_leaf hyperparameters with DecisionTreeClassifier algorithm

My dataset consists of 775 samples and 117 features. The features represent developer skills (C++, Hadoop, AWS, etc.) and the output variable is a developer's profile (Frontend Developer, Quality ...
1
vote
0answers
43 views

XGBoost: Typical gamma and min_child_weight range

What is the typical accepted range of gamma and min_child_weight parameters for the XGBoost algorithm? Is the range of min_child_weight correlated with the number of feature or samples in the training ...
1
vote
1answer
31 views

Is there a rule of thumb for a sufficient number of trials for hyperparameter search

I am implementing a quite complicated Bayesian hyperparameter search in hyperopt library on a CNN. Is there a rule of thumb for a "sufficient" number of ...
1
vote
1answer
26 views

GridSearchCV with custom tune grid

What is the best way to perform custom parameter search CV with the Scikit-learn API? I really like GridSearchCV. However for my case the param_grid parameter is ...
3
votes
3answers
76 views

Optimal selection of k in K-NN

I am currently reviewing some concepts related to Machine Learning, and I started to wonder about the hyperparameter selection of K-NN classifier. Suppose you need to solve a classification task with ...
0
votes
0answers
9 views

Are there good hyperparameter optimization networks for Generative Adversarial Nets?

Finding good hyperparameters, as the learning rate for gradient descent, is crucial for good performance in deep learning. There exist various automatic methods for tasks as segmentation or ...
0
votes
1answer
145 views

Understand RandomizedSearchCV output

I am trying to do hyperparameter tunning for the Randomforest regression model. I'm using RandomizedSearchCV (scikit-learn) and I defined verbose=10. For that reason, I'm getting messaged while it's ...
1
vote
2answers
34 views

Order of hyperparamater tunning

I'm trying to do hyperparameter tunning for random forest regression model. My question is- is there any order I should do it? like starting with specific parameter and then move on on the other? ...
1
vote
1answer
49 views

Hyperparameter tunning for Random Forest- choose the best max depth

I'm trying to choose the best parameters for random forest model. For that goal I hae run my model in loop with only one parameter and each time I have changed the number for the parameter max depth. ...
2
votes
1answer
32 views

Is data subsampling appropriate for hyperparameter optimisation?

Fundamentally, under what circumstance is it reasonable to do HPO only on a subsample of the training set? I am using Population Based Training to optimise hparameters for a sequence model. My dataset ...
0
votes
0answers
10 views

Is the Tree Parzen Estimators topological structure supposed to be updated during HPO?

IN the original hyperopt paper, it is not clear whether the tree parzen search space is supposed to be updating its topological order with every update.
1
vote
0answers
20 views

How can I use a validation set to tune the hyperparameters of an XGBClassifier?

I'm currently building a ranking model using an XGBClassifier. I have training, testing, and validation sets. I want to use the validation set to tune the hyperparameters of the XGBClassifier before ...
0
votes
1answer
42 views

Does GridSearchCV not save the best parameters?

So I tuned the hyperparameters using GridSearchCV, fitted the model to the data, and then used best_params_. I'm just curious ...
1
vote
1answer
47 views

small dataset CV

I have a very small dataset ( 150 records) with 20 features, trying to predict a binary outcome. Due to the small size, i chose to do 10 CV instead of train/test as the train/test split. I was ...
1
vote
1answer
38 views

Metric to use to choose between different models - Hyperparameters tuning [closed]

I'm building a Feedforward Neural Network with Pytorch and doing hyperparameters tuning using Ray Tune. I have train, validation and test set, train and validation used during the training procedure. ...
0
votes
1answer
18 views

With automated hyperparameter tuning available, do we still need to learn hyperparameter tuning [closed]

Tools like AWS Sagemaker have capability to do automated hyperparameter tuning, even with complex algos like Neural Networks using Tensorflow. So do we still need to learn how to do hyperparameter ...
2
votes
1answer
173 views

how to prevent machine crash while searching for hyper parameters of XGBoost with GridSearchCV

I am searching for best hyper parameters of XGBRegressor using GridSearchCV. Here is the code: ...
1
vote
2answers
78 views

Hyperparameter tuning XGBoost

I'm trying to tune hyperparameters with bayesian optimization. It is a regression problem with the objective function: objective = 'reg:squaredlogerror' $\frac{1}{2}[log(pred+1)-log(true+1)]^2$ My ...
1
vote
1answer
19 views

How to quantify ‘compute cost’ of training of xgboost model?

I want to quantify compute cost of hyper-parameter search for xgboost model. One way can be to measure training time with one particular hyper parameter configuration chosen for training and use it as ...
4
votes
1answer
42 views

Exploring variables to guide xgboost tuning

In short: How to think about the type and distribution of my variables when choosing parameter values for xgboost? Context: I have a dataset which I want to classify using the ...
0
votes
1answer
138 views

Hyperparameter Tuning in Random Forest Model

I'm new to the machine learning field, and I'm learning ML models by practice, and I'm facing an issue while using the machine learning model. While I'm implementing the ...
0
votes
0answers
56 views

Hyper tuning reduce the accuracy score, why?

I have performed hyper tuning grid CV search on KNN model. The actual accuracy score for my KNN was accuracy of 42.31 % without performing hyper tuning. However, after performing hyper tuning, the ...
0
votes
1answer
39 views

Regularization hyperparam tuning during training

I have an idea for a regularization-hyperparam selection method, which I haven't encountered before and can't find on Google, but I'm sure someone has already tried it and I'm wondering what are the ...
2
votes
0answers
34 views

Initial value space for Random Forest hyperparameter tuning

I'm building a Random Forest Classifier using Scikit Learn. My problem consists in a 4 class classification task, the values are distributed as follows (after splitting my data in training set and ...
2
votes
2answers
47 views

How to treat data transformation choices as hyperparemeters?

While reading the book hands-on ML by Aurelien Geron, I came across this line- Treat your data transformation choices as hyperparameters, especially when you are not sure about them (e.g., if you’re ...
0
votes
0answers
233 views

SVM is taking too long for hyperparameter tuning

I am running SVM,Logistic Rregression and Random Forest on the credit card dataset. My training dataset has the shape (454491, 30). I performed 5-fold cross validation(which took more than an hour) ...
1
vote
1answer
49 views

How do you identify whether your RMSE score is good or not?

Im building a XGBoost regression model to predict the values in the range of -3 to 3. Im using Root Mean Squared Error to evaluate the model. With hyper-parameter tuning and everything the best scores ...
-1
votes
1answer
43 views

How to identify Overfitting in RandomForestClassifier?

Im building a sentiment classification model using RandomForestClassifier. I got the training accuracy of 99.65 & cross-validation( RepeatedStratifiedKFold-5 folds) accuracy of 97.29. I used f1 ...
1
vote
2answers
178 views

Is a test set necessary after cross validation on training set?

I'd like to cite a paragraph from the book Hands On Machine Learning with Scikit Learn and TensorFlow by Aurelien Geron regarding evaluating on a final test set after hyperparameter tuning on the ...
2
votes
1answer
73 views

Which is better: Cross validation or a validation set for hyperparameter optimization?

For hyperparameter optimization I see two approaches: Splitting the dataset into train, validation and test, and optimize the hyperparameters based on the results of training on the train dataset and ...
0
votes
0answers
33 views

How do you calculate the probability that a certain number of hyper-parameter combinations contains the optimum combination?

How do you calculate the probability that a certain number of hyper-parameter combinations contains the optimum combination? Side Note: After doing more research, I have decided that Randomized Search ...
2
votes
2answers
66 views

Faster grid search with small dataset to derive best params instead of full dataset?

I have a dataset of 300 000 rows and an ensemble model, which include grid search to find the best params of every algorithm. Unfortunately the grid search needs to long and I have problems to ...
1
vote
0answers
65 views

Adaptive Resampling in Caret with Pre-specified Validation Set

I was wondering if this is the correct way to get adaptive sampling in caret working with a pre-specified validation set using index. I can get this to work using the 'cv' method in caret like so <...
1
vote
0answers
68 views

How can I use the Brier Skill Score in cross-validation for imbalanced data?

I wish to use search for model hyper-parameters (by a grid search) using the Brier score as the scoring method (see code below): ...
0
votes
0answers
83 views

Good mathematically explained algorithm for Hyperparameter Optimization (Bayesian) for implementing in Java

I have implemented Random Forest, Bagging, Gradient Boosting, etc... in java myself. It took a long time to complete these machine learning algorithm codes. But at last all of them are well running. ...