Questions tagged [hyperparameter-tuning]

Hyperparameter tuning (also called hyperparameter optimization) refers to the process of finding the optimal set of hyperparameters for a given machine learning algorithm.

84 questions with no upvoted or accepted answers
Filter by
Sorted by
Tagged with
5
votes
3answers
734 views

Hyperparameter tuning in multiclass classification problem: which scoring metric?

I'm working with an imbalanced multi-class dataset. I try to tune the parameters of a DecisionTreeClassifier, ...
4
votes
0answers
151 views

Why do BERT classification do worse with longer sequence length?

I've been experimenting using transformer networks like BERT for some simple classification tasks. My tasks are binary assignment, the datasets are relatively balanced, and the corpus are abstracts ...
4
votes
2answers
205 views

Benefits of using Deep Learning-specific hyperparameter optimization tools vs. sklearn?

There are quite a few library for hyperparameter optimization that are specific to Keras or other Deep Learning libraries, like Hyperas or Talos. My question is, what's the main benefit of using ...
3
votes
2answers
252 views

High Recall but too low Precision result in imbalanced data

I was training a model using XGBoost Classifier on a heavy imbalanced database with 232:1 of binary class. Because my training data contains 750k rows and 320 features (after doing many feature ...
3
votes
0answers
119 views

How to measure the stability of hyperparameter selection in a model-building procedure?

For my project I run several model-building-procedures. I use the mean and standard deviation of the test scores in the outer folds as an estimator for the generalizability of the model-building ...
2
votes
0answers
19 views

Rule of thumb for number of leafs and trees in a random forest?

I wonder if there is a rough estimate or rule of thumb or the like to guess the optimal number of leafs and trees in a random forest? Or would you just use a hyperparameter search method?
2
votes
1answer
19 views

My own model trained on the full data is better than the best_estimator I get from GridSearchCV with refit=True?

I am using an XGBoost model to classify some data. I have cv splits (train, val) and a separate test set that I never use until the end. I have used GridSearchCV to determine the best parameters and ...
2
votes
1answer
41 views

Is data subsampling appropriate for hyperparameter optimisation?

Fundamentally, under what circumstance is it reasonable to do HPO only on a subsample of the training set? I am using Population Based Training to optimise hparameters for a sequence model. My dataset ...
2
votes
0answers
51 views

Initial value space for Random Forest hyperparameter tuning

I'm building a Random Forest Classifier using Scikit Learn. My problem consists in a 4 class classification task, the values are distributed as follows (after splitting my data in training set and ...
2
votes
0answers
21 views

Negative correlation between OOB statistics and test set statistics during tuning of a RandomForest

I am tuning the parameters of a binary random forest classifier using a random search with a priority queue for training. After training with a fixed number of estimators (3000), the strategy is to ...
2
votes
2answers
2k views

How can I tune LSTM hyperparameters?

If anyone is there to answer these, that'll be great. I'm in the midst of a Final Year Project on LSTM. Currently, I’m stuck and confused over LSTM codes. There are 4 hyperparameters that I can play ...
2
votes
0answers
31 views

Parameter optimization and selection in dynamic neural networks

I have used a Bayesian optimization to tune machine learning parameters. The optimized parameters are "Hidden layer size" and "...
2
votes
0answers
290 views

Error while trying to do hyperparameter tuning using hyperas

I am getting a syntax error while using hyperas and am not sure why. My code: ...
2
votes
1answer
307 views

Hyperas LSTM configuration assignment error

I have been working on my trivial keras lstm model trying to implement Hyperas with the following code that gives me an error I cannot resolve. I have just been experimenting around with Hyperas and ...
2
votes
0answers
52 views

Interpreting hyper-engine output for hyperparameter optimization of a neural network?

I am currently trying to optimize the learning rate of a neural network built in tensorflow. The network has 3 hidden layers, with 500, 250 and 100 neurons respectively. I have adapted code from an ...
1
vote
1answer
11 views

Tune learning rate while tuning other HP

When doing hyperparameters optimisation, like a Random Search, should you add a search space for the learning rate ? My intuition is that some HP might work better with a certain LR, and be sub-...
1
vote
1answer
15 views

Python Script on tuning tree in orange datamining

Is there any python script that can be used to check the best tuning in tree model on orange python datamining? Since theres an error when I applied the script from http://docs.biolab.si/orange/2/...
1
vote
1answer
21 views

Could I directly apply techniques for hyper-parameter tuning, and choose the best model?

I have noticed in some sources the author first trains the model (say a model from scikit-learn) with the default hyper-parameters, and the model naturally gives a ...
1
vote
0answers
12 views

RandomizedSearchCV doesn't stop running

I'm trying to optimize the hyperparameters of my model using RandomizedSearchCV. However, it doesn't stop running even if I define few iterations. Someone could help me? The code I'm using is ...
1
vote
0answers
59 views

Worse performance after Hyperparameter tuning

I first construct a base model (using default parameters) and obtain MAE. ...
1
vote
0answers
58 views

Hyper parameters (window size and vector dimensions) tuning in word2vec using Grey Wolf Optimization

Using Grey wolf Optimization, I want to calculate optimal values of two hyper parameters: context window size and embedding size (vector dimensions) for word2vec skipgram model used for word embedding....
1
vote
0answers
73 views

XGBoost regressor hyperparameter tuning with hyperopt leads to overfit

Using hyperopt to hyperparameter tuning on XGBoost regressor, I am receiving overfiting on the train set. There is any suggestion how to solve it ? I have used cross validation with ...
1
vote
0answers
109 views

Why does Adam optimizer work slower than Adagrad, Adadelta, and SGD for Neural Collaborative Filtering (NCF)?

I've been working on Neural Collaborative Filtering (NCF) recently to build a recommender system using Tensorflow Recommenders. Doing some hyperparameter tuning with different optimizers available in ...
1
vote
1answer
23 views

Can I apply different hyper-parameters for different sliding time windows?

Question Can I apply different hyper-parameters for different training sets? I can see the point of using the shared parameters but I cannot see the point of using shared hyper-parameters. The ...
1
vote
0answers
65 views

XGBoost: Typical gamma and min_child_weight range

What is the typical accepted range of gamma and min_child_weight parameters for the XGBoost algorithm? Is the range of min_child_weight correlated with the number of feature or samples in the training ...
1
vote
0answers
41 views

How can I use a validation set to tune the hyperparameters of an XGBClassifier?

I'm currently building a ranking model using an XGBClassifier. I have training, testing, and validation sets. I want to use the validation set to tune the hyperparameters of the XGBClassifier before ...
1
vote
0answers
82 views

Adaptive Resampling in Caret with Pre-specified Validation Set

I was wondering if this is the correct way to get adaptive sampling in caret working with a pre-specified validation set using index. I can get this to work using the 'cv' method in caret like so <...
1
vote
0answers
112 views

How can I use the Brier Skill Score in cross-validation for imbalanced data?

I wish to use search for model hyper-parameters (by a grid search) using the Brier score as the scoring method (see code below): ...
1
vote
1answer
378 views

How to tune learning rate with HParams Dashboard on Tensorflow?

In Tensorflow documentation, it is shown how to tune several hyperparameters but not the learning rate.I have searched how to tune learning rate using HParams dashboard but could not find much. The ...
1
vote
0answers
207 views

Hyperparameter tuning one-class svm

I have a problem where I am trying to apply a one-class svm to detect outliers. I am training on a dataset of true cases using a one-class radial svm and then predicting for both false and true cases. ...
1
vote
0answers
55 views

Hyperparameter tuning results yield no improvement over spot-check

There is a balanced binary classified dataset as seen below. Things I have tried using RandomForestClassifier as chosen model: TimeSeriesSplit with n_splits=3 and ...
1
vote
1answer
36 views

Two questions on hyper-parameter tuning

Question 1: In the example of logistic regression, I often see the regularization constant and penalty methods being tuned by a grid search. However, it seems like there are a lot more options for ...
1
vote
0answers
74 views

Since is not possible test all the possible combination, what is the correct procedure to follow on building Machine Learning?

Sorry, I'm a little confused and this is a general question. How can I be sure that the procedure that I am following is the correct one? Following the steps for building a machine learning model, we ...
1
vote
0answers
20 views

Techniques for hyperparameter search in non-stationary environments

I'm tuning a supervised machine learning model over time, also called incremental learning. I do not want to assume the environment is non-stationary. Grid search and random search do not appear to ...
1
vote
0answers
164 views

Validation curve/RandomizedSearchCV difference train and test score

Ive build a RF model for an imbalanced data set that after feature selection has an F1 score of 54.26%. I am now trying to do hyper parameter tuning using RandomizedSearchCV, after creating validation ...
1
vote
0answers
75 views

SVM C vs gamma hyperparameter tuning

While running SVC(), how we can hyperparameter tune C vs gamma combination? I could see changes in C and gamma are impacting the accuracy differently. Also, what I understand about C and gamma are: C ...
1
vote
0answers
229 views

Tuning parameters in Affinity Propagation

I am doing Affinity Propagation clustering and trying to do tuning, but it takes time. A lot of time actually. As I am beginner I do not know how to get clusters. I need cluster numbers from 1 to 20 ...
1
vote
1answer
3k views

hyperparameter tuning with validation set

For what I know, and correct me if I am wrong, the use of cross-validation for hyperparameter tuning is not advisable when I have a huge dataset. So, in this case it is better to split the data in ...
1
vote
0answers
25 views

Minimizing overfitting when doing hyperparameter Tuning

Generaly when using Sklearn's GridSearchCV (or RandomizedGridSearchCV), we get best model with best test score even if the model overfits a little bit. How can we compute generalization error ...
1
vote
0answers
371 views

Ensure class balanced batches while hyperparameter tuning keras models with grid search

Ensuring class balanced batches while training keras models is possible using fit_generator method. I used imblearn.keras.BalancedBatchGenerator for that and it works fine! But i wanted to do that ...
1
vote
0answers
22 views

Analyzing the search space of hyperparameter optimization

My goal is to train a CNN via transfer learning on a given dataset and to analyze and document the training process. I selected a few CNN architectures and hyperparameters to perform a random search. ...
1
vote
0answers
23 views

Cross-validation for Timeseries Counterfactual Analysis

We are looking to predict counterfactual states from time-series data. In our problem we are looking to determine the energy savings from a grid-installed device that is varied on and off for many ...
1
vote
0answers
123 views

Optimizing parameters for CNN autoencoder based on training and validation loss

I have designed an autoencoder with a encoder and decoder consiting of 2D convolutational layers (the input are 40'000 2D images). I train the autoencoder using adam optimizer. The autoencoders has ...
1
vote
0answers
323 views

How to put KerasClassifier, Hyperopt and Sklearn cross-validation together

I am performing a hyperparameter tuning optimization (hyperopt) tasks with sklearn on a Keras models. I am trying to optimize KerasClassifiers using the Sklearn cross-validation, Some code follows: <...
1
vote
0answers
175 views

How to tune the parameters of ANN in R?

I tried below code where I used method as 'mxnet': classifier = train(form = Survived ~ ., data = training_set_scaled, method = 'mxnet') From this code I got ...
1
vote
0answers
1k views

Bayesian optimization for a Light GBM Model

I am able to successfully improve the performance of my XGBoost model through Bayesian optimization, but the best I can achieve through Bayesian optimization when using Light GBM (my preferred choice) ...
1
vote
0answers
26 views

Hyperparameter Tuning with Simulated Data

I'm trying to create a SVM classifier which can predict some fault, and to train it I'm using simulated examples of the fault. Of course, the simulations are not perfect, but they appear to be good ...
1
vote
0answers
27 views

Any heuristic for minimal DCGAN latent space dimension?

I am highly interested in approaching minimal latent space dimension (as many other may be) for DCGANs or autoencoders. In this example of DCGAN on the MNIST dataset, the person uses a 100-...
1
vote
0answers
116 views

std::bad_alloc with densenet and hyperas

I am filing this issue after being stagnated here for couple of weeks. I am using hyperas to find the hyperparameters for my network, Densenet. My issue here is that my evaluation always fails with ...
1
vote
0answers
25 views

What to do when two algorithms perform better on different set of classes while doing multi-class classification?

If an algorithm X performed better on a set of classes U and another algorithm Y works better on set of classes V where U and V don's share any class, what improvements can be done from a data ...