Questions tagged [hyperparameter-tuning]

Hyperparameter tuning (also called hyperparameter optimization) refers to the process of finding the optimal set of hyperparameters for a given machine learning algorithm.

Filter by
Sorted by
Tagged with
3 votes
1 answer
58 views

How are parameters selected in cross-validation?

Suppose I'm training a linear regression model using k-fold cross-validation. I'm training K times each time with a different training and test data set. So each time I train, I get different ...
user avatar
  • 109
0 votes
0 answers
24 views

How to suppress "Estimator fit failed. The score on this train-test" warning message?

I am working on hyper-tuning random forest classifier with following parameters in random search CV ...
user avatar
0 votes
0 answers
7 views

How to choose max layers and units to search over in hyper parameter tuning

When performing any hyper parameter tuning, let's say random search for simplicity, and I want to search over a minimum to max units/nodes in a layer, and a minimum to max number of layers, are there ...
user avatar
  • 21
0 votes
1 answer
13 views

What does a leaf size of 1 in K-neighbors regression mean?

I am doing hyperparameter tuning + cross validation and I'm constantly getting that the optimal size of the leaf should be 1. Should I worry? Is this a sign of overfitting?
user avatar
  • 109
0 votes
0 answers
12 views

When using optuna I should return accuracy or loss as objective value?

I am using optuna for hyperparameter tuning for my segmentation model. At the model, I am returning accuracy as an objective value since I realised that it tries to optimize to get the best result ...
user avatar
0 votes
0 answers
15 views

Is Loss value (e.g., MSE loss) used in the calculation for parameter update when doing gradient descent?

My question is really simple. I know the theory behind gradient descent and parameter updates, what I really haven't found clarity on is that is the loss value (e.g., MSE value) used, i.e., multiplied ...
user avatar
  • 1
0 votes
0 answers
8 views

Efficient Searching for a basis of information as a hyperparameter in a large possible hyperparameter space

I have a set of inputs, let's call them 'I', that can be fed through a complicated group of functions to produce/calculate a wide variety of outputs (let's call them 'O'). I want to find a subset of ...
user avatar
  • 13
0 votes
0 answers
11 views

Rules, rules of thumb, intuitions, on how to set up the best possible hyperparameter search

When I set up my neural networks, I really have very little idea what I'm doing in advance. It may just be a bit of educated guesswork as to "it may need a few layers only" or "this ...
user avatar
  • 21
0 votes
0 answers
9 views

Hyperparameter tuning on training data vs validation data

If we divide the data into training data, validation data, and testing data, I remember the lesson from Andrew Ng saying we use the validation data for hyperparameter tuning purpose. (you can see this ...
user avatar
1 vote
1 answer
18 views

How variable alpha changes SGDRegressor behavior for outlier?

I am using SGDRegressor with a constant learning rate and default loss function. I am curious to know how changing the alpha parameter in the function from 0.0001 to 100 will change regressor behavior....
user avatar
  • 111
0 votes
0 answers
12 views

What is the difference between keras tuned hyperparameters and manually defined Sequential model with same hyperparameters?

I have a dataset that I divided into 10 splits of training, validation and test sets for a regression problem. I used the first split and RandomSearch in ...
user avatar
  • 1
0 votes
0 answers
10 views

Activation Function Hyperparameter Optimisation

If I have a model, say: ...
user avatar
  • 21
1 vote
1 answer
21 views

HyperOpt: Finding the best modeling based on precision or f1 score

I have been using the hyperopt for 2 days now and I am trying to create logistic regression models using the hyperopt and choosing the best combination of parameters by their f1 scores. However, ...
user avatar
  • 11
0 votes
0 answers
20 views

Drop Out in Hyperparameter Optimisation

Is it correct to add dropout to each layer and that it is done as in the below example? class MyHyperModel(kt.HyperModel): def build_model(self, hp): ...
user avatar
  • 21
0 votes
0 answers
12 views

Should hyperparameter optimisation focus on many trials (models) lower epochs first, then a second round with few models, many epochs?

Rather than a hyperparameter optimisation with kt.tuners.RandomSearch, say, that does (option A), say X model trials (e.g. 100), Y epochs each (say 100, so a total of 10,000 epochs across all models) ...
user avatar
  • 21
0 votes
0 answers
33 views

best trial always found at first optuna trial

I am using optuna as part of the pytorch forecasting library. I executed the following code: ...
user avatar
  • 101
0 votes
1 answer
21 views

Training Loss or Validation Loss for Hyperparameter Optimisation

When performing HO, should I be looking to train each model (each with different hyperparameter values, e.g. with RandomSearch picking those values) on the training data, and then the best one is ...
user avatar
  • 21
0 votes
0 answers
19 views

High loss but low rmse, how?

I have trained an lstm model on a dataset but its loss during training is ten times than the rmse during test. How is it possible, and can I use this model if rmse is very low but loss is high? How ...
user avatar
0 votes
0 answers
8 views

sklearn.neighbors.KernelDensity - score(X) explanation

For sklearn.neighbors.KernelDensity, its score(X) method according to the sklearn KDE documentation says: Compute the log-...
user avatar
  • 121
2 votes
0 answers
55 views

How many dense layers is enough?

I'm pretty new to NNs so sorry if the following is a silly question but: Is it helpful to stack multiple Keras dense layers on each other? I am familiar with the universal approximation theorem. But I ...
user avatar
0 votes
0 answers
10 views

Discussion: Is hyperparameter tuning on linux/ubuntu virtual machine a usefull approach?

In order to get an R/Shiny forecasting app ready for production, I am concerned about speeding up the model tuning process. I am already using parallel processing on Windows 10. There are some other ...
user avatar
0 votes
0 answers
69 views

KerasClassifier with random search: can't pickle _thread.RLock objects

I created a simple neural network for binary spam/ham text classification using pretrained BERT transformer. Now I want to apply randomized search for tuning the hyperparameters. For now the only ...
user avatar
  • 257
0 votes
1 answer
37 views

Is it a good practice to use hyperparameter tuning in production pipeline?

I'm studying TensorFlow Extended and I can see that it's training pipeline includes a "Tuner" component for hyperparameter tuning. As a consequence, I'm wondering if inclusion of tuning is a ...
user avatar
  • 867
0 votes
2 answers
87 views

binary classification pipeline to select threshold

There are quite a few questions regarding the optimisation of binary threshold in a classification problem. However, I haven't found a single end-to-end solution to this problem. In an existing ...
user avatar
  • 41
0 votes
1 answer
73 views

Recommendations for tuning XGBoost Hyperparams?

XGBoost has quite a few hyperparameters to tune: max depth, min child weight, number of iterations, eta, gamma, percent of columns considered, and percent of samples considered. It's computationally ...
user avatar
0 votes
1 answer
29 views

why sign flip to indicate loss in hyperopt? [closed]

I am using the hyperopt to find best hyperparameters for Random forest. My objective is to get the parameters which returns the best f1-score as my dataset is ...
user avatar
  • 2,129
0 votes
1 answer
38 views

How to train multioutput classification with hyperparameter tuning in sklearn?

I am working on a simple multioutput classification problem and noticed this error showing up whenever running the below code: ...
user avatar
  • 257
2 votes
1 answer
15 views

Find smooth global maximum from noisy points

Let's say I have a number of sampled data points and resulting values for each. In practice this may be a high dimensional problem, but here's a one dimensional example: In the above example, the ...
user avatar
1 vote
1 answer
38 views

Dimension error when tuning LSTM layer

I am working on a sentiment analysis problem which is a binary classification. These are some of the parameters that might be useful: 1.) Length of train list = 203 2.) Length of test list = 51 3.) ...
user avatar
  • 1,241
0 votes
0 answers
246 views

HuggingFace transformer: CUDA out memory only when performing hyperparameter search

I am working with a GTX3070, which only has 8GB of GPU RAM. When I am running using trainer.train(), I run fine with a maximum batch size of 7 (6 if running in Jupiter notebook). However, when I ...
user avatar
0 votes
0 answers
9 views

How to specify Search Space in Auto-Sklearn

I know how to specify Feature Selection methods and the list of the Algorithms used in Auto-Sklearn 2.0 ...
user avatar
  • 497
0 votes
0 answers
16 views

version control for code and output models

I have a question about version control for both code and the models it generates. We are developing ML models that often involve hyperparameters and so we might do many runs with different ...
user avatar
0 votes
0 answers
80 views

Xgboost taking some time to run vs hyperopt

Sorry for long post,im triying to run a xgb model but for some reason takes like 20 to 30 min(per run) with a specific set of hyperparams, but when i run hyperopt to get best params, takes like 7 ...
user avatar
0 votes
1 answer
74 views

How to combine preprocessor/estimator selection with hyperparameter tuning using sklearn pipelines?

I'm aware of how to use sklearn.pipeline.Pipeline() for simple and slightly more complicated use cases alike. I know how to set up pipelines for homogeneous as well ...
user avatar
  • 257
0 votes
0 answers
42 views

Time Series Hyperparameter Tuning

My question is about the intuition for hyperparameter tuning of time series. In other models, like Linear or Logistic Regression there is labeled data and according to accuracy or precision, the ...
user avatar
  • 73
0 votes
0 answers
65 views

Variance Smoothing in Hyperparameter Tuning of Naive Bayes

param_grid_nb = { 'var_smoothing': np.logspace(0,-9, num=100) } var_smoothing: float, default=1e-9 --> Meaning- Portion of the largest variance of all features that is added to variances for ...
user avatar
  • 221
0 votes
0 answers
15 views

Query regarding variance smoothing in Naive Bayes classification

var_smoothing: float, default=1e-9 --> Portion of the largest variance of all features that is added to variances for calculation stability. Source: https://scikit-learn.org/stable/modules/...
user avatar
  • 221
0 votes
0 answers
8 views

Implement NestedCV into Neural Networks

I have a regression task for which I am using ML models. My input features are 64. I implement NestedCV to get best ML models and hyperparameters. I have recently learned Neural Networks and want to ...
user avatar
  • 1,241
2 votes
1 answer
80 views

Cross validation and hyperparameter tuning workflow

After reading a lot of articles on cross validation, I am now confused. I know that cross validation is used to get an estimate of model performance and is used to select the best algorithm out of ...
user avatar
  • 1,241
0 votes
1 answer
148 views

How to loop through multiple lists/dict?

I have the following code which finds the best value of k parameter in the KNNImputer. Basically it is looping through the list ...
user avatar
  • 1,241
1 vote
0 answers
63 views

Tuning Batch size and Learning rate in neural net

The following MCQ question is provided in "Exam Readiness: AWS Certified Machine Learning - Specialty" document. The correct answer has been marked in the document but I am not able to ...
user avatar
0 votes
0 answers
14 views

regression quality with meta score using R2 and MAE for optimisation

Considering quality of regression models I currently try to compare two types of information: The $R^2$ score that give me the information about the tendency of the predictor The $MAE$ (or $RMSE$) ...
user avatar
0 votes
1 answer
431 views

Optimal batch size and number of epoch for BERT

I use this tutorial https://www.tensorflow.org/text/tutorials/classify_text_with_bert and get different accuracy depend on epoch numbers and batch sizes. What's optimal parameters?
user avatar
0 votes
0 answers
22 views

Reduce Training steps for SSD-300

I am new to deep learning and I am trying to train my SSD-300 (single shot detector) model which is taking too long. For example even though I ran 50 epochs, it is training for 108370+ global steps. I ...
user avatar
0 votes
0 answers
12 views

BayesianOptimization tuning the same parameter with different results

I'm running a hyperparameter search using Keras wherein there is only one hyperparameter explicitly specified (# of LSTM units). However when running BayesianOptimization, after a while I notice it ...
user avatar
1 vote
1 answer
16 views

Tune learning rate while tuning other HP

When doing hyperparameters optimisation, like a Random Search, should you add a search space for the learning rate ? My intuition is that some HP might work better with a certain LR, and be sub-...
user avatar
0 votes
1 answer
34 views

Comparing accuracies of Grid Search CV & Randomized Search CV with K-Fold Cross Validation?

Are Grid Search CV & Randomized Search CV always/necessarily supposed to give more accurate results after hyperparameter tuning as compared to K-Fold Cross Validation?
user avatar
  • 221
1 vote
1 answer
51 views

Python Script on tuning tree in orange datamining

Is there any python script that can be used to check the best tuning in tree model on orange python datamining? Since theres an error when I applied the script from http://docs.biolab.si/orange/2/...
user avatar
0 votes
0 answers
14 views

selection of loss function to avoid overfitting by autoencoder in prediction a figure with a sharp rise

I have to select the loss function to avoid overfitting by autoencoder in prediction of this figure that has a sharp raise, I would like to find how to avoid overfitting by autoencoder in prediction a ...
user avatar
  • 1,526
1 vote
1 answer
27 views

Could I directly apply techniques for hyper-parameter tuning, and choose the best model?

I have noticed in some sources the author first trains the model (say a model from scikit-learn) with the default hyper-parameters, and the model naturally gives a ...
user avatar

1
2 3 4 5