Skip to main content
OverflowAI is here! AI power for your Stack Overflow for Teams knowledge community. Learn more

Questions tagged [hyperparameter-tuning]

Hyperparameter tuning (also called hyperparameter optimization) refers to the process of finding the optimal set of hyperparameters for a given machine learning algorithm.

Filter by
Sorted by
Tagged with
0 votes
0 answers
9 views

Objective function in Bayesian Hyperparameter Tuning

I have a question that has been going around in my head for a while and I'd like to leverage the wisdom of the crowd for getting a few opinions on it. Let me describe the Problem: I have a relatively ...
Hive5's user avatar
  • 1
0 votes
0 answers
10 views

How to assess the stability of a DL model, after using k-fold cross-validation for hyperparameter tuning

I've recently completed the training of a deep learning model for a classification task, using a process that involves k-fold cross-validation for hyperparameter tuning Initially, I have divided my ...
o'hara's user avatar
  • 1
1 vote
1 answer
35 views

How do I automate testing and comparison of the performance of models with different layer depths, layer types, and unit counts?

I am testing the effects of different layer counts/depths, unit counts, and layer types for natural language processing. I made a Kaggle notebook where I manually create different layers and then ...
Joachim Rives's user avatar
2 votes
1 answer
128 views

How to select the optimal beam size for beam search?

Most Text Generation Models use beam search to select the optimal output candidate. How does one choose the optimal beam size? It would probably vary from task to task, dataset to dataset, and model ...
Tathagato Roy's user avatar
1 vote
1 answer
39 views

Feature selection or hyperparameter tuning first for 30 feature data

I have about 30 variables and trying to create a Random Forest model. All the variables are expected to be predictors of outcome. I want to find the best model based on a C-stat score with any number ...
user2704338's user avatar
1 vote
1 answer
139 views

Why is it so common to focus only validation performance during hyper-parameter optimization

Assuming a standard train/validation/test split, the common practice is (a) to train multiple models with different hyper-parameter configurations on the training set, (b) to evaluate these models ...
Enk9456's user avatar
  • 105
0 votes
0 answers
12 views

Does it make sense to do hp tuning for a Random Forest for top k precision or recall?

I've trained an RF with a binary classification task that achieves mediocre performance. However, they way it is intended to be used would have end-users look only at predictions with high scores (...
ds_banter's user avatar
1 vote
1 answer
21 views

Hyperparameter tuning

Jane trains three different classifiers: Logistic Regression, Decision Tree, and Support Vector Machines on the training set. Each classifier has one hyper-parameter (regularisation parameter, depth-...
Tom's user avatar
  • 11
0 votes
0 answers
23 views

Hyperopt: initializing the search with manually defined points

So I am making a transition from the bayesian_optimization (bayesopt) package to hyperopt. A great feature of bayesopt is that it allows to initiate the search with manually selected points (probing). ...
Enk9456's user avatar
  • 105
0 votes
1 answer
37 views

Add tuning stage to DVC pipeline

I have an ML pipeline built with DVC that I use for experiment tracking. This allows running and tracking several experiments. Also, using hydra integration I can grid search hyper parameters. However,...
giulatona's user avatar
1 vote
1 answer
89 views

Ordering of Train/Val/Test set use in hyperparameter tuning

The way I read almost lots of ML advice on these datasets sounds like "You train a model that's randomly chosen hyperparameters first on the training set, then you ignore this bit of the work, ...
Socorro's user avatar
  • 111
0 votes
0 answers
22 views

Bayes HyperParameter tuning using wandb

Here in below code, I'm trying to use wandb sweep to find optimal lr, weight-decay using the below code: ...
Sarvagya Porwal's user avatar
0 votes
0 answers
42 views

What are typical hyperparameter ranges and significance for SB3's offline algorithm?

I'm trying to perform an hyperparameter tuning on a SAC algorithm (Stable Baselines 3) with optuna lib. I guess hyperparameters have different significance and typical ranges depending of the selected ...
GerardL's user avatar
  • 13
0 votes
0 answers
5 views

Hyperparameter tuning in Deep Learning Regression Problem

I have searched many resources, but know comprehensive guide as which HyperModel to use and why? And how to do the calculations step by step. If anybody can help I will be very grateful.
Subhadip Chattopadhyay's user avatar
0 votes
1 answer
37 views

For some reason getting an odd error when hyper tuning my model

I was hoping to hypertune my decisiontree model , however I keep running into this error: TypeError: DecisionTreeClassifier() got an unexpected keyword argument 'criterion' here what I tried: ...
John Adams's user avatar
0 votes
0 answers
8 views

Has someone designed a neural network which can select its own activation functions and/or have multiple activation functions in one model?

I'm wonder if there are any papers or implementations where a neural network has multiple activation functions in a single model (and layer), and preferably also where such activation functions ...
BigMistake's user avatar
0 votes
0 answers
154 views

GridSearchCV with TimeSeriesSplit

I am tuning the HPs for a time dependant neural network model (heating energy consumption prediction) in a bachelor thesis. Total amount of samples is 145.860, with minutely granularity from January ...
Anna Clara Dottaviano Morelli's user avatar
0 votes
0 answers
20 views

Parameter outside specified range using HGboost/Hyperopt library

I am trying to use the HGboost library which uses the Hyperopt library for doing hyperparameter optimization of an XGboost model. The script runs fine but the optimized parameter for "...
AWGIS's user avatar
  • 101
0 votes
0 answers
29 views

How are the successive sets of training samples that are allocated for each iteration of HalvingGridSearchCV determined?

The scikit-learn classes HalvingGridSearchCV and HalvingRandomSearchCV implement a hyperparameter tuning method known as successive halving. It is an iterative selection process in which all the ...
Evan Aad's user avatar
  • 175
0 votes
0 answers
24 views

What are the most important hyperparameters to tune to optimize the training of a 3D U-NET used for pixel classification?

Leaving aside the training loss, the optimizer (ADAM), the number of U-NET blocks (tuned to a meaningful target receptive field for the problem) and the number of filters per layer (set to the maximum ...
Sebastien's user avatar
0 votes
0 answers
111 views

Using "precomputed" distance matrices as input to scikit-learn clustering metrics

Is there any validity to using a distance matrix instead of the raw points with metrics such as davies_bouldin_score and ...
Chris Coffee's user avatar
0 votes
0 answers
49 views

How to get best result by using hyperparameter tuning and stacking regression?

I want to use regression ML models to get the best possible R2 score. So I decided to do both hyperparameter tuning and combining models with stacking regressor. I'm wondering if I should do ...
User's user avatar
  • 1
0 votes
0 answers
141 views

Training with optuna-tuned hyperparameters leads to different results

I'm training an image classifier in Pytorch Lightning and tuning hyperparameters with Optuna. When I use the best hyperparameters to train a separate model, the accuracies differ from those obtained ...
Tirtha's user avatar
  • 33
0 votes
0 answers
10 views

What are 'standard' parameters to try when hyperparameter tuning?

I'm curious what parameters people tend to tune for various models and the values they try. For instance, on this post, the author had a go-to set of parameters he would use for tuning. What is the ...
Marsha T's user avatar
0 votes
0 answers
31 views

why is there no research on machine learning algorithms to determine optimal hyperparameters for metaheuristics?

I am not shure if I am in the correct forum for this question. I'm sorry, if I'm in the wrong place here. Question: Why is there no research on machine learning algorithms to determine hyperparameters ...
Andre's user avatar
  • 101
0 votes
0 answers
78 views

Problem calculating f-1 score when when running hyperparameter tuning using Optuna

I'm trying to run Stratified CV hyperparameter tuning on an XGBClassifier using Optuna. This is a multiclass problem with 3 classes (labelled 0 through 2 in the "classes" variable). These ...
Metrician's user avatar
0 votes
0 answers
13 views

Test accuracy plateaus when increasing max_depth -> inf

I've built a Random Forest model that classifies into four categories based on around 10 input features. To test the accuracy, I performed 5-fold stratified cross validation using the ...
okjdlsksjdwi's user avatar
0 votes
1 answer
34 views

Cross validation and train_test_split

I am building a class that follows the workflow: Model Selection and Fitting The class accepts a list of models and their respective hyperparameter grids. It then performs a standard fitting process ...
Guilherme Raibolt's user avatar
2 votes
1 answer
48 views

Changing model architecture doesn't impact results

I am currently learning binary classification. The problem is classifying positive and negative movie reviews. The dataset is 25,000 reviews with each review represented by 10,000 of the most used ...
Omer Mualem's user avatar
0 votes
0 answers
51 views

Weights And Biases Sweep Across Multiple Datasets and Model Architectures

TLDR; How can I conduct a sweep in weights and biases across not just hyperparameters, but across model architectures and datasets as well, in a way that respects sensible aggregation and ...
Warlax56's user avatar
  • 430
0 votes
0 answers
23 views

Can somebody explain to me, how the Hyperparameters in this expamle work? I only uderstood the meaning for gamma

I am quite new to this topic, but I want to understand how Reinforcement Learning (RL) works in this example (https://gymnasium.farama.org/tutorials/training_agents/reinforce_invpend_gym_v26/). I have ...
Peter's user avatar
  • 1
0 votes
0 answers
7 views

How to interpet the bolded lines in bayesopt

I am using bayesopt to maximize a function that is everywhere less or equal to zero $(f(x) \leq 0)$. The score is essentially a negated Mean Absolute Error because the default behavior of bayesopt is ...
Enk9456's user avatar
  • 105
0 votes
0 answers
55 views

Build a model with cross-validation on entire dataset to learn insights?

Goal : Use XGBoost regression to learn insights from data. Prediction or forecasting not needed. Hypothesis : If the model fits the entire dataset well, it can maybe capture its "physics" in ...
cwanderroycbooks's user avatar
0 votes
0 answers
67 views

How to grid search LSTM manually using TimeSeriesSplit while preserve/load best_models?

This is a self-written LSTM tuner class. ...
frr0717's user avatar
0 votes
0 answers
38 views

Surprise NMF object is not callable

I am building a recommender system using the Sushi Preference Dataset and the NMF (Non-negative Matrix Factorization) model. I am implementing the same using the Surprise library. I want to use ...
Sumant Chopde's user avatar
1 vote
2 answers
46 views

which hyperparameters are returned as best in cross validation?

The description on the RandomizedSearchCV says this about best hyperparameters : "Estimator that was chosen by the search, i.e. estimator which gave highest ...
pppp_prs's user avatar
0 votes
0 answers
33 views

How do I automatically evaluate an objective_plot after BayesSearchCV to find the *theoretical* optimal model?

I did a hyper optimization for a XGBClassifier using BayesSearchCV. I increased the kappa ...
Jack Sabbath's user avatar
0 votes
0 answers
132 views

Why it gives Node: 'model_8/dense_23/Relu' Matrix size-incompatible

This is my code: ...
Tay Kim Gaik's user avatar
0 votes
1 answer
385 views

Is hyperparameter tuning done on training or validation data set?

Is hyperparameter tuning done on training or validation data set? The post here gives mixed opinion as of whether the training set should be used for hyperparameter tuning. And I would like to know ...
Student's user avatar
  • 411
0 votes
0 answers
40 views

Hyperparameters values changing on every run

...
Akshita's user avatar
0 votes
1 answer
29 views

Flow of machine learning model including code

I'm towards the completion of my first data science project that will go into my GitHub portfolio. I'll be happy for some clarification regarding the machine learning models section: I got a little ...
Sigal Cohen's user avatar
0 votes
1 answer
251 views

Is it mandatory to set a random_state when using RandomizedSearchCV?

When I use RandomizedSearchCV, if I put the random state I always obtain the same results with the same hyperparams trainer. So, is it mandatory to use? Because in my opinion it is better to always ...
Flavio Brienza's user avatar
0 votes
0 answers
70 views

Is it possible to perform probability calibration with a model with the best hyperparams?

If I use RandomizedSearchCV to find the optimal hyperparams of a model, can I create another model, with those parameters, to calibrate probabilities using CalibratedClassifierCV? The new model is not ...
Flavio Brienza's user avatar
1 vote
0 answers
258 views

Optuina pruning during CrossValidation, does it make sense?

I'm currently trying to build a model using CatBoost. For my parameter tuning, I'm using optuna and cross-validation and pruning the trial checking on the intermediate cross-validation scores. Here ...
GiusWestsideDS's user avatar
1 vote
1 answer
150 views

Can I fit a model with the parameters found with RandomizedSearchCV?

I want to ask you a question. Suppose I use the following RandomizedSearchCV to find the model's best hyperparams: ...
Flavio Brienza's user avatar
0 votes
1 answer
50 views

Question about grid search and KFold

I am trying an example which I am training on a huge dataset 5M (only 4 features) rows with Cudf and CUml and I am using SGD logistic regression because I must predict if the patient if is sick or not ...
gkasap's user avatar
  • 1
1 vote
1 answer
84 views

Which of 2 options is better practice for model optimization: 1) Nested CV wrongly averaging inner CV scores. 2) Two successive CVs on X_all. Altrntv?

Goal: Compare preprocessing methods, models, and hyperparameters without leaking into the final generalization estimate, applying cross-validation (cv), i.e. NOT applying any fixed train/test splits. ...
le8rning's user avatar
0 votes
1 answer
399 views

Tuned model has higher CV accuracy, but a lower test accuracy. Should I use the tuned or untuned model?

I am working on a classification problem using Sci Kit Learn and am confused on how to properly tune hyper parameters to get the "best" model. Before any tuning, my logistic regression ...
d0dg3r_k1d's user avatar
0 votes
1 answer
34 views

Train/val/test approach for hyperparameter tuning

When looking to train a model, does it make sense to have a 60-20-20 train val test split, first hyper parameter tuning over the training dataset, using the validation set, picking the best model. ...
Socorro's user avatar
  • 111
1 vote
1 answer
393 views

How to determine which combinations of parameters to include in GridSearchCV

I am using MLPClassifier from sklearn and I would like to tune it with GridSearchCV. But I don't know which set of values to include for hidden_layer_sizes, max_iter, activation, solver, etc. How can ...
Penjan's user avatar
  • 11

1
2 3 4 5 6