Questions tagged [hyperparameter-tuning]

Hyperparameter tuning (also called hyperparameter optimization) refers to the process of finding the optimal set of hyperparameters for a given machine learning algorithm.

Filter by
Sorted by
Tagged with
0 votes
0 answers
10 views

Bayes HyperParameter tuning using wandb

Here in below code, I'm trying to use wandb sweep to find optimal lr, weight-decay using the below code: ...
Sarvagya Porwal's user avatar
0 votes
0 answers
17 views

What are typical hyperparameter ranges and significance for SB3's offline algorithm?

I'm trying to perform an hyperparameter tuning on a SAC algorithm (Stable Baselines 3) with optuna lib. I guess hyperparameters have different significance and typical ranges depending of the selected ...
GerardL's user avatar
0 votes
0 answers
5 views

Hyperparameter tuning in Deep Learning Regression Problem

I have searched many resources, but know comprehensive guide as which HyperModel to use and why? And how to do the calculations step by step. If anybody can help I will be very grateful.
Subhadip Chattopadhyay's user avatar
0 votes
1 answer
29 views

For some reason getting an odd error when hyper tuning my model

I was hoping to hypertune my decisiontree model , however I keep running into this error: TypeError: DecisionTreeClassifier() got an unexpected keyword argument 'criterion' here what I tried: ...
John Adams's user avatar
0 votes
0 answers
7 views

Has someone designed a neural network which can select its own activation functions and/or have multiple activation functions in one model?

I'm wonder if there are any papers or implementations where a neural network has multiple activation functions in a single model (and layer), and preferably also where such activation functions ...
BigMistake's user avatar
0 votes
0 answers
27 views

GridSearchCV with TimeSeriesSplit

I am tuning the HPs for a time dependant neural network model (heating energy consumption prediction) in a bachelor thesis. Total amount of samples is 145.860, with minutely granularity from January ...
Anna Clara Dottaviano Morelli's user avatar
0 votes
0 answers
4 views

Parameter outside specified range using HGboost/Hyperopt library

I am trying to use the HGboost library which uses the Hyperopt library for doing hyperparameter optimization of an XGboost model. The script runs fine but the optimized parameter for "...
AWGIS's user avatar
  • 101
0 votes
0 answers
19 views

How are the successive sets of training samples that are allocated for each iteration of HalvingGridSearchCV determined?

The scikit-learn classes HalvingGridSearchCV and HalvingRandomSearchCV implement a hyperparameter tuning method known as successive halving. It is an iterative selection process in which all the ...
Evan Aad's user avatar
  • 101
0 votes
0 answers
9 views

What are the most important hyperparameters to tune to optimize the training of a 3D U-NET used for pixel classification?

Leaving aside the training loss, the optimizer (ADAM), the number of U-NET blocks (tuned to a meaningful target receptive field for the problem) and the number of filters per layer (set to the maximum ...
Sebastien's user avatar
0 votes
0 answers
42 views

Using "precomputed" distance matrices as input to scikit-learn clustering metrics

Is there any validity to using a distance matrix instead of the raw points with metrics such as davies_bouldin_score and ...
Chris Coffee's user avatar
0 votes
0 answers
24 views

How to get best result by using hyperparameter tuning and stacking regression?

I want to use regression ML models to get the best possible R2 score. So I decided to do both hyperparameter tuning and combining models with stacking regressor. I'm wondering if I should do ...
User's user avatar
  • 1
0 votes
0 answers
62 views

Training with optuna-tuned hyperparameters leads to different results

I'm training an image classifier in Pytorch Lightning and tuning hyperparameters with Optuna. When I use the best hyperparameters to train a separate model, the accuracies differ from those obtained ...
Tirtha's user avatar
  • 33
0 votes
0 answers
10 views

What are 'standard' parameters to try when hyperparameter tuning?

I'm curious what parameters people tend to tune for various models and the values they try. For instance, on this post, the author had a go-to set of parameters he would use for tuning. What is the ...
Marsha T's user avatar
0 votes
0 answers
27 views

why is there no research on machine learning algorithms to determine optimal hyperparameters for metaheuristics?

I am not shure if I am in the correct forum for this question. I'm sorry, if I'm in the wrong place here. Question: Why is there no research on machine learning algorithms to determine hyperparameters ...
Andre's user avatar
  • 101
0 votes
0 answers
42 views

Problem calculating f-1 score when when running hyperparameter tuning using Optuna

I'm trying to run Stratified CV hyperparameter tuning on an XGBClassifier using Optuna. This is a multiclass problem with 3 classes (labelled 0 through 2 in the "classes" variable). These ...
Metrician's user avatar
0 votes
0 answers
9 views

Test accuracy plateaus when increasing max_depth -> inf

I've built a Random Forest model that classifies into four categories based on around 10 input features. To test the accuracy, I performed 5-fold stratified cross validation using the ...
okjdlsksjdwi's user avatar
0 votes
1 answer
29 views

Cross validation and train_test_split

I am building a class that follows the workflow: Model Selection and Fitting The class accepts a list of models and their respective hyperparameter grids. It then performs a standard fitting process ...
Guilherme Raibolt's user avatar
2 votes
1 answer
30 views

Changing model architecture doesn't impact results

I am currently learning binary classification. The problem is classifying positive and negative movie reviews. The dataset is 25,000 reviews with each review represented by 10,000 of the most used ...
Omer Mualem's user avatar
0 votes
0 answers
28 views

Weights And Biases Sweep Across Multiple Datasets and Model Architectures

TLDR; How can I conduct a sweep in weights and biases across not just hyperparameters, but across model architectures and datasets as well, in a way that respects sensible aggregation and ...
Warlax56's user avatar
  • 430
0 votes
0 answers
22 views

Can somebody explain to me, how the Hyperparameters in this expamle work? I only uderstood the meaning for gamma

I am quite new to this topic, but I want to understand how Reinforcement Learning (RL) works in this example (https://gymnasium.farama.org/tutorials/training_agents/reinforce_invpend_gym_v26/). I have ...
Peter's user avatar
  • 1
0 votes
0 answers
6 views

How to interpet the bolded lines in bayesopt

I am using bayesopt to maximize a function that is everywhere less or equal to zero $(f(x) \leq 0)$. The score is essentially a negated Mean Absolute Error because the default behavior of bayesopt is ...
Enk9456's user avatar
  • 93
0 votes
0 answers
47 views

Build a model with cross-validation on entire dataset to learn insights?

Goal : Use XGBoost regression to learn insights from data. Prediction or forecasting not needed. Hypothesis : If the model fits the entire dataset well, it can maybe capture its "physics" in ...
cwanderroycbooks's user avatar
0 votes
0 answers
28 views

How to grid search LSTM manually using TimeSeriesSplit while preserve/load best_models?

This is a self-written LSTM tuner class. ...
frr0717's user avatar
0 votes
0 answers
28 views

Surprise NMF object is not callable

I am building a recommender system using the Sushi Preference Dataset and the NMF (Non-negative Matrix Factorization) model. I am implementing the same using the Surprise library. I want to use ...
Sumant Chopde's user avatar
1 vote
2 answers
41 views

which hyperparameters are returned as best in cross validation?

The description on the RandomizedSearchCV says this about best hyperparameters : "Estimator that was chosen by the search, i.e. estimator which gave highest ...
pppp_prs's user avatar
0 votes
0 answers
23 views

How do I automatically evaluate an objective_plot after BayesSearchCV to find the *theoretical* optimal model?

I did a hyper optimization for a XGBClassifier using BayesSearchCV. I increased the kappa ...
Jack Sabbath's user avatar
0 votes
0 answers
84 views

Why it gives Node: 'model_8/dense_23/Relu' Matrix size-incompatible

This is my code: ...
Tay Kim Gaik's user avatar
0 votes
1 answer
96 views

Is hyperparameter tuning done on training or validation data set?

Is hyperparameter tuning done on training or validation data set? The post here gives mixed opinion as of whether the training set should be used for hyperparameter tuning. And I would like to know ...
Student's user avatar
  • 411
0 votes
0 answers
208 views

Keras Tuner Error: Failed to find data adapter that can handle input: <class 'function'>, <class 'NoneType'>

I am working on a project where I have a dataset of emails with information such as subject, body, attachment names, etc. My goal is to classify/predict around ~5,000 different sites. I am using Keras ...
patchie's user avatar
  • 101
0 votes
0 answers
34 views

Hyperparameters values changing on every run

...
Akshita's user avatar
0 votes
0 answers
47 views

Walk forward cross-validation with Optuna and deepar in pytorch forecasting

I want to perform 3 splits walk forward cross validation with expanding training set for the deepar model from the pytorch forecasting framework. When I do walk forward validation, I also want to do ...
Jose_Peeterson's user avatar
0 votes
1 answer
26 views

Flow of machine learning model including code

I'm towards the completion of my first data science project that will go into my GitHub portfolio. I'll be happy for some clarification regarding the machine learning models section: I got a little ...
Sigal Cohen's user avatar
0 votes
0 answers
31 views

Is this a valid cross validation approach to choose hyperparameters and get a good estimate of model performace?

I am using lightGBM on time series data. I first split my data set into 10% folds. The last fold is used as a test set. For each choice of hyperparameters I first use 6 folds to train, then predict on ...
user3376601's user avatar
0 votes
0 answers
43 views

how does Optuna execute the objective function?

how does Optuna execute the objective function when calling study.optimize(objective, n_trials=10, is it, for each trial ===> call the function ??, if this is the case, this means we shouldn't ...
Kaoutar's user avatar
  • 101
0 votes
1 answer
82 views

Is it mandatory to set a random_state when using RandomizedSearchCV?

When I use RandomizedSearchCV, if I put the random state I always obtain the same results with the same hyperparams trainer. So, is it mandatory to use? Because in my opinion it is better to always ...
Flavio Brienza's user avatar
0 votes
0 answers
38 views

Is it possible to perform probability calibration with a model with the best hyperparams?

If I use RandomizedSearchCV to find the optimal hyperparams of a model, can I create another model, with those parameters, to calibrate probabilities using CalibratedClassifierCV? The new model is not ...
Flavio Brienza's user avatar
0 votes
0 answers
31 views

Matlab - Machine Learning: Hyperparameter tuning of a fitcecoc - model + training it on new data

I have trained a logistic regression multi-class model in Matlab for multi-class classification using XTrainSet_A / YTrainSet_A, look at this simplified code: ...
De Ro's user avatar
  • 1
0 votes
0 answers
36 views

how to define search space over hyperparameters automatically?

I'm trying to automate retrain steps of our ML models. My aim is hyper-parameter tuning with current data (newer performance window) using same algorithm and features on production environment. ...
Atacan's user avatar
  • 1
1 vote
0 answers
167 views

Optuina pruning during CrossValidation, does it make sense?

I'm currently trying to build a model using CatBoost. For my parameter tuning, I'm using optuna and cross-validation and pruning the trial checking on the intermediate cross-validation scores. Here ...
GiusWestsideDS's user avatar
1 vote
1 answer
78 views

Can I fit a model with the parameters found with RandomizedSearchCV?

I want to ask you a question. Suppose I use the following RandomizedSearchCV to find the model's best hyperparams: ...
Flavio Brienza's user avatar
0 votes
0 answers
17 views

Choosing which parameters and what range to hyperparameter tune in a model?

Lets say I built an XGBoost model with XGBClassifier(). I know I can get default params by calling the default parameters function. However, there are so many ...
Katsu's user avatar
  • 111
0 votes
0 answers
23 views

Rules/Guidelines for Custom Weightage and Hyper-parameter tuning

I have a movie and user-ratings dataset. After implementing the content-based filtering technique, I figured, I can improvise the results even further by assigning weightage to the parameters based on ...
shripal mehta's user avatar
0 votes
0 answers
12 views

Why does cross validation and hyperparameter tuning work?

To my understanding, optimizing a model with k-fold cross validation and hyperparameter tuning are tools to be used mostly for small datasets to really make the most out of very limited/expensive data,...
frankL's user avatar
  • 1
0 votes
0 answers
9 views

Optimize wordembedding and neural network at the same time

I have a lot of (domain)-specific text that I want to classify into 100+ categories. I want to train a wordembedding (FastText) and use that in conjuction with a CNN, thus I'm running into the problem ...
CutePoison's user avatar
0 votes
1 answer
39 views

Question about grid search and KFold

I am trying an example which I am training on a huge dataset 5M (only 4 features) rows with Cudf and CUml and I am using SGD logistic regression because I must predict if the patient if is sick or not ...
gkasap's user avatar
  • 1
1 vote
1 answer
66 views

Which of 2 options is better practice for model optimization: 1) Nested CV wrongly averaging inner CV scores. 2) Two successive CVs on X_all. Altrntv?

Goal: Compare preprocessing methods, models, and hyperparameters without leaking into the final generalization estimate, applying cross-validation (cv), i.e. NOT applying any fixed train/test splits. ...
le8rning's user avatar
0 votes
0 answers
64 views

Objective function in optuna seems to return at random points

I am trying to use a K-fold cross-validation within the optuna objective function. Unfortunately, the output of the objective function seems to pop out at random places, not at the end of the cross-...
Adrian Mureșan's user avatar
0 votes
1 answer
209 views

Tuned model has higher CV accuracy, but a lower test accuracy. Should I use the tuned or untuned model?

I am working on a classification problem using Sci Kit Learn and am confused on how to properly tune hyper parameters to get the "best" model. Before any tuning, my logistic regression ...
d0dg3r_k1d's user avatar
0 votes
1 answer
24 views

Train/val/test approach for hyperparameter tuning

When looking to train a model, does it make sense to have a 60-20-20 train val test split, first hyper parameter tuning over the training dataset, using the validation set, picking the best model. ...
Socorro's user avatar
  • 91
1 vote
1 answer
228 views

How to determine which combinations of parameters to include in GridSearchCV

I am using MLPClassifier from sklearn and I would like to tune it with GridSearchCV. But I don't know which set of values to include for hidden_layer_sizes, max_iter, activation, solver, etc. How can ...
Penjan's user avatar
  • 11

1
2 3 4 5 6