Questions tagged [hyperparameter-tuning]
Hyperparameter tuning (also called hyperparameter optimization) refers to the process of finding the optimal set of hyperparameters for a given machine learning algorithm.
290
questions
0
votes
0
answers
10
views
Bayes HyperParameter tuning using wandb
Here in below code, I'm trying to use wandb sweep to find optimal lr, weight-decay using the below code:
...
0
votes
0
answers
17
views
What are typical hyperparameter ranges and significance for SB3's offline algorithm?
I'm trying to perform an hyperparameter tuning on a SAC algorithm (Stable Baselines 3) with optuna lib.
I guess hyperparameters have different significance and typical ranges depending of the selected ...
- 1
0
votes
0
answers
5
views
Hyperparameter tuning in Deep Learning Regression Problem
I have searched many resources, but know comprehensive guide as which HyperModel to use and why? And how to do the calculations step by step.
If anybody can help I will be very grateful.
0
votes
1
answer
29
views
For some reason getting an odd error when hyper tuning my model
I was hoping to hypertune my decisiontree model , however I keep running into this error:
TypeError: DecisionTreeClassifier() got an unexpected keyword argument 'criterion'
here what I tried:
...
0
votes
0
answers
7
views
Has someone designed a neural network which can select its own activation functions and/or have multiple activation functions in one model?
I'm wonder if there are any papers or implementations where a neural network has multiple activation functions in a single model (and layer), and preferably also where such activation functions ...
- 101
0
votes
0
answers
27
views
GridSearchCV with TimeSeriesSplit
I am tuning the HPs for a time dependant neural network model (heating energy consumption prediction) in a bachelor thesis.
Total amount of samples is 145.860, with minutely granularity from January ...
0
votes
0
answers
4
views
Parameter outside specified range using HGboost/Hyperopt library
I am trying to use the HGboost library which uses the Hyperopt library for doing hyperparameter optimization of an XGboost model. The script runs fine but the optimized parameter for "...
- 101
0
votes
0
answers
19
views
How are the successive sets of training samples that are allocated for each iteration of HalvingGridSearchCV determined?
The scikit-learn classes HalvingGridSearchCV and HalvingRandomSearchCV implement a hyperparameter tuning method known as successive halving. It is an iterative selection process in which all the ...
- 101
0
votes
0
answers
9
views
What are the most important hyperparameters to tune to optimize the training of a 3D U-NET used for pixel classification?
Leaving aside the training loss, the optimizer (ADAM), the number of U-NET blocks (tuned to a meaningful target receptive field for the problem) and the number of filters per layer (set to the maximum ...
0
votes
0
answers
42
views
Using "precomputed" distance matrices as input to scikit-learn clustering metrics
Is there any validity to using a distance matrix instead of the raw points with metrics such as davies_bouldin_score and ...
- 21
0
votes
0
answers
24
views
How to get best result by using hyperparameter tuning and stacking regression?
I want to use regression ML models to get the best possible R2 score. So I decided to do both hyperparameter tuning and combining models with stacking regressor. I'm wondering if I should do ...
- 1
0
votes
0
answers
62
views
Training with optuna-tuned hyperparameters leads to different results
I'm training an image classifier in Pytorch Lightning and tuning hyperparameters with Optuna. When I use the best hyperparameters to train a separate model, the accuracies differ from those obtained ...
- 33
0
votes
0
answers
10
views
What are 'standard' parameters to try when hyperparameter tuning?
I'm curious what parameters people tend to tune for various models and the values they try. For instance, on this post, the author had a go-to set of parameters he would use for tuning.
What is the ...
- 1
0
votes
0
answers
27
views
why is there no research on machine learning algorithms to determine optimal hyperparameters for metaheuristics?
I am not shure if I am in the correct forum for this question. I'm sorry, if I'm in the wrong place here.
Question:
Why is there no research on machine learning algorithms to determine hyperparameters ...
- 101
0
votes
0
answers
42
views
Problem calculating f-1 score when when running hyperparameter tuning using Optuna
I'm trying to run Stratified CV hyperparameter tuning on an XGBClassifier using Optuna. This is a multiclass problem with 3 classes (labelled 0 through 2 in the "classes" variable). These ...
- 11
0
votes
0
answers
9
views
Test accuracy plateaus when increasing max_depth -> inf
I've built a Random Forest model that classifies into four categories based on around 10 input features.
To test the accuracy, I performed 5-fold stratified cross validation using the ...
0
votes
1
answer
29
views
Cross validation and train_test_split
I am building a class that follows the workflow:
Model Selection and Fitting
The class accepts a list of models and their respective hyperparameter grids. It then performs a standard fitting process ...
2
votes
1
answer
30
views
Changing model architecture doesn't impact results
I am currently learning binary classification.
The problem is classifying positive and negative movie reviews.
The dataset is 25,000 reviews with each review represented by 10,000 of the most used ...
- 23
0
votes
0
answers
28
views
Weights And Biases Sweep Across Multiple Datasets and Model Architectures
TLDR; How can I conduct a sweep in weights and biases across not just hyperparameters, but across model architectures and datasets as well, in a way that respects sensible aggregation and ...
- 430
0
votes
0
answers
22
views
Can somebody explain to me, how the Hyperparameters in this expamle work? I only uderstood the meaning for gamma
I am quite new to this topic, but I want to understand how Reinforcement Learning (RL) works in this example (https://gymnasium.farama.org/tutorials/training_agents/reinforce_invpend_gym_v26/). I have ...
- 1
0
votes
0
answers
6
views
How to interpet the bolded lines in bayesopt
I am using bayesopt to maximize a function that is everywhere less or equal to zero $(f(x) \leq 0)$. The score is essentially a negated Mean Absolute Error because the default behavior of bayesopt is ...
- 93
0
votes
0
answers
47
views
Build a model with cross-validation on entire dataset to learn insights?
Goal : Use XGBoost regression to learn insights from data. Prediction or forecasting not needed.
Hypothesis : If the model fits the entire dataset well, it can maybe capture its "physics" in ...
0
votes
0
answers
28
views
How to grid search LSTM manually using TimeSeriesSplit while preserve/load best_models?
This is a self-written LSTM tuner class.
...
- 1
0
votes
0
answers
28
views
Surprise NMF object is not callable
I am building a recommender system using the Sushi Preference Dataset and the NMF (Non-negative Matrix Factorization) model. I am implementing the same using the Surprise library. I want to use ...
1
vote
2
answers
41
views
which hyperparameters are returned as best in cross validation?
The description on the RandomizedSearchCV says this about best hyperparameters :
"Estimator that was chosen by the search, i.e. estimator which gave highest ...
- 31
0
votes
0
answers
23
views
How do I automatically evaluate an objective_plot after BayesSearchCV to find the *theoretical* optimal model?
I did a hyper optimization for a XGBClassifier using BayesSearchCV. I increased the kappa ...
- 101
0
votes
0
answers
84
views
Why it gives Node: 'model_8/dense_23/Relu' Matrix size-incompatible
This is my code:
...
0
votes
1
answer
96
views
Is hyperparameter tuning done on training or validation data set?
Is hyperparameter tuning done on training or validation data set? The post here gives mixed opinion as of whether the training set should be used for hyperparameter tuning. And I would like to know ...
- 411
0
votes
0
answers
208
views
Keras Tuner Error: Failed to find data adapter that can handle input: <class 'function'>, <class 'NoneType'>
I am working on a project where I have a dataset of emails with information such as subject, body, attachment names, etc. My goal is to classify/predict around ~5,000 different sites. I am using Keras ...
- 101
0
votes
0
answers
34
views
0
votes
0
answers
47
views
Walk forward cross-validation with Optuna and deepar in pytorch forecasting
I want to perform 3 splits walk forward cross validation with expanding training set for the deepar model from the pytorch forecasting framework. When I do walk forward validation, I also want to do ...
- 101
0
votes
1
answer
26
views
Flow of machine learning model including code
I'm towards the completion of my first data science project that will go into my GitHub portfolio.
I'll be happy for some clarification regarding the machine learning models section:
I got a little ...
- 15
0
votes
0
answers
31
views
Is this a valid cross validation approach to choose hyperparameters and get a good estimate of model performace?
I am using lightGBM on time series data. I first split my data set into 10% folds. The last fold is used as a test set.
For each choice of hyperparameters I first use 6 folds to train, then predict on ...
0
votes
0
answers
43
views
how does Optuna execute the objective function?
how does Optuna execute the objective function when calling study.optimize(objective, n_trials=10,
is it, for each trial ===> call the function ??, if this is the case, this means we shouldn't ...
- 101
0
votes
1
answer
82
views
Is it mandatory to set a random_state when using RandomizedSearchCV?
When I use RandomizedSearchCV, if I put the random state I always obtain the same results with the same hyperparams trainer. So, is it mandatory to use? Because in my opinion it is better to always ...
0
votes
0
answers
38
views
Is it possible to perform probability calibration with a model with the best hyperparams?
If I use RandomizedSearchCV to find the optimal hyperparams of a model, can I create another model, with those parameters, to calibrate probabilities using CalibratedClassifierCV?
The new model is not ...
0
votes
0
answers
31
views
Matlab - Machine Learning: Hyperparameter tuning of a fitcecoc - model + training it on new data
I have trained a logistic regression multi-class model in Matlab for multi-class classification using XTrainSet_A / YTrainSet_A, look at this simplified code:
...
- 1
0
votes
0
answers
36
views
how to define search space over hyperparameters automatically?
I'm trying to automate retrain steps of our ML models. My aim is hyper-parameter tuning with current data (newer performance window) using same algorithm and features on production environment. ...
- 1
1
vote
0
answers
167
views
Optuina pruning during CrossValidation, does it make sense?
I'm currently trying to build a model using CatBoost. For my parameter tuning, I'm using optuna and cross-validation and pruning the trial checking on the intermediate cross-validation scores. Here ...
1
vote
1
answer
78
views
Can I fit a model with the parameters found with RandomizedSearchCV?
I want to ask you a question.
Suppose I use the following RandomizedSearchCV to find the model's best hyperparams:
...
0
votes
0
answers
17
views
Choosing which parameters and what range to hyperparameter tune in a model?
Lets say I built an XGBoost model with XGBClassifier().
I know I can get default params by calling the default parameters function.
However, there are so many ...
- 111
0
votes
0
answers
23
views
Rules/Guidelines for Custom Weightage and Hyper-parameter tuning
I have a movie and user-ratings dataset. After implementing the content-based filtering technique, I figured, I can improvise the results even further by assigning weightage to the parameters based on ...
- 111
0
votes
0
answers
12
views
Why does cross validation and hyperparameter tuning work?
To my understanding, optimizing a model with k-fold cross validation and hyperparameter tuning are tools to be used mostly for small datasets to really make the most out of very limited/expensive data,...
- 1
0
votes
0
answers
9
views
Optimize wordembedding and neural network at the same time
I have a lot of (domain)-specific text that I want to classify into 100+ categories.
I want to train a wordembedding (FastText) and use that in conjuction with a CNN, thus I'm running into the problem ...
- 460
0
votes
1
answer
39
views
Question about grid search and KFold
I am trying an example which I am training on a huge dataset 5M (only 4 features) rows with Cudf and CUml and I am using SGD logistic regression because I must predict if the patient if is sick or not ...
- 1
1
vote
1
answer
66
views
Which of 2 options is better practice for model optimization: 1) Nested CV wrongly averaging inner CV scores. 2) Two successive CVs on X_all. Altrntv?
Goal: Compare preprocessing methods, models, and hyperparameters without leaking into the final generalization estimate, applying cross-validation (cv), i.e. NOT applying any fixed train/test splits.
...
- 11
0
votes
0
answers
64
views
Objective function in optuna seems to return at random points
I am trying to use a K-fold cross-validation within the optuna objective function.
Unfortunately, the output of the objective function seems to pop out at random places, not at the end of the cross-...
0
votes
1
answer
209
views
Tuned model has higher CV accuracy, but a lower test accuracy. Should I use the tuned or untuned model?
I am working on a classification problem using Sci Kit Learn and am confused on how to properly tune hyper parameters to get the "best" model.
Before any tuning, my logistic regression ...
0
votes
1
answer
24
views
Train/val/test approach for hyperparameter tuning
When looking to train a model, does it make sense to have a 60-20-20 train val test split, first hyper parameter tuning over the training dataset, using the validation set, picking the best model. ...
- 91
1
vote
1
answer
228
views
How to determine which combinations of parameters to include in GridSearchCV
I am using MLPClassifier from sklearn and I would like to tune it with GridSearchCV. But I don't know which set of values to include for hidden_layer_sizes, max_iter, activation, solver, etc. How can ...
- 11