Questions tagged [hyperparameter-tuning]

The tag has no usage guidance.

Filter by
Sorted by
Tagged with
1
vote
0answers
11 views

How can I use a validation set to tune the hyperparameters of an XGBClassifier?

I'm currently building a ranking model using an XGBClassifier. I have training, testing, and validation sets. I want to use the validation set to tune the hyperparameters of the XGBClassifier before ...
0
votes
1answer
21 views

Does GridSearchCV not save the best parameters?

So I tuned the hyperparameters using GridSearchCV, fitted the model to the data, and then used best_params_. I'm just curious ...
1
vote
1answer
30 views

small dataset CV

I have a very small dataset ( 150 records) with 20 features, trying to predict a binary outcome. Due to the small size, i chose to do 10 CV instead of train/test as the train/test split. I was ...
1
vote
1answer
23 views

Metric to use to choose between different models - Hyperparameters tuning [closed]

I'm building a Feedforward Neural Network with Pytorch and doing hyperparameters tuning using Ray Tune. I have train, validation and test set, train and validation used during the training procedure. ...
0
votes
1answer
14 views

With automated hyperparameter tuning available, do we still need to learn hyperparameter tuning [closed]

Tools like AWS Sagemaker have capability to do automated hyperparameter tuning, even with complex algos like Neural Networks using Tensorflow. So do we still need to learn how to do hyperparameter ...
1
vote
1answer
29 views

how to prevent machine crash while searching for hyper parameters of XGBoost with GridSearchCV

I am searching for best hyper parameters of XGBRegressor using GridSearchCV. Here is the code: ...
1
vote
0answers
31 views

Triplet Loss - Choice of margin parameter

The triplet loss function, which is typically employed in triplet network architectures, was first introduced in Schultz, M.; Joachims, T. (2004). "Learning a distance metric from relative ...
0
votes
2answers
30 views

Hyperparameter tuning XGBoost

I'm trying to tune hyperparameters with bayesian optimization. It is a regression problem with the objective function: objective = 'reg:squaredlogerror' $\frac{1}{2}[log(pred+1)-log(true+1)]^2$ My ...
1
vote
1answer
19 views

How to quantify ‘compute cost’ of training of xgboost model?

I want to quantify compute cost of hyper-parameter search for xgboost model. One way can be to measure training time with one particular hyper parameter configuration chosen for training and use it as ...
4
votes
0answers
24 views

Exploring variables to guide xgboost tuning

In short: How to think about the type and distribution of my variables when choosing parameter values for xgboost? Context: I have a dataset which I want to classify using the ...
0
votes
1answer
46 views

Hyperparameter Tuning in Random Forest Model

I'm new to the machine learning field, and I'm learning ML models by practice, and I'm facing an issue while using the machine learning model. While I'm implementing the ...
0
votes
0answers
22 views

Hyper tuning reduce the accuracy score, why?

I have performed hyper tuning grid CV search on KNN model. The actual accuracy score for my KNN was accuracy of 42.31 % without performing hyper tuning. However, after performing hyper tuning, the ...
0
votes
2answers
29 views

Regularization hyperparam tuning during training

I have an idea for a regularization-hyperparam selection method, which I haven't encountered before and can't find on Google, but I'm sure someone has already tried it and I'm wondering what are the ...
1
vote
0answers
17 views

Initial value space for Random Forest hyperparameter tuning

I'm building a Random Forest Classifier using Scikit Learn. My problem consists in a 4 class classification task, the values are distributed as follows (after splitting my data in training set and ...
2
votes
2answers
35 views

How to treat data transformation choices as hyperparemeters?

While reading the book hands-on ML by Aurelien Geron, I came across this line- Treat your data transformation choices as hyperparameters, especially when you are not sure about them (e.g., if you’re ...
0
votes
0answers
45 views

SVM is taking too long for hyperparameter tuning

I am running SVM,Logistic Rregression and Random Forest on the credit card dataset. My training dataset has the shape (454491, 30). I performed 5-fold cross validation(which took more than an hour) ...
1
vote
1answer
32 views

How do you identify whether your RMSE score is good or not?

Im building a XGBoost regression model to predict the values in the range of -3 to 3. Im using Root Mean Squared Error to evaluate the model. With hyper-parameter tuning and everything the best scores ...
-1
votes
1answer
31 views

How to identify Overfitting in RandomForestClassifier?

Im building a sentiment classification model using RandomForestClassifier. I got the training accuracy of 99.65 & cross-validation( RepeatedStratifiedKFold-5 folds) accuracy of 97.29. I used f1 ...
1
vote
2answers
62 views

Is a test set necessary after cross validation on training set?

I'd like to cite a paragraph from the book Hands On Machine Learning with Scikit Learn and TensorFlow by Aurelien Geron regarding evaluating on a final test set after hyperparameter tuning on the ...
2
votes
1answer
61 views

Which is better: Cross validation or a validation set for hyperparameter optimization?

For hyperparameter optimization I see two approaches: Splitting the dataset into train, validation and test, and optimize the hyperparameters based on the results of training on the train dataset and ...
0
votes
0answers
16 views

How do you calculate the probability that a certain number of hyper-parameter combinations contains the optimum combination?

How do you calculate the probability that a certain number of hyper-parameter combinations contains the optimum combination? Side Note: After doing more research, I have decided that Randomized Search ...
2
votes
2answers
57 views

Faster grid search with small dataset to derive best params instead of full dataset?

I have a dataset of 300 000 rows and an ensemble model, which include grid search to find the best params of every algorithm. Unfortunately the grid search needs to long and I have problems to ...
1
vote
0answers
24 views

Adaptive Resampling in Caret with Pre-specified Validation Set

I was wondering if this is the correct way to get adaptive sampling in caret working with a pre-specified validation set using index. I can get this to work using the 'cv' method in caret like so <...
1
vote
0answers
30 views

How can I use the Brier Skill Score in cross-validation for imbalanced data?

I wish to use search for model hyper-parameters (by a grid search) using the Brier score as the scoring method (see code below): ...
0
votes
0answers
41 views

Good mathematically explained algorithm for Hyperparameter Optimization (Bayesian) for implementing in Java

I have implemented Random Forest, Bagging, Gradient Boosting, etc... in java myself. It took a long time to complete these machine learning algorithm codes. But at last all of them are well running. ...
0
votes
1answer
61 views

MLP classifier Gridsearch CV parameters to tune?

I'm looking to tune the parameters for sklearn's MLP classifier but don't know which to tune/how many options to give them? Example is learning rate. should i give it[.0001,.001,.01,.1,.2,.3]? or is ...
0
votes
1answer
44 views

Final Model fitting - subset vs entire training data

If I used a subset of the entire available training data for model tuning and hyperparamater selection, should I fit the final model to the subset training dataset or the entire available training ...
1
vote
1answer
116 views

XGboost and regularization

Does the XGBClassifier method utilizes the two regularization terms reg_alpha and reg_lambda,...
0
votes
0answers
37 views

How to grid search class.weights hyper parameter in Ranger?

I am currently using ranger for binary classification. My dataset is highly imbalanced (10:1). I went over the documentation, and it appeared to me that ...
0
votes
1answer
19 views

What does mtry < 1 in TuneRF mean?

I'm running a random forest model using the randomForest package in R, using the TuneRF function. I have the option to set the 'step factor', which is how much the mtry parameter is changed at each ...
1
vote
1answer
180 views

How to tune learning rate with HParams Dashboard on Tensorflow?

In Tensorflow documentation, it is shown how to tune several hyperparameters but not the learning rate.I have searched how to tune learning rate using HParams dashboard but could not find much. The ...
1
vote
1answer
14 views

Tuning SVM C parameter

I would like to ask for help regarding my model. I have a dataset of preprocessed images and I performed a binary classification with SVM on Python. I tuned the value of the c parameter from 0.001 to ...
4
votes
2answers
92 views

Robustness of hyperparameter tuning

I use a Bayesian hyperparameter (HP) optimization approach (BOHB) to tune a deep learning model. However, the resulting model is not robust when repeatedly applied to the same data. I know, I could ...
1
vote
0answers
101 views

Hyperparameter tuning one-class svm

I have a problem where I am trying to apply a one-class svm to detect outliers. I am training on a dataset of true cases using a one-class radial svm and then predicting for both false and true cases. ...
0
votes
2answers
46 views

Which hyperparameters of a neural network can be tunned independently?

The hyperparameter search is computationally expensive. I am wondering if one can tune the hyperparameters independently: tune one hyperparameter for a fixed value of other hyperparameters. For ...
0
votes
1answer
42 views

How to improve a CNN without changing the architecture?

I'm currently using an autoencoder CNN that's built upon the VGG-16 architecture that was designed by someone else. I want to replicate their results using their dataset first but I'm finding that: -...
0
votes
0answers
11 views

Can we use both ridge-lasso and PCA in the same model for better results?

My question here is if we are using the PCA, the dimensionality is reduced and no question of feature selection is required using ridge & lasso. So should I use ride-lasso followed by PCA or I ...
0
votes
1answer
66 views

Hill Climbing Algorithm - Optimum Step Size

I am implementing a standard hill climbing algorithm to optimise hyper-parameters for a predictive model. The hill climbing algorithm is being applied as part of a two-stage approach: Apply grid ...
1
vote
2answers
538 views

Opinions on an LSTM hyper-parameter tuning process I am using

I am training an LSTM to predict a price chart. I am using Bayesian optimization to speed things slightly since I have a large number of hyperparameters and only my CPU as a resource. Making 100 ...
0
votes
1answer
148 views

Hyperparameter tuning does not improve accuracy?

I am working on titanic dataset, I achieved 92% accuracy using random forest. However, the accuracy score dropped to 89% after I tuned it using Gridsearch. Now, I ...
1
vote
2answers
4k views

Hyper-parameter tuning of NaiveBayes Classier

I'm fairly new to machine learning and I'm aware of the concept of hyper-parameters tuning of classifiers, and I've come across a couple of examples of this technique. However, I'm trying to use ...
0
votes
0answers
70 views

Am I doing bayesian optimization correctly for MLP?

I am trying to optimize the below mentioned hyper-parameters of MLP with range as follows. Number of hidden layers (n): 1-10 Number of perceptrons (p): 25, 50, 75, ..., 200 Activation function: ...
0
votes
1answer
44 views

What is the objective that is optimized with Random Search?

I have recently learned about Random Search (or sklearn.model_selection.RandomizedSearchCV in Python) and was thinking about the theory behind the optimization process. In particular my question is, ...
1
vote
0answers
50 views

Hyperparameter tuning results yield no improvement over spot-check

There is a balanced binary classified dataset as seen below. Things I have tried using RandomForestClassifier as chosen model: TimeSeriesSplit with n_splits=3 and ...
0
votes
1answer
20 views

How to choose the best hyper-parameter when it is directly influenced by the random_state?

While trying to evaluate my Ridge Regression model and using GridSearchCV to find the best parameter. I noticed that the best estimator changes every time I change my random_state. With this in mind ...
0
votes
0answers
11 views

Negative correlation between OOB statistics and test set statistics during tuning of a RandomForest

I am tuning the parameters of a binary random forest classifier using a random search with a priority queue for training. After training with a fixed number of estimators (3000), the strategy is to ...
1
vote
1answer
23 views

Two questions on hyper-parameter tuning

Question 1: In the example of logistic regression, I often see the regularization constant and penalty methods being tuned by a grid search. However, it seems like there are a lot more options for ...
0
votes
1answer
32 views

Hyperparameter tuning of neural networks using Bayesian Optimization

One of the assumptions for finding good hyperparameters using Bayesian optimization (GP) is that the unknown function is smooth. Is this assumption valid for neural networks or at least for most of ...
0
votes
0answers
8 views

what does “Tree” refer to in Tree-structured Parzen Estimators

I am going through the literature of Hyperparameter optimization techniques and came across TPE. There is very little to no explanation on why the name has "Tree" in it. What is Tree referring to? and ...
1
vote
2answers
92 views

Is the search space of Hyperparameters Continuous or Discrete?

I am looking into hyper-parameter tunning and was curious about whether the search space is considered continuous or discrete? My understanding of both those cases: 1. Continuous would make it 'easier'...