Questions tagged [hyperparameter-tuning]

The tag has no usage guidance.

Filter by
Sorted by
Tagged with
0
votes
1answer
15 views

Hyperparameter tuning of neural networks using Bayesian Optimization

One of the assumptions for finding good hyperparameters using Bayesian optimization (GP) is that the unknown function is smooth. Is this assumption valid for neural networks or at least for most of ...
0
votes
0answers
6 views

what does “Tree” refer to in Tree-structured Parzen Estimators

I am going through the literature of Hyperparameter optimization techniques and came across TPE. There is very little to no explanation on why the name has "Tree" in it. What is Tree referring to? and ...
1
vote
2answers
31 views

Is the search space of Hyperparameters Continuous or Discrete?

I am looking into hyper-parameter tunning and was curious about whether the search space is considered continuous or discrete? My understanding of both those cases: 1. Continuous would make it 'easier'...
0
votes
0answers
63 views

Since is not possible test all the possible combination, what is the correct procedure to follow on building Machine Learning?

Sorry, I'm a little confused and this is a general question. How can I be sure that the procedure that I am following is the correct one? Following the steps for building a machine learning model, we ...
1
vote
1answer
23 views

Asynchronous Hyperparameter Optimization - Dependency between iterations

When using Asynchronous Hyperparameter Optimization packages such as scikit optimize or hyperopt with cross validation (e.g., cv = 2 or 4) and setting the number of iteration to N (e.g., N=100), ...
3
votes
2answers
55 views

Shuffle the data before splitting into folds

I am running a 4-folds cross validation hyperparameter tuning using sklearn's 'cross_validate' and 'KFold' functions. Assuming that my training dataset is already shuffled, then should I for each ...
1
vote
0answers
10 views

Techniques for hyperparameter search in non-stationary environments

I'm tuning a supervised machine learning model over time, also called incremental learning. I do not want to assume the environment is non-stationary. Grid search and random search do not appear to ...
3
votes
1answer
59 views

Small number of estimators in gradient boosting

I am tuning a regression gradient boosting-based model to determine the appropriate hyperparameters using 4-folds cross validation. More specifically, I am using XGBoost and lightGBM for the models ...
1
vote
1answer
30 views

order of features for model tuning vs model fitting

Assuming that the same columns (i.e., features) are used for hyperparameter tuning and model fitting, and ensemble models are used for modeling (e.g., Random forest or XGboost), then does the order of ...
4
votes
1answer
152 views

ROC AUC score is much less than average cross validation score

Using Lending club Dataset to find the propability of default. I am using hyperopt library to fine tune hyper parameter for an XGBclassifier and trying to maximize the ROC AUC score. I am also using ...
1
vote
1answer
50 views

Relation between hyperparamters and training set for an object detection model

I have 2 instances of an object detection model. The only difference between these two models is the training data used: The first model was trained with a small training set The second model was ...
1
vote
1answer
74 views

How can I tune LSTM hyperparameters?

If anyone is there to answer these, that'll be great. I'm in the midst of a Final Year Project on LSTM. Currently, I’m stuck and confused over LSTM codes. There are 4 hyperparameters that I can play ...
1
vote
0answers
31 views

Validation curve/RandomizedSearchCV difference train and test score

Ive build a RF model for an imbalanced data set that after feature selection has an F1 score of 54.26%. I am now trying to do hyper parameter tuning using RandomizedSearchCV, after creating validation ...
2
votes
0answers
30 views

How to tune the hyperparameters of XGBoost and RF? [closed]

How to tune the hyperparameters of XGBoost and RF in python? There are several methods to tune hyperparameteres of XGBoost and RF such as Bayesian Optimization and meta learning and gridseachcv? ...
2
votes
0answers
26 views

Parameter optimization and selection in dynamic neural networks

I have used a Bayesian optimization to tune machine learning parameters. The optimized parameters are "Hidden layer size" and "...
2
votes
1answer
78 views

Hyperparameter optimization performance comparison

I have used Bayesian optimization for hyperparameter tuning in a machine learning model. What is the best way to compare the performance of network with and without Bayesian optimization? I found some ...
2
votes
1answer
152 views

Cannot clone object <keras.wrappers.scikit_learn.KerasRegressor object at 0x7fdc9c3ba550>

Trying to hypertune ANN but getting an error while using fit..(grid1.fit(X_train, y_train)) Below is the code ...
1
vote
1answer
22 views

Including the validation file in the training process after tunning

Should I include the validation file in the training process after finishing the tuning process (e.g. searching for params using the validation file)?
1
vote
0answers
43 views

Why do BERT classification do worse with longer sequence length?

I've been experimenting using transformer networks like BERT for some simple classification tasks. My tasks are binary assignment, the datasets are relatively balanced, and the corpus are abstracts ...
1
vote
1answer
27 views

Which kNN model to chose?

I am trying to tune the "n_neighbors" for a kNN model andI have the following problem : Based on the mean cross validation score the optimal kNN model should be the one with 10 neighbors. On the ...
1
vote
1answer
43 views

Which combination of 3 hyperparameters to combat overfitting of a convolutional neural network?

I have a small dataset with which I want to train a CNN by using Data Augmentation. Since the CNN is overfitting due to the small data set, I would like to optimize some hyperparameters. However, ...
2
votes
1answer
141 views

New parameters in final training

I am training an Xgboost using 60% of my data and use 40% for testing. In the 60% of data, I use 5-fold validation to find the best number of trees. I find that the optimal number of trees is around ...
1
vote
1answer
31 views

ML in R (caret-package) missing hyperparameters

I have a pretty specific question regarding the caret package however I still hope to finde help here. I recently worked with the caret package and trained a multilayer perceptron with ...
1
vote
1answer
27 views

Random Forest Model Giving Same Accuracy for different feature sets after tuning

I am having this weird issue and cannot seem to find a solution. I am trying to tune a different random forest model for every different feature-set. Basically from a given data set, I have created 3 ...
1
vote
0answers
32 views

How to avoid bias when optimizing time-series model?

I have 4000 days of data. I am trying create a time-series model with parameters P to forecast the value of the target Y using the last N days of data. The parameters P include: lookback window for ...
0
votes
0answers
49 views

Tuning parameters in Affinity Propagation

I am doing Affinity Propagation clustering and trying to do tuning, but it takes time. A lot of time actually. As I am beginner I do not know how to get clusters. I need cluster numbers from 1 to 20 ...
2
votes
3answers
853 views

How to combine GridSearchCV with Early Stopping?

I'm a beginner in machine learning and want to train a CNN (for image recognition) with optimized hyperparameter like dropout rate, learning rate and number of epochs. The optimal hyperparameter I ...
0
votes
0answers
12 views

Standardizing in each fold - Learning Curve

Problem Description Hello, I have a classification problem and I want to perform cross validation (with hyper parameter tuning) in order to evaluate the generalization of my models. Basically the ...
2
votes
3answers
107 views

GridSearchCV vs RandomSearchCV and How it works?

GridSearchCV vs RandomSearchCV Can somebody explain in-detailed differences between GridSearchCV and RandomSearchCV? And how the algorithms work under the hood? As per my understanding from the ...
0
votes
0answers
8 views

How to proceed after tuning hyperparameters?

As I am still on the journey to understand what when and how to use, I am now at the point how to proceed after finding the best hyperparameters: Define Model (NN) Split Data into ...
0
votes
2answers
62 views

Parameters optimization algorithms in Weka

In Weka, I used the Grid and Random search parameters tuning algorithms but unfortunately, their performance (in terms of better prediction accuracy) is observed worst when we use the ML algorithms (...
1
vote
1answer
113 views

Hyperparameter tuning and cross validation

I have some confusion about proper usage of cross-validation to tune hyperparameters and evaluate estimator performance and generalizeability. As I understand it, this would be the process you ...
0
votes
0answers
20 views

Getting worse results after Hyperparameter Tuning(Grid/Random Search/TPOT)

I have a problem with Hyperparameter Tuning. Usually I getting almost the same results(or worse) than before tuning. Usually default parameters of classificator(regressor) give me a best score. ...
0
votes
1answer
482 views

hyperparameter tuning with validation set

For what I know, and correct me if I am wrong, the use of cross-validation for hyperparameter tuning is not advisable when I have a huge dataset. So, in this case it is better to split the data in ...
1
vote
0answers
18 views

Minimizing overfitting when doing hyperparameter Tuning

Generaly when using Sklearn's GridSearchCV (or RandomizedGridSearchCV), we get best model with best test score even if the model overfits a little bit. How can we compute generalization error ...
1
vote
0answers
155 views

Ensure class balanced batches while hyperparameter tuning keras models with grid search

Ensuring class balanced batches while training keras models is possible using fit_generator method. I used imblearn.keras.BalancedBatchGenerator for that and it works fine! But i wanted to do that ...
2
votes
1answer
339 views

how to pass parameters over sklearn pipeline's stages?

I'm working on a deep neural model for text classification using Keras. To fine tune some hyperparameters i'm using Keras Wrappers for the Scikit-Learn API. So I builded a Sklearn Pipeline for that: <...
0
votes
1answer
38 views

CNN for subsets of a dataset - how to tune hyperparameters

I have a dataset and would like to train CNNs on subsets of different size of the dataset. I already have a CNN, which classifies very well if I use the entire dataset. Now the question arises if I ...
1
vote
0answers
19 views

Analyzing the search space of hyperparameter optimization

My goal is to train a CNN via transfer learning on a given dataset and to analyze and document the training process. I selected a few CNN architectures and hyperparameters to perform a random search. ...
1
vote
1answer
61 views

Track underlying observation when using GridSearchCV and make_scorer

I'm doing a GridSearchCV, and I've defined a custom function (called custom_scorer below) to optimize for. So the setup is like this: ...
1
vote
0answers
13 views

Cross-validation for Timeseries Counterfactual Analysis

We are looking to predict counterfactual states from time-series data. In our problem we are looking to determine the energy savings from a grid-installed device that is varied on and off for many ...
0
votes
1answer
29 views

Difference between a log scale and linear scale : np.random.rand()

How and why will a linear np.random.rand() (to generate a linear scale between 0.0001 and 1) not result in better distributed result but ...
2
votes
0answers
77 views

How to measure the stability of hyperparameter selection in a model-building procedure?

For my project I run several model-building-procedures. I use the mean and standard deviation of the test scores in the outer folds as an estimator for the generalizability of the model-building ...
0
votes
1answer
192 views

Hyperparameter Tuning Time Series in Production

I have a time series data that handled using GDBT to predict the next value. I always use previous 30 days data to train daily, but overtime the data to predict and train is increased because the ...
0
votes
1answer
24 views

Do we need to tune same model differently for different window sizes in time series data classification?

I am currently working on the time series data classification problem using deep learning. As we all know that in time series, we process the time-series data sequentially for some time steps at a ...
1
vote
0answers
89 views

Optimizing parameters for CNN autoencoder based on training and validation loss

I have designed an autoencoder with a encoder and decoder consiting of 2D convolutational layers (the input are 40'000 2D images). I train the autoencoder using adam optimizer. The autoencoders has ...
0
votes
0answers
42 views

Hyperopt Model runs with 0 seconds duration

I use Hyperopt for Random Forest Regression hyperparameter tuning. my parameterspace is : ...
1
vote
0answers
163 views

How to put KerasClassifier, Hyperopt and Sklearn cross-validation together

I am performing a hyperparameter tuning optimization (hyperopt) tasks with sklearn on a Keras models. I am trying to optimize KerasClassifiers using the Sklearn cross-validation, Some code follows: <...
2
votes
0answers
180 views

Error while trying to do hyperparameter tuning using hyperas

I am getting a syntax error while using hyperas and am not sure why. My code: ...
1
vote
1answer
217 views

CNNs - Hyperparameter tuning with different training sizes of the same data set

I would like to compare how much the classification performance (test accuracy) of CNNs changes depending on the size of the data set. For this I would like to use a data set like MNIST or Fashion ...