Questions tagged [hyperparameter]

Hyperparameters of a model are the kind of parameters that cannot be directly learned during training but are set beforehand. Hyperparameters can define, for example, the complexity of the model or its capacity to learn.

Filter by
Sorted by
Tagged with
0
votes
1answer
13 views

Output of multi-fold cross validation

I have been reading a lot about K-fold cross validation. By a way I attended a class project presentation on this topic recently and I still wonder if by the end of this validation method, one has K ...
1
vote
2answers
31 views

Is the search space of Hyperparameters Continuous or Discrete?

I am looking into hyper-parameter tunning and was curious about whether the search space is considered continuous or discrete? My understanding of both those cases: 1. Continuous would make it 'easier'...
0
votes
0answers
18 views

Set params to keras estimator model

I started with tensorflow 2 days ago and I am trying to set the hyperparameters with model_to_estimator. However, I cannot figure out how. As it is, the model can ...
1
vote
1answer
23 views

Asynchronous Hyperparameter Optimization - Dependency between iterations

When using Asynchronous Hyperparameter Optimization packages such as scikit optimize or hyperopt with cross validation (e.g., cv = 2 or 4) and setting the number of iteration to N (e.g., N=100), ...
1
vote
0answers
20 views

Classification Model showing different accuracy for SAME data?

This is my first post here, so kindly pardon any commonplace errors. So, i have been training an XGBoost multi-class model on Google Colab. I am using a balanced dataset, with 31000 rows, where each ...
3
votes
2answers
53 views

Shuffle the data before splitting into folds

I am running a 4-folds cross validation hyperparameter tuning using sklearn's 'cross_validate' and 'KFold' functions. Assuming that my training dataset is already shuffled, then should I for each ...
1
vote
0answers
10 views

Techniques for hyperparameter search in non-stationary environments

I'm tuning a supervised machine learning model over time, also called incremental learning. I do not want to assume the environment is non-stationary. Grid search and random search do not appear to ...
3
votes
1answer
58 views

Small number of estimators in gradient boosting

I am tuning a regression gradient boosting-based model to determine the appropriate hyperparameters using 4-folds cross validation. More specifically, I am using XGBoost and lightGBM for the models ...
4
votes
1answer
152 views

ROC AUC score is much less than average cross validation score

Using Lending club Dataset to find the propability of default. I am using hyperopt library to fine tune hyper parameter for an XGBclassifier and trying to maximize the ROC AUC score. I am also using ...
3
votes
1answer
16 views

Do i need to use hyperparamters from Gridsearch to train on WHOLE training set to get final model?

I just want to make sure i am on the right lines so please correct me if wrong. I am testing which hyperparmets are best for logisitic regession on my data X, y where X is featrues and y is target. X, ...
1
vote
1answer
67 views

How can I tune LSTM hyperparameters?

If anyone is there to answer these, that'll be great. I'm in the midst of a Final Year Project on LSTM. Currently, I’m stuck and confused over LSTM codes. There are 4 hyperparameters that I can play ...
5
votes
1answer
43 views

Search for hyperparameters whith different features using Random Forest

I have a dataset in which I would like to perform a classification model, so I have decided to use Random Forest. The number of features that I have is approximately 200 and I would like to test which ...
1
vote
0answers
29 views

Validation curve/RandomizedSearchCV difference train and test score

Ive build a RF model for an imbalanced data set that after feature selection has an F1 score of 54.26%. I am now trying to do hyper parameter tuning using RandomizedSearchCV, after creating validation ...
2
votes
0answers
26 views

Parameter optimization and selection in dynamic neural networks

I have used a Bayesian optimization to tune machine learning parameters. The optimized parameters are "Hidden layer size" and "...
2
votes
1answer
74 views

Hyperparameter optimization performance comparison

I have used Bayesian optimization for hyperparameter tuning in a machine learning model. What is the best way to compare the performance of network with and without Bayesian optimization? I found some ...
1
vote
0answers
42 views

Why do BERT classification do worse with longer sequence length?

I've been experimenting using transformer networks like BERT for some simple classification tasks. My tasks are binary assignment, the datasets are relatively balanced, and the corpus are abstracts ...
2
votes
2answers
76 views

Why SVM gridsearch takes longer time?

I have a dataset of 5K records and 60 features focussed on binary classification. Please find my code below for SVM paramter tuning. It's running for a longer time than ...
1
vote
0answers
33 views

Is copying parameter considered as plagarism?

So my friends and i are writing a kaggle assignment and the base code is written by me. One of my friend use my base code(feature engineering, labeling, etc.) and put it into a loop to find the best ...
2
votes
2answers
89 views

Hyperparameter optimization, ensembling instead of selecting with CV criteria

While burning CPUs performing a CV selection on a thin grid put on some hyperparameter space. I am using the `scikit-learn' API, for which the end result is a single point on the hyperparameter space, ...
1
vote
1answer
31 views

ML in R (caret-package) missing hyperparameters

I have a pretty specific question regarding the caret package however I still hope to finde help here. I recently worked with the caret package and trained a multilayer perceptron with ...
0
votes
0answers
17 views

Track validation_curve during hyperparameter optimization

To study the influence of a single (hyper-)parameter, I use validation_curve: ...
1
vote
2answers
48 views

is learning rate schedule a hyperparameter? [duplicate]

I believe term ‘learning rate schedule’ is a certain solution for tuning the learning rate. But at the same time, every parameter evaluating the parameter itself can be called a hyperparameter. So can ...
3
votes
1answer
243 views

Grid search or gradient descent?

Assume we have a neural network and one if its activation functions is a function of parameter a. We want to find the weights and parameter a that leads to the minimum loss on the validation set which ...
0
votes
1answer
37 views

Difference between validation and prediction

As a follow-up to Validate via predict() or via fit()? I wonder about the difference between validation and prediction. To keep it simple, I will refer to train, <...
1
vote
1answer
111 views

Hyperparameter tuning and cross validation

I have some confusion about proper usage of cross-validation to tune hyperparameters and evaluate estimator performance and generalizeability. As I understand it, this would be the process you ...
0
votes
1answer
34 views

Similarity of XGBoost models?

Is xgboost with n_estimators = 100 and learning_rate = 0.1, same as xgboost with n_estimators = 50 and learning_rate = 0.2 ?
1
vote
1answer
29 views

Accuracy noise patterns during model training

I'm training a logistic regression model on a small dataset. I have about 1300 samples that I split into a training and a testing set (70% and 30% respectively). The training seems ok, however when I ...
0
votes
1answer
258 views

How to choose the model parameters (RandomizedSearchCV, .GridSearchCV) or manually

Faced with the task of selecting parameters for the lightgbm model, the question accordingly arises, what is the best way to select them? I used the RandomizedSearchCV method, within 10 hours the ...
1
vote
0answers
19 views

Analyzing the search space of hyperparameter optimization

My goal is to train a CNN via transfer learning on a given dataset and to analyze and document the training process. I selected a few CNN architectures and hyperparameters to perform a random search. ...
1
vote
0answers
13 views

Cross-validation for Timeseries Counterfactual Analysis

We are looking to predict counterfactual states from time-series data. In our problem we are looking to determine the energy savings from a grid-installed device that is varied on and off for many ...
0
votes
1answer
48 views

How to optimize hyperparameters in stacked model?

I was wondering whether somebody could explain how to optimize hyperparameters for the base learners and meta algorithm when stacking? In many tutorials they seem to be plucked out of thin air! ...
0
votes
0answers
42 views

Hyperopt Model runs with 0 seconds duration

I use Hyperopt for Random Forest Regression hyperparameter tuning. my parameterspace is : ...
1
vote
2answers
46 views

Parallel hyperparameter optimization techniques?

Most hyperparameter optimization technique want to evaluate points one by one. I have an expensive optimization problem, but i can run hundreds of evaluations in parallel. The dimension of the problem ...
1
vote
1answer
132 views

KNN RandomizedSearchCV typerror

While trying to study a binary classification problem with KNN and trying to tune the parameters of the model I'm getting a typerror that I quite don't understand. Is a parameter missing or something? ...
2
votes
0answers
180 views

Error while trying to do hyperparameter tuning using hyperas

I am getting a syntax error while using hyperas and am not sure why. My code: ...
1
vote
0answers
17 views

Slowly decreasing validation / training cost and their abnormal values

I have a dataset of size ~100,000 of images, I'm training a CNN model on them for regression. optimizer: Adam batch_size: 64 Number of epochs: 50 When I set the ...
0
votes
0answers
48 views

Confused about hyper-parameter search and network architecture search in NAS

I want to understand automatic Neural Architecture Search (NAS). I read already multiple papers, but I cannot figure out what the actual search space of NAS is / how are classical hyper-parameters ...
1
vote
1answer
15 views

A doubt about the GridSearchCV function in Sklearn?

When creating different hyperparameter combinations, does the function evaluate combination 1 on the same fold as combination 2? As in, are the folds the same across combinations? I understand that ...
1
vote
2answers
90 views

Is it a good idea to tune the number of folds for cross validation when tuning hyperparameters of RF

I'm new to data science. I'm trying to get the best model for Random Forest. Unfortunately, I'm not sure if my idea can produce a good generalized model. 1) I have split data to TrainingSet (70%) and ...
0
votes
0answers
61 views

Has anybody used alternative hyperparameter optimization techniques (other than default one) in SK-Learn?

I've been using Sklearn for Gaussian process regression that has L-BFGS-B (“fmin_l_bfgs_b”) as a default optimization algorithm. I want to implement some other ...
4
votes
1answer
164 views

Hyperopt vs Default Values

When I use the hyperopt library to tune my Random Forest classifier, I get the following results: Hyperopt estimated optimum {'max_depth': 10.0, 'n_estimators': 300.0} However, when I train the ...
0
votes
1answer
39 views

Generative network understanding

I was going through GAN's notebook by fchallot on Generative Adversarial Networks where, in the Generator Network, he creates a Dense layer with $16*16 * 128$ (...
2
votes
1answer
148 views

How to handle the parameter space of neural networks?

This question is very broad (and might even be closed as "too broad"). It can be considered as a beginners question, because it is largely about getting started in terms of heading into a direction ...
1
vote
0answers
623 views

Bayesian optimization for a Light GBM Model

I am able to successfully improve the performance of my XGBoost model through Bayesian optimization, but the best I can achieve through Bayesian optimization when using Light GBM (my preferred choice) ...
0
votes
0answers
47 views

Reasonable hyperparameters for NuSVR?

I'm looking for reasonable hyperparameters grid for NuSVR. For now I have: ...
1
vote
1answer
126 views

Tuning C hyper parameter in Soft Margin SVM in Matlab

How to tune the C 'BoxConstraint' hyperparameter in soft margin SVM to get the best optimal value?
1
vote
0answers
189 views

SVM hard and soft margins in matlab,

I am comparing the performances of several SVM models in matlab using the fitcsvm function, and I want to double check that I am using the correct syntax for hard ...
0
votes
0answers
192 views

SVM hyperparameters using Matlab's fitcsvm and OptimizeHyperparameters

I am building SVM models and will compare their performances, linear vs RBF, and I'm using OptimizeHyperparameters to get best hyperparameters C (BoxConstraints) ...
7
votes
1answer
283 views

Is it OK to try to find the best PCA k parameter as we do with other hyperparameters?

Principal Component Analysis (PCA) is used to reduce n-dimensional data to k-dimensional data to speed things up in machine learning. After PCA is applied, one can check how much of the variance of ...
2
votes
2answers
357 views

Setting best SVM hyper parameters

I have a non linear data set, and I am using SVM (RBF kernel) to build a classification model, but not sure how to set the best hyperparameters of the SVM, C and gamma in Matlab ...