Questions tagged [hyperparameter]

Hyperparameters of a model are the kind of parameters that cannot be directly learned during training but are set beforehand. Hyperparameters can define, for example, the complexity of the model or its capacity to learn.

39 questions with no upvoted or accepted answers
Filter by
Sorted by
Tagged with
3
votes
1answer
199 views

Benefits of using Deep Learning-specific hyperparameter optimization tools vs. sklearn?

There are quite a few library for hyperparameter optimization that are specific to Keras or other Deep Learning libraries, like Hyperas or Talos. My question is, what's the main benefit of using ...
2
votes
2answers
1k views

How can I tune LSTM hyperparameters?

If anyone is there to answer these, that'll be great. I'm in the midst of a Final Year Project on LSTM. Currently, I’m stuck and confused over LSTM codes. There are 4 hyperparameters that I can play ...
2
votes
0answers
30 views

Parameter optimization and selection in dynamic neural networks

I have used a Bayesian optimization to tune machine learning parameters. The optimized parameters are "Hidden layer size" and "...
2
votes
0answers
267 views

Error while trying to do hyperparameter tuning using hyperas

I am getting a syntax error while using hyperas and am not sure why. My code: ...
2
votes
0answers
384 views

How to tune parameters for Time Series Analysis, when forecasting is only dominated by one feature and error is not getting reduced?

I am trying to predict time series based on 150 features. When I plot correlation of these features, I am getting 20 features with more or less importance but every model I use, it is completely ...
2
votes
2answers
9k views

Hyper-parameter tuning of NaiveBayes Classier

I'm fairly new to machine learning and I'm aware of the concept of hyper-parameters tuning of classifiers, and I've come across a couple of examples of this technique. However, I'm trying to use ...
1
vote
0answers
11 views

Can I dynamically change the hyper-parameters of a model?

Question Can I apply different hyper-parameters for different training sets? I can see the point of using the shared parameters but I cannot see the point of using shared hyper-parameters. The ...
1
vote
0answers
18 views

How to compare hyperparameter tuning in R and Python

I tried random forest in both R (Caret) and Python (Scikit-learn), but the results differ drastically. Pearson correlation between predicted value and actual value was 0.2 in python whereas 0.8 in R. ...
1
vote
0answers
46 views

XGBoost: Typical gamma and min_child_weight range

What is the typical accepted range of gamma and min_child_weight parameters for the XGBoost algorithm? Is the range of min_child_weight correlated with the number of feature or samples in the training ...
1
vote
0answers
68 views

Adaptive Resampling in Caret with Pre-specified Validation Set

I was wondering if this is the correct way to get adaptive sampling in caret working with a pre-specified validation set using index. I can get this to work using the 'cv' method in caret like so <...
1
vote
1answer
31 views

Two questions on hyper-parameter tuning

Question 1: In the example of logistic regression, I often see the regularization constant and penalty methods being tuned by a grid search. However, it seems like there are a lot more options for ...
1
vote
0answers
24 views

Classification Model showing different accuracy for SAME data?

This is my first post here, so kindly pardon any commonplace errors. So, i have been training an XGBoost multi-class model on Google Colab. I am using a balanced dataset, with 31000 rows, where each ...
1
vote
0answers
18 views

Techniques for hyperparameter search in non-stationary environments

I'm tuning a supervised machine learning model over time, also called incremental learning. I do not want to assume the environment is non-stationary. Grid search and random search do not appear to ...
1
vote
0answers
133 views

Validation curve/RandomizedSearchCV difference train and test score

Ive build a RF model for an imbalanced data set that after feature selection has an F1 score of 54.26%. I am now trying to do hyper parameter tuning using RandomizedSearchCV, after creating validation ...
1
vote
0answers
63 views

SVM C vs gamma hyperparameter tuning

While running SVC(), how we can hyperparameter tune C vs gamma combination? I could see changes in C and gamma are impacting the accuracy differently. Also, what I understand about C and gamma are: C ...
1
vote
0answers
120 views

Why do BERT classification do worse with longer sequence length?

I've been experimenting using transformer networks like BERT for some simple classification tasks. My tasks are binary assignment, the datasets are relatively balanced, and the corpus are abstracts ...
1
vote
0answers
20 views

Analyzing the search space of hyperparameter optimization

My goal is to train a CNN via transfer learning on a given dataset and to analyze and document the training process. I selected a few CNN architectures and hyperparameters to perform a random search. ...
1
vote
0answers
22 views

Cross-validation for Timeseries Counterfactual Analysis

We are looking to predict counterfactual states from time-series data. In our problem we are looking to determine the energy savings from a grid-installed device that is varied on and off for many ...
1
vote
0answers
18 views

Slowly decreasing validation / training cost and their abnormal values

I have a dataset of size ~100,000 of images, I'm training a CNN model on them for regression. optimizer: Adam batch_size: 64 Number of epochs: 50 When I set the ...
1
vote
0answers
1k views

Bayesian optimization for a Light GBM Model

I am able to successfully improve the performance of my XGBoost model through Bayesian optimization, but the best I can achieve through Bayesian optimization when using Light GBM (my preferred choice) ...
1
vote
0answers
383 views

SVM hard and soft margins in matlab,

I am comparing the performances of several SVM models in matlab using the fitcsvm function, and I want to double check that I am using the correct syntax for hard ...
1
vote
0answers
61 views

Problem about tuning hyper-parametres

I have tried GridSearchCV and BayesSearchCV for tuning my LightGBM algorithm (for binary classification). I have used 10 iterations and I have indicated scoring ="roc_auc" In the first iteration, I ...
1
vote
0answers
27 views

Any heuristic for minimal DCGAN latent space dimension?

I am highly interested in approaching minimal latent space dimension (as many other may be) for DCGANs or autoencoders. In this example of DCGAN on the MNIST dataset, the person uses a 100-...
1
vote
0answers
110 views

std::bad_alloc with densenet and hyperas

I am filing this issue after being stagnated here for couple of weeks. I am using hyperas to find the hyperparameters for my network, Densenet. My issue here is that my evaluation always fails with ...
1
vote
0answers
368 views

How to perform platt scaling for hyperparameter-optimized model?

I'm using Python and have a best estimator from a grid search. Wanted to be able to calibrate the probability output accordingly, but would like to know more about implementing platt scaling. From ...
1
vote
0answers
199 views

Experimental design - hyperparameter optimization

I'm reading a paper by Bergstra and Bengio (2012) on random search for hyperparameter optimization. I'm confused by their graph and explanation in Section 2.2: "Figure 2 illustrates the results of a ...
1
vote
0answers
235 views

How do I choose the optimal parameters for reliefF

For feature selection I use reliefF provided by matlab. The reliefF function offers a parameter k to influence its ouput, additionaly for my specific task I can also vary a window length l on which ...
1
vote
0answers
181 views

Global vs. local bias-variance tradeoff

In the standard example of decomposing the MSE into Bias, Variance and Irreducible error: $$MSE(x) = \left(\mathbb{E}[\hat{f}(x)] - f(x) \right)^2 + \mathbb{E}\left[\left(\hat{f}(x) - f(x)\right)^2\...
1
vote
0answers
3k views

Why is my loss so high?

I am struggling to understand why i am getting such a high loss/val_loss rate on my training. I am training a regression network. I've normalized the input data to range between -1 to 1, and left the ...
0
votes
0answers
16 views

How to tune the hyperparameters in graphical models such as a Markov random field?

What's a good way to tune the hyperparemter of a Markov random field (MRF) with a Gaussian prior that's not uncommon in image segmentation tasks? The structure of the graphical model is not a big ...
0
votes
0answers
73 views

Hyper tuning reduce the accuracy score, why?

I have performed hyper tuning grid CV search on KNN model. The actual accuracy score for my KNN was accuracy of 42.31 % without performing hyper tuning. However, after performing hyper tuning, the ...
0
votes
0answers
89 views

Good mathematically explained algorithm for Hyperparameter Optimization (Bayesian) for implementing in Java

I have implemented Random Forest, Bagging, Gradient Boosting, etc... in java myself. It took a long time to complete these machine learning algorithm codes. But at last all of them are well running. ...
0
votes
0answers
107 views

How to grid search class.weights hyper parameter in Ranger?

I am currently using ranger for binary classification. My dataset is highly imbalanced (10:1). I went over the documentation, and it appeared to me that ...
0
votes
0answers
45 views

Is copying parameter considered as plagarism?

So my friends and i are writing a kaggle assignment and the base code is written by me. One of my friend use my base code(feature engineering, labeling, etc.) and put it into a loop to find the best ...
0
votes
0answers
126 views

Has anybody used alternative hyperparameter optimization techniques (other than default one) in SK-Learn?

I've been using Sklearn for Gaussian process regression that has L-BFGS-B (“fmin_l_bfgs_b”) as a default optimization algorithm. I want to implement some other ...
0
votes
1answer
47 views

Generative network understanding

I was going through GAN's notebook by fchallot on Generative Adversarial Networks where, in the Generator Network, he creates a Dense layer with $16*16 * 128$ (...
0
votes
2answers
548 views

Random Forest - Explanation Parameter

I got some question about the "standard" parameter from a random forest. Following I write my understanding about these parameters. I would be glad if I could confirm my understanding or correct it. :)...
-1
votes
1answer
184 views

Denoising Autoencoder Parameter Search

I have ran a hyperparameter search for a denoising autoencoder and the results suggest I should make the sizes of my hidden layers as large as possible (within the range of values I allowed it to ...