Stack Exchange Network

Stack Exchange network consists of 175 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [hyperparameter]

Hyperparameters of a model are the kind of parameters that cannot be directly learned during training but are set beforehand. Hyperparameters can define, for example, the complexity of the model or its capacity to learn.

1
vote
1answer
30 views

Tuning C hyper parameter in Soft Margin SVM in Matlab

How to tune the C 'BoxConstraint' hyperparameter in soft margin SVM to get the best optimal value?
1
vote
0answers
24 views

SVM hard and soft margins in matlab,

I am comparing the performances of several SVM models in matlab using the fitcsvm function, and I want to double check that I am using the correct syntax for hard ...
0
votes
0answers
28 views

SVM hyperparameters using Matlab's fitcsvm and OptimizeHyperparameters

I am building SVM models and will compare their performances, linear vs RBF, and I'm using OptimizeHyperparameters to get best hyperparameters C (BoxConstraints) ...
7
votes
1answer
76 views

Is it OK to try to find the best PCA k parameter as we do with other hyperparameters?

Principal Component Analysis (PCA) is used to reduce n-dimensional data to k-dimensional data to speed things up in machine learning. After PCA is applied, one can check how much of the variance of ...
2
votes
2answers
62 views

Setting best SVM hyper parameters

I have a non linear data set, and I am using SVM (RBF kernel) to build a classification model, but not sure how to set the best hyperparameters of the SVM, C and gamma in Matlab ...
0
votes
1answer
16 views

What is the possible range of SVR parameters range?

I'm working on a regression problem. While tunning the Parameters of SVR I got the following values c=100, gamma= 10 and epsilon =100. For which I got 95 percent r-square. My question is what is the ...
1
vote
0answers
40 views

Problem about tuning hyper-parametres

I have tried GridSearchCV and BayesSearchCV for tuning my LightGBM algorithm (for binary classification). I have used 10 iterations and I have indicated scoring ="roc_auc" In the first iteration, I ...
6
votes
2answers
84 views

What is the most efficient method for hyperparameter optimization in scikit-learn?

An overview of the hyperparameter optimization process in scikit-learn is here. Exhaustive grid search will find the optimal set of hyperparameters for a model. The downside is that exhaustive grid ...
3
votes
3answers
56 views

How to set hyperparameters in SVM classification

I am studying image classification using SVMs and it is generally defined as so... N = number of training examples W = is the weights f(x, W) = dot product λ is explained to be set through cross-...
1
vote
0answers
25 views

xgboost or lightgbm to handle Binomial problems

I have a dataset containing a column of trials, a column of successes and other features; and, obviously, I can generate a probability column. I would like to use gradient boosting methods (like ...
1
vote
0answers
13 views

Any heuristic for minimal DCGAN latent space dimension?

I am highly interested in approaching minimal latent space dimension (as many other may be) for DCGANs or autoencoders. In this example of DCGAN on the MNIST dataset, the person uses a 100-...
0
votes
0answers
20 views

How to adjust deep learning parameters using Particle swarm optimization (PSO)?

As success of deep learning depends upon appropriately setting of its parameters to achieve high-quality results. The number of hidden layers and the number of neurons in each layer of a deep machine ...
0
votes
1answer
27 views

hypeparameters tuning neural network according to loss vs according to scoring function

During hyperparameters tuning we select a metric to measure performance of the model. Example of metrics : f1 score, precision, recall, AUC ... In general, for the training of neural networks, back-...
0
votes
0answers
10 views

Hyper-parameter tuning when you don't have an access to the test data

I'm building models for SQUAD (Stanford Question Answering) dataset (https://rajpurkar.github.io/SQuAD-explorer). Stanford doesn't release its test set. It only provides us with training and dev ...
5
votes
4answers
205 views

Which is first ? Tuning the parameters or selecting the model

I've been reading about how we split our data into 3 parts; generally, we use the validation set to help us tune the parameters and the test set to have an unbiased estimate on how well does our model ...
3
votes
1answer
213 views

Hyperparameter tuning for stacked models

I'm reading the following kaggle post for learning how to incorporate model stacking http://blog.kaggle.com/2016/12/27/a-kagglers-guide-to-model-stacking-in-practice/ in ML models. The structure ...
2
votes
0answers
78 views

How to tune parameters for Time Series Analysis, when forecasting is only dominated by one feature and error is not getting reduced?

I am trying to predict time series based on 150 features. When I plot correlation of these features, I am getting 20 features with more or less importance but every model I use, it is completely ...
1
vote
0answers
13 views

How to optimize the separation of two distributions from binary classfication

Given a sample where for each individual a classification is predetermined (e.g. sick or not) and 5 random variables are measured. The random variables are on the same scale but from differnt bins. E....
1
vote
0answers
29 views

std::bad_alloc with densenet and hyperas

I am filing this issue after being stagnated here for couple of weeks. I am using hyperas to find the hyperparameters for my network, Densenet. My issue here is that my evaluation always fails with ...
0
votes
1answer
36 views

Denoising Autoencoder Parameter Search

I have ran a hyperparameter search for a denoising autoencoder and the results suggest I should make the sizes of my hidden layers as large as possible (within the range of values I allowed it to ...
1
vote
1answer
72 views

What is difference between final episodes of training and test in DQN?

What is difference between running in final episode of training mode and running in test mode in DQN? Is there any difference more than after training and tune the hyper-parameters, we test for one ...
3
votes
1answer
58 views

How to make it possible for a neural network to tune its own hyper parameters?

I am curious about what would happen to hyperparameters when they would be set by a neural network itself or by creating a neural network that encapsulates and influences the hyperparameters of the ...
1
vote
0answers
108 views

How to perform platt scaling for hyperparameter-optimized model?

I'm using Python and have a best estimator from a grid search. Wanted to be able to calibrate the probability output accordingly, but would like to know more about implementing platt scaling. From ...
3
votes
2answers
573 views

How to choose the random seed?

I understand this question can be strang, but how do I pick the final random_seed for my classifier? Below is an example code. It uses the ...
1
vote
1answer
127 views

Benefits of using Deep Learning-specific hyperparameter optimization tools vs. sklearn?

There are quite a few library for hyperparameter optimization that are specific to Keras or other Deep Learning libraries, like Hyperas or Talos. My question is, what's the main benefit of using ...
2
votes
4answers
220 views

Is it better to optimize hyperparameters or run multiple epochs?

Whenever I train a neural network I only have it go through a few epochs ( 1 to 3). This is because I am training them on a bad CPU and it would take some time to have the neural network go though ...
0
votes
0answers
26 views

Difference between MOE and Spearmint?

MOE seems to be a plain Bayesian optimization. Just curious if anyone knows about the difference.
0
votes
1answer
86 views

Random Forest - Explanation Parameter

I got some question about the "standard" parameter from a random forest. Following I write my understanding about these parameters. I would be glad if I could confirm my understanding or correct it. :)...
3
votes
1answer
87 views

Hyper parameters and ValidationSet

Please correct me if I am wrong. "Training Set is used for calculating parameters of a machine learning model, Validation data is used for calculating hyperparameters of the same model (we use same ...
4
votes
1answer
392 views

What does the “dual” parameter in sklearn.svm.LinearSVC and sklearn.svm.LinearSVR do?

While I am more or less familiar with the idea of the SVM, I do not understand the meaning of the dual parameter, which is described in the documentation as: ...
1
vote
2answers
116 views

How to tune bandwidth in machine learning kernel model?

Gaussian kernel $k(x,y) = \exp(-\lVert x-y \rVert^2/\sigma^2)$ has a hyperparameter $\sigma$. I know grid search cross validation, but this would require a lot of computation since computational ...
1
vote
2answers
5k views

Which parameters are hyper parameters in a linear regression?

Can the number of features used in a linear regression be regarded as a hyperparameter? Perhaps the choice of features?
0
votes
1answer
342 views

Is there any alternative to L-BFGS-B algorithm for hyperparameter optimization in Scikit learn?

The Gaussian process regression can be computed in scikit learn using an object of class GaussianProcessRegressor as: ...
1
vote
1answer
694 views

Optimising Kernel parameters using training data in GaussianProcessRegressor of Scikit-learn

I want to optimize the Kernel parameters or hyper-parameters using my training data in GaussianProcessRegressor of Scikit-learn.Following is my query: My training datasets are: X: 2-D Cartesian ...
4
votes
1answer
213 views

Why do we need the hyperparameters beta and alpha in LDA?

I'm trying to understand the technical part of Latent Dirichlet Allocation (LDA), but I have a few questions on my mind: First: Why do we need to add alpha and gamma every time we sample the equation ...
2
votes
2answers
168 views

Do we need to increase training data size when increasing dropouts?

I am using a fully connected feed forward neural network built using keras for text classification. It consists of 3 hidden layer. I am planning to add a dropout layer after each hidden layer to ...
1
vote
0answers
141 views

How many epochs to run during hyperparameter search?

If I'm doing a hyperparameter search and comparing two different hyperparameters (but not number of epochs), is there some established rule of thumb for how many epochs to run? If I just compare ...
2
votes
2answers
80 views

Comparison of machine learning approaches for a topic in a scientific paper

As part of my master's thesis, I have made a prediction of data with approaches of machine learning in a topic where are no papers yet. The topic is a regression problem for which several machine ...
2
votes
1answer
162 views

Parameter tuning for machine learning algorithms

When it comes to the topic of tuning parameters, most of the time you read grid search. But if you have 6 parameters, for which you want to test 10 variants, you get to 10^6 = 1000000 runs. Which in ...
3
votes
1answer
689 views

Automated tuning of Hyperparameter

Are there any advanced packages that allows automated tuning of hyperparameters for neural network and traditional machine learning algorithms like XGBoost, random forest (using method like Bayesian, ...
18
votes
4answers
2k views

How to set the number of neurons and layers in neural networks

I am a beginner to neural networks and have had trouble grasping two concepts: How does one decide the number of middle layers a given neural network have? 1 vs. 10 or whatever. How does one decide ...
3
votes
1answer
226 views

Basic method of optimizing hyperparameters

I recently read the LIPO blog post on the dlib blog: http://blog.dlib.net/2017/12/a-global-optimization-algorithm-worth.html It mentions that it can be used for optimizing hyperparameters of eg ...
2
votes
1answer
49 views

How to think about prediction error that is not convex in hyperparameter, or over the course of training

Take the following case of a hyperparameter and prediction error: Imagine that the hyperparameter is a L2 penalty or a dropout rate -- something that we think that should have a single sweet spot -- ...
1
vote
4answers
2k views

Parameter Tuning by Cross Validation for Random Forest

I train a binary random forest classifier on scikit-learn's 20 newsgroups dataset. I want to tune the parameters and try so by gridsearch and 3-fold cross validation on the training data. Is there ...
2
votes
1answer
208 views

Strategies for automatically tuning the hyper-parameters of deep learning models

I'm interested in trying to develop a framework for automatically tuning the hyperparameters of deep learning models and I'm looking for advice / references to resources that may be useful and ...
1
vote
3answers
120 views

Grid Search and High Variance

I am currently trying to optimise some parameters on my model (15000 samples). What I am finding is a relatively large variance in the loss function 2%-10% which makes it hard to identify which ...
2
votes
1answer
2k views

Which is better: Out of Bag (OOB) or Cross-Validation (CV) error estimates?

I have seen other posts in this forum but didn't find any convincing answer. Random Forest has an another way of tuning hyperparameter via OOB by design. OOB and CV are not the same as OOB error is ...
0
votes
1answer
179 views

Can the learning rate be considered both a parameter AND a hyper-parameter?

Here is my understanding of those 2 terms: Hyper-parameter: A variable that is set by a human before the training process starts. Examples are the number of hidden-layers in a Neural Network, the ...
1
vote
1answer
12k views

How to implement Python's MLPClassifier with gridsearchCV?

I am trying to implement Python's MLPClassifier with 10 fold cross-validation using gridsearchCV function. Here is a chunk of my code: ...
1
vote
1answer
656 views

Bounds for MLP hyperparameter search

I'm trying to optimize a neural network architecture for a particular problem, but there just seems to be so many hyperparameters that I'm concerned that there are much better options that I'm not ...