Stack Exchange Network

Stack Exchange network consists of 174 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [hyperparameter-tuning]

The tag has no usage guidance.

0
votes
2answers
32 views

Validation accuracy is always close to training accuracy

I am trying to tune the hyperparameters of a LSTM I have to do time series forecasting. I have noticed that my validation accuracy is always very close to my training accuracy. I am not sure whether ...
0
votes
0answers
3 views

Hyper-parameter tuning when you don't have an access to the test data

I'm building models for SQUAD (Stanford Question Answering) dataset (https://rajpurkar.github.io/SQuAD-explorer). Stanford doesn't release its test set. It only provides us with training and dev ...
0
votes
0answers
26 views

What makes a Tree-Structured Parzen Estimator “tree-structured?”

From what I understand the Tree-Structured Parzen Estimator (TPE) creates two probability models based on hyperparameters that exceed the performance of some threshold and hyperparameters that don't. ...
0
votes
0answers
14 views

Choosing the right hyperparameter and score for building ensemble

I want to build an ensemble model from individual classifiers(e.g KNN,SVM etc) for classification purpose. Before building the ensemble mode, I want to select the best hyperparameter from the ...
0
votes
0answers
24 views

Help in tuning hyperparameters of DQN based on resulted stats

I'm not an expert in machine learning. I'm trying to make an agent learn to play in a Gym environment that I created. I'm not asking for help in debugging the code. I'm asking for help in interpreting ...
5
votes
4answers
113 views

Which is first ? Tuning the parameters or selecting the model

I've been reading about how we split our data into 3 parts; generally, we use the validation set to help us tune the parameters and the test set to have an unbiased estimate on how well does our model ...
3
votes
1answer
31 views

Hyperparameter tuning for stacked models

I'm reading the following kaggle post for learning how to incorporate model stacking http://blog.kaggle.com/2016/12/27/a-kagglers-guide-to-model-stacking-in-practice/ in ML models. The structure ...
0
votes
0answers
9 views

Per parameter learning rate for AdamOptimizer by scaling gradients

I'm using an AdamOptimizer, and I compute the gradients, but before applying the descent step, I scale (i.e.: multiply) gradients with constants, to mimic having a different learning rate per ...
1
vote
0answers
12 views

std::bad_alloc with densenet and hyperas

I am filing this issue after being stagnated here for couple of weeks. I am using hyperas to find the hyperparameters for my network, Densenet. My issue here is that my evaluation always fails with ...
1
vote
1answer
34 views

Free parameters in logistic regression

When applying logistic regression, one is essentially applying the following function $1/(1 + e^{\beta x})$ to provide a decision boundary, where $\beta$ are a set of parameters that are learned by ...
1
vote
1answer
47 views

Tuning svm and cart hyperparameters

I am trying to optimize the hyperparameters of SVM and CART with tune() function of e1071 R package, but I have a doubt. Should I tune the parameters on the training data, fit the model on the ...
0
votes
0answers
41 views

What are the reasons of select a optimizer to be SGD or Adam in DQN?Why?

I saw several comparison between SGD, RMSPROP and ADAM but what I am looking for is their comparsion in DQN algorithm? What is best to select as optimizer SGD or Adam in DQN? Why? Please check the ...
1
vote
1answer
43 views

What is difference between final episodes of training and test in DQN?

What is difference between running in final episode of training mode and running in test mode in DQN? Is there any difference more than after training and tune the hyper-parameters, we test for one ...
1
vote
2answers
19 views

A way to Identify tuning parameters and their possible range

I am a novice in Machine Learning. But when I started learning, I figure out that all the methods have some tuning parameters and those parameters take a range of possible values. By grid searching, ...
0
votes
0answers
50 views

how to speed up running the DQN with the experience replay?

how to speed up running the DQN with the experience replay? Playing with which hyper parameters have the more effect on it? Because my dqn with experience replay is too slow!
0
votes
0answers
24 views

How to deal with overfitting in Gradient boosting classification algorithm?

I am training my model on Gradient boosting algorithm with parameters as follows: learning rate: 0.1 number of iterations: 100 depth of tree: 12 I am not getting the output for cross-validation to ...
3
votes
1answer
310 views

overfit a Random Forest

I am trying to overfit to the maximum a random forest classifier using scikit-learn to make some tests. Does somebody know what hyperparameters I can tune to do that? Or does somebody know which ...
3
votes
1answer
46 views

How to make it possible for a neural network to tune its own hyper parameters?

I am curious about what would happen to hyperparameters when they would be set by a neural network itself or by creating a neural network that encapsulates and influences the hyperparameters of the ...
1
vote
1answer
25 views

Where can we find the application of bayes's theorem in Bayesian optimiation with gaussian processing

I am trying to learn bayesian optimisation by following this tutorial. However, until now I don't get the relation between bayes's theorem to the gaussian process formalism. Any ideas?
1
vote
1answer
2k views

How to adjust the hyperparameters of MLP classifier to get more perfect performance

I am just getting touch with Multi-layer Perceptron. And, I got this accuracy when classifying the DEAP data with MLP. However, I have no idea how to adjust the hyperparameters for improving the ...
2
votes
2answers
259 views

How to choose the random seed?

I understand this question can be strang, but how do I pick the final random_seed for my classifier? Below is an example code. It uses the ...
0
votes
1answer
78 views

Benefits of using Deep Learning-specific hyperparameter optimization tools vs. sklearn?

There are quite a few library for hyperparameter optimization that are specific to Keras or other Deep Learning libraries, like Hyperas or Talos. My question is, what's the main benefit of using ...
2
votes
3answers
104 views

Hyperparameter Optimization for a Machine Learning Algorithm

I have a question regarding Hyperparameter Optimization for a Machine Learning Algorithm. I try to fit a Support Vector Classifier and use Hyperparameter-Tuning (but it could be also another ...
2
votes
4answers
108 views

Is it better to optimize hyperparameters or run multiple epochs?

Whenever I train a neural network I only have it go through a few epochs ( 1 to 3). This is because I am training them on a bad CPU and it would take some time to have the neural network go though ...
1
vote
0answers
22 views

What to do when two algorithms perform better on different set of classes while doing multi-class classification?

If an algorithm X performed better on a set of classes U and another algorithm Y works better on set of classes V where U and V don's share any class, what improvements can be done from a data ...
0
votes
0answers
17 views

Difference between MOE and Spearmint?

MOE seems to be a plain Bayesian optimization. Just curious if anyone knows about the difference.
3
votes
1answer
62 views

Hyper parameters and ValidationSet

Please correct me if I am wrong. "Training Set is used for calculating parameters of a machine learning model, Validation data is used for calculating hyperparameters of the same model (we use same ...
1
vote
2answers
2k views

Which parameters are hyper parameters in a linear regression?

Can the number of features used in a linear regression be regarded as a hyperparameter? Perhaps the choice of features?
2
votes
0answers
36 views

How does one interpret output from hyper-engine while optimizing a single hyperparameter of a neural network?

I am currently trying to optimize the learning rate of a neural network built in tensorflow. The network has 3 hidden layers, with 500, 250 and 100 neurons respectively. I have adapted code from an ...
0
votes
1answer
184 views

Is there any alternative to L-BFGS-B algorithm for hyperparameter optimization in Scikit learn?

The Gaussian process regression can be computed in scikit learn using an object of class GaussianProcessRegressor as: ...
1
vote
1answer
370 views

Optimising Kernel parameters using training data in GaussianProcessRegressor of Scikit-learn

I want to optimize the Kernel parameters or hyper-parameters using my training data in GaussianProcessRegressor of Scikit-learn.Following is my query: My training datasets are: X: 2-D Cartesian ...
0
votes
0answers
56 views

Training score at parameter tuning lower than on hold out test set (RandomForestClassifier)

I'm doing hyperparameter tuning using RandomizedSearchCV (sklearn) with a 3 fold cross validation on my training set. After that I'm checking my score (accuracy, ...
1
vote
1answer
234 views

trying to decrease overfitting with regularisation in CNN

I am doing transfer learning by retraining the publicly available inception layer, without regularisation here are my initial parameters and results: ...
1
vote
1answer
30 views

tuning a convolution neural net, sample size

I keep reading that convolution neural net (CNN) performs best with lots and lots (100k+) of data. Is there any rule of thumb, or lower limit for data size during the grid search phase? For example, ...
1
vote
0answers
119 views

How many epochs to run during hyperparameter search?

If I'm doing a hyperparameter search and comparing two different hyperparameters (but not number of epochs), is there some established rule of thumb for how many epochs to run? If I just compare ...
2
votes
1answer
155 views

Parameter tuning for machine learning algorithms

When it comes to the topic of tuning parameters, most of the time you read grid search. But if you have 6 parameters, for which you want to test 10 variants, you get to 10^6 = 1000000 runs. Which in ...
3
votes
1answer
556 views

Automated tuning of Hyperparameter

Are there any advanced packages that allows automated tuning of hyperparameters for neural network and traditional machine learning algorithms like XGBoost, random forest (using method like Bayesian, ...
14
votes
4answers
1k views

How to set the number of neurons and layers in neural networks

I am a beginner to neural networks and have had trouble grasping two concepts: How does one decide the number of middle layers a given neural network have? 1 vs. 10 or whatever. How does one decide ...
3
votes
1answer
147 views

Basic method of optimizing hyperparameters

I recently read the LIPO blog post on the dlib blog: http://blog.dlib.net/2017/12/a-global-optimization-algorithm-worth.html It mentions that it can be used for optimizing hyperparameters of eg ...
3
votes
1answer
331 views

Grid seach is unavailable for Keras in case of multiple outputs?

I do experiments with the following Keras architecture with multiple outputs: ...