35 votes
Accepted

How to adjust the hyperparameters of MLP classifier to get more perfect performance

If you are using SKlearn, you can use their hyper-parameter optimization tools. For example, you can use: GridSearchCV RandomizedSearchCV If you use GridSearchCV,...
user avatar
27 votes

How to set the number of neurons and layers in neural networks

The consideration of the number of neurons for each layer and number of layers in fully connected networks depends on the feature space of the problem. For illustrating what happens in the two ...
user avatar
  • 13.4k
16 votes

How to set the number of neurons and layers in neural networks

Very good question, as there doesn't exist an exact answer to this question yet. This is an active field of research. Ultimately, the architecture of your network is related to the dimensionality of ...
user avatar
12 votes

How to set the number of neurons and layers in neural networks

Short Answer: It is very related to the dimensions of your data and the type of the application. Choosing the right number of layers can only be achievable with practice. There is no general answer ...
user avatar
  • 1,165
9 votes
Accepted

Which is first ? Tuning the parameters or selecting the model

You can tune parameters only if you have already trained the model, otherwise there is nothing to tune. However, i've also read that model selection shoud be done before tuning the parameters. ...
user avatar
8 votes
Accepted

Which parameters are hyper parameters in a linear regression?

I like the way Wikipedia generally defines it: In machine learning, a hyperparameter is a parameter whose value is set before the learning process begins. By contrast, the values of other ...
user avatar
  • 4,016
7 votes
Accepted

ROC AUC score is much less than average cross validation score

Your test score is incorrect. The ROC curve needs the probability scores from the model, not the class decisions. So replace ...
user avatar
  • 9,698
6 votes
Accepted

How to choose the random seed?

TL:DR, I would suggest not to optimise over the random seed. A better investment of the time would be to improve other parts of your model, such as the pipeline, the underlying algorithms, the loss ...
user avatar
  • 14k
6 votes
Accepted

What makes a Tree-Structured Parzen Estimator "tree-structured?"

It means that your hyperparameter space is tree-like: the value chosen for one hyperparameter determines what hyperparameter will be chosen next and what values are available for it. From a HyperOpt ...
user avatar
  • 9,698
6 votes

What is the most efficient method for hyperparameter optimization in scikit-learn?

Optimization isn't my field, but as far as I know, efficient and effective hyper-parameter optimization these days heavily revolves around building a surrogate model. As models increase in complexity, ...
user avatar
  • 351
5 votes

How to set the number of neurons and layers in neural networks

Working with neural networks since two years ago, this is a problem I always have each time I wan't to model a new system. The best approach I've found is the following: Look for similar problems ...
user avatar
5 votes

Which parameters are hyper parameters in a linear regression?

Hyper-parameters by definition are input parameters which are necessarily required by an algorithm to learn from data. For standard linear regression i.e OLS, there is none. The number/ choice of ...
user avatar
5 votes
Accepted

GridSearchCV vs RandomSearchCV and How it works?

Imagine the following scenario: ...
user avatar
5 votes
Accepted

Is a test set necessary after cross validation on training set?

In "The Elements of Statistical Learning" by Hastie et al the authors describe two tasks regarding model performance measurement: Model selection: estimating the performance of different ...
user avatar
  • 4,877
5 votes

How are parameters selected in cross-validation?

Usually, the aim of K-fold cross-validation is to check how a model performs (both on average and how much it varies across folds) given some hyper-parameters setting. We then pick the "best"...
user avatar
  • 449
4 votes
Accepted

Validation accuracy is always close to training accuracy

The fact that the training accuracy and the validation accuracy are close it is nothing to be concerned about. As you mention, it means your model are generalizing well. The thing to be worried about ...
user avatar
  • 1,588
4 votes

When to use BayesianSearchCV and how it works?

As mentioned in that Kaggle notebook, you can use it pretty much as just a drop-in replacement for other search methods (grid or random). Bayesian searches still are random searches over a predefined ...
user avatar
  • 9,698
4 votes

Why my network needs so many epochs to learn?

My opinion: You should try to increase the learning rate of your model (or even other parameters of your optimizer - e.g. momentum). To answer your questions: Why the network is still learning after ...
user avatar
  • 7,558
4 votes
Accepted

New parameters in final training

Unpopular opinion: Second quickest way to overfit (next to data-leakage) is hyper-parameter optimization. Why? You are assuming you wont have covariate-shift, while in most of the cases you can bet ...
user avatar
  • 5,321
4 votes
Accepted

Opinions on an LSTM hyper-parameter tuning process I am using

First of all you might want to know there is a "new" Keras tuner, which includes BayesianOptimization, so building an LSTM with keras and optimizing its hyperparams is completely a plug-in task with ...
user avatar
  • 2,409
3 votes
Accepted

Hyperparameter Optimization for a Machine Learning Algorithm

If your error stays at about 20%, it sounds like your features are not really helping. It is likely the case that your relationship in data/features is not simple, so you need to allow your SVM model ...
user avatar
  • 14k
3 votes

Hyperparameter Optimization for a Machine Learning Algorithm

You are right, what has to be changed is the objective. You are currently using accuracy as a measure of how good your classifier is. Accuracy is not a good measure when you have class imbalance. For ...
user avatar
  • 5,664
3 votes

How to adjust the hyperparameters of MLP classifier to get more perfect performance

As a complement to the very practical answer of @BrunoGL, I'd like to give a more theoretical answer. I'd like to suggest everyone trying to adjust hyperparameters of a simple Neural Network to read ...
user avatar
  • 2,155
3 votes
Accepted

Optimising Kernel parameters using training data in GaussianProcessRegressor of Scikit-learn

Given scikit-learn's API, you create a separate instance for each optimizer and compare the results to see which optimizer makes better predictions. It would looks something like: ...
user avatar
3 votes

Hyper parameters and ValidationSet

Not completely true. In validation set, we find the best hyperparameters, but not with the same parameters of the model. That is, for every value of the hyperparameters we run the training process and ...
user avatar
  • 5,664
3 votes

Automated Hyperparameter tuning

There are a number of methods to automate the optimization of your hyper-parameters, such as GridSearch and RandomSearch which the article you linked discusses briefly. The main reason to choose one ...
user avatar
  • 1,642
3 votes
Accepted

How to make it possible for a neural network to tune its own hyper parameters?

I am curious about what would happen to hyperparameters when they would be set by a neural network itself In general this is not possible as many hyper-parameters are discrete, so they are not ...
user avatar
  • 27.4k
3 votes
Accepted

overfit a Random Forest

Decision Trees are definitely easier to overfit than Random Forests. The averaging effect (see bagging) is meant to combat overfitting. Other than that I think the default parameters will overfit. ...
user avatar
  • 7,558
3 votes

What is the most efficient method for hyperparameter optimization in scikit-learn?

You can take a look at auto-sklearn. That's an automated machine learning toolkit which is a direct extension of scikit-learn.
user avatar
  • 526
3 votes

Setting best SVM hyper parameters

Well, there is a bunch of articles that tries to tackle this problem but basically, to guarantee a good solution you will need to do Grid Search (sklearn tutorial on it) You can use various ...
user avatar

Only top scored, non community-wiki answers of a minimum length are eligible