# Tag Info

Accepted

### What is the difference between model hyperparameters and model parameters?

Hyperparameters and parameters are often used interchangeably but there is a difference between them. You can call something a 'hyperparameter' if it cannot be learned within the estimator directly. ...
• 2,961

### What is the difference between model hyperparameters and model parameters?

In addition to the answer above. Model parameters are the properties of the training data that are learnt during training by the classifier or other ml model. For example in case of some NLP task: ...
• 2,057

### How to set the number of neurons and layers in neural networks

The consideration of the number of neurons for each layer and number of layers in fully connected networks depends on the feature space of the problem. For illustrating what happens in the two ...
• 13.5k

### How to set the number of neurons and layers in neural networks

Very good question, as there doesn't exist an exact answer to this question yet. This is an active field of research. Ultimately, the architecture of your network is related to the dimensionality of ...
• 5,196

### What is the difference between model hyperparameters and model parameters?

Hyper-parameters are those which we supply to the model, for example: number of hidden Nodes and Layers,input features, Learning Rate, Activation Function etc in Neural Network, while Parameters are ...

### Choosing a learning rate

Selecting a learning rate is an example of a "meta-problem" known as hyperparameter optimization. The best learning rate depends on the problem at hand, as well as on the architecture of the model ...
• 558

### How to set the number of neurons and layers in neural networks

Short Answer: It is very related to the dimensions of your data and the type of the application. Choosing the right number of layers can only be achievable with practice. There is no general answer ...
• 1,185

### Choosing a learning rate

Copy-pasted from my masters thesis: If the loss does not decrease for several epochs, the learning rate might be too low. The optimization process might also be stuck in a local minimum. Loss being ...
• 17.9k

### How can you decide the window size on a pooling layer?

When you get a bit more insight into network topologies these hyperparameters will make more sense, but in general this is just like any other hyperparameter, you will have to test some settings and ...
• 9,118
Accepted

### Which is first ? Tuning the parameters or selecting the model

You can tune parameters only if you have already trained the model, otherwise there is nothing to tune. However, i've also read that model selection shoud be done before tuning the parameters. ...

### What is the difference between model hyperparameters and model parameters?

In machine learning, a model $M$ with parameters and hyper-parameters looks like, $Y \approx M_{\mathcal{H}}(\Phi | D)$ where $\Phi$ are parameters and $\mathcal{H}$ are hyper-parameters. $D$ is ...
• 1,303
Accepted

### How to implement Python's MLPClassifier with gridsearchCV?

A tuple of the form $(i_1, i_2, i_3, ... , i_n)$ gives you a network with $n$ hidden layers, where $i_k$ gives you the number of neurons in the $k$th hidden layer. If you want three hidden layers ...
• 6,035
Accepted

### Which parameters are hyper parameters in a linear regression?

I like the way Wikipedia generally defines it: In machine learning, a hyperparameter is a parameter whose value is set before the learning process begins. By contrast, the values of other ...
• 4,046

### What is the most efficient method for hyperparameter optimization in scikit-learn?

Optimization isn't my field, but as far as I know, efficient and effective hyper-parameter optimization these days heavily revolves around building a surrogate model. As models increase in complexity, ...
• 361
Accepted

### ROC AUC score is much less than average cross validation score

Your test score is incorrect. The ROC curve needs the probability scores from the model, not the class decisions. So replace ...
• 9,982

### Choosing a learning rate

Learning rate , transformed as "step size" during our iteration process , has been a hot issue for years , and it will go on . There are three options for step size in my concerning : One is related ...
• 399
Accepted

### How do scientists come up with the correct Hidden Markov Model parameters and topology to use?

I'm familiar with three main approaches: A priori. You might know that there are four base pairs to pick from, and so allow the HMM to have four states. Or you might know that English has 44 phonemes,...
Accepted

### How to choose the random seed?

TL:DR, I would suggest not to optimise over the random seed. A better investment of the time would be to improve other parts of your model, such as the pipeline, the underlying algorithms, the loss ...
• 14.1k
Accepted

### XGBoost and Random Forest: ntrees vs. number of boosting rounds vs. n_estimators

As I understand it, iterations is equivalent to boosting rounds. However, number of trees is not necessarily equivalent to the above, as xgboost has a parameter ...
• 1,517

### What is the difference between model hyperparameters and model parameters?

In simplified words, Model Parameters are something that a model learns on its own. For example, 1) Weights or Coefficients of independent variables in Linear regression model. 2) Weights or ...

### Hyperparameter search for LSTM-RNN using Keras (Python)

I would recommend Bayesian Optimization for hyper parameter search and had good results with Spearmint. You might have to use an older version for commercial use.

### Hyperparameter search for LSTM-RNN using Keras (Python)

An embedding layer turns positive integers (indexes) into dense vectors of fixed size. For instance, [[4], [20]] -> [[0.25, 0.1], [0.6, -0.2]]. This ...
• 790

### Choosing a learning rate

Adding to David's answer, in fastai is where I found the concept of finding the best learning rate for that data, using a particular architecture. But that thing exists only on fastai/pytorch. ...
• 626

### How to set the number of neurons and layers in neural networks

Working with neural networks since two years ago, this is a problem I always have each time I wan't to model a new system. The best approach I've found is the following: Look for similar problems ...

### Which parameters are hyper parameters in a linear regression?

Hyper-parameters by definition are input parameters which are necessarily required by an algorithm to learn from data. For standard linear regression i.e OLS, there is none. The number/ choice of ...
• 790

### How to set hyperparameters in SVM classification

You could use cross validation with grid search as shown here

### Is it OK to try to find the best PCA k parameter as we do with other hyperparameters?

Your emphasis on using a validation set rather than the training set for selecting $k$ is a good practice and should be followed. However, we can do even better! The parameter $k$ in $\text{PCA}$ is ...
• 8,667
Accepted

### Grid search or gradient descent?

The main distinction to be made here is between a parameter and a hyperparameter; once we have this clarified, the rest is easy: grid search is not used for tuning the parameters (only the ...
• 1,820