Skip to main content
36 votes
Accepted

What is the difference between model hyperparameters and model parameters?

Hyperparameters and parameters are often used interchangeably but there is a difference between them. You can call something a 'hyperparameter' if it cannot be learned within the estimator directly. ...
enterML's user avatar
  • 3,041
33 votes

What is the difference between model hyperparameters and model parameters?

In addition to the answer above. Model parameters are the properties of the training data that are learnt during training by the classifier or other ml model. For example in case of some NLP task: ...
tastyminerals's user avatar
17 votes

What is the difference between model hyperparameters and model parameters?

Hyper-parameters are those which we supply to the model, for example: number of hidden Nodes and Layers,input features, Learning Rate, Activation Function etc in Neural Network, while Parameters are ...
Lakshmi Prasad Y's user avatar
9 votes

What is the correct way to compute Mean F1 score?

This paper Apples-to-apples in cross-validation studies: pitfalls in classifier performance measurement by Forman and Scholz discuss the different methods for computing the average F-score in cross ...
tiagotvv's user avatar
  • 276
8 votes
Accepted

What is the difference of R-squared and adjusted R-squared?

A google search for r-squared adjusted yielded several easy to follow explanations. I am going to paste a few directly from such results. Meaning of Adjusted R2 Both R2 and the adjusted R2 give you ...
grldsndrs's user avatar
  • 567
8 votes

What is the difference between model hyperparameters and model parameters?

In machine learning, a model $M$ with parameters and hyper-parameters looks like, $Y \approx M_{\mathcal{H}}(\Phi | D)$ where $\Phi$ are parameters and $\mathcal{H}$ are hyper-parameters. $D$ is ...
Dynamic Stardust's user avatar
7 votes
Accepted

Do models without parameters exist?

Is there any model in machine learning that does not have parameters? Yes. k-nearest neighbors is parameterless (there is only a single hyper-parameter $k$). If such parameterless models exist, ...
E_net4's user avatar
  • 364
5 votes

What is the difference between model hyperparameters and model parameters?

In simplified words, Model Parameters are something that a model learns on its own. For example, 1) Weights or Coefficients of independent variables in Linear regression model. 2) Weights or ...
Manju Savanth's user avatar
5 votes
Accepted

Should you care about truncation and padding in an LLM even if it has a very large tokenizer.max_length so that truncation will never happen?

The large number you are seeing is not the maximum length, but the maximum representable integer at that precision. It's there because no maximum length has been set. The original GPT-2 has a maximum ...
noe's user avatar
  • 27k
4 votes

How does C have effects on bias and variance of a Support Vector Machine?

The C being a regularized parameter, controls how much you want to punish your model for each misclassified point for a given curve. If you put large value to C it will try to reduce errors but at the ...
prashant0598's user avatar
  • 1,511
2 votes
Accepted

Regression model with variable number of parameters in dataset?

One simple idea, no imputation needed: build a model using the parameters have always existed, then each time a new set of parameters gets added, use them to model the residual of the previous model. ...
Ken Arnold's user avatar
2 votes
Accepted

What is the correct way to compute Mean F1 score?

As mentioned by other users, the solution is not very clear. The general approach is to follow what is mentioned here. Also, as suggested by one of the senior R&D employee and my mentor, the ...
Pinkesh Badjatiya's user avatar
2 votes
Accepted

Max pooling has no parameters and therefore doesn't affect the backpropagation?

Pooling layers does not have parameters which affect the back propagation. Backpropagation is an algorithm to efficiently implement gradient descent in a neural network by using the chain rule and ...
ignatius's user avatar
  • 1,668
2 votes

How can you get a Huggingface fine-tuning model with the Trainer class from your own text where you can set the arguments for truncation and padding?

If you use transformers example scripts, for example, summarization, you can control padding and truncation with command line arguments --pad_to_max_length, ...
Valentas's user avatar
  • 1,229
2 votes
Accepted

How can I know how to interpret the output coefficients (`coefs_`) from the model sklearn.svm.LinearSVC()?

Here's one (admittedly hard) way. If you really want to understand the low-level details, you can always work through the source code. For example, we can see that the ...
jncraton's user avatar
  • 578
2 votes
Accepted

Why does bidirectional LSTM have half the parameter count compared to LSTM in keras?

For Unidirectional LSTM the number of parameters are 4*[(numHiddenUnit+inputSize)*numHuddenUnits+numHuddenUnits] where 4 is for 4 LSTM gate equations. For your case numHuddenUnits = 256, inputSize is ...
Saurabh Tiwari's user avatar
2 votes

How is DS used in the case of Payment Gateways?

Common use cases include: Fraud detection Transactions volume prediction Next transaction date Fraud detection This is usually tackled with anomaly detection. It requires information on the two ...
Bruno Lubascher's user avatar
1 vote

eval_metric of XGBoost // ML model in general

Which metric to choose is not related to the model, but to the problem to be solved. If you are unsure, go back and think about the objective - why we need to build the model, and what the model needs ...
lpounng's user avatar
  • 1,094
1 vote

what exactly is the "order"-parameter in pandas interpolation?

The order argument simply refers to the order of the function that is used when interpolating values. As the documentation mentiones, you only need to provide a ...
Oxbowerce's user avatar
  • 7,592
1 vote
Accepted

Number of parameters in CNN

A convolutional layer is composed of a grid of numbers called filter (or kernel). This is the filter that scans the image (talking about 2D convolutions here). Applying means simply multiplying the ...
serali's user avatar
  • 1,242
1 vote

Derivative of activation function used in gradient descent algorithms

As the name suggests, Gradient Descent ( GD ) optimization works on the principle of gradients which basically is a vector of all partial derivatives of a particular function. According to Wikipedia, ...
Shubham Panchal's user avatar
1 vote

Generative network understanding

latent_dim does not become of shape 16*16 x = layers.Dense(128 * 16 * 16)(generator_input) mean: The input of size 32 (the latent_dim) is connected to a layer ...
vico's user avatar
  • 148
1 vote

When I include validation_data=(x_val, y_val) in model.fit_generator, should I create another test dataset for accuracy measures?

You are correct in saying that it would be unfair - and if avoidable, you shouldn't do it. In order to truly be able to claim (in a statistical sense) that a model achieves e.g. 90% accuracy, the ...
n1k31t4's user avatar
  • 14.9k
1 vote

Coefficients from Logistic Regression using Scikit-Learn

In logistic regression, we don't fit a linear line to our data points. Instead, we fit a linear line to the log-likelihood of our data point. Therefore, you need to take exp to fit it accurately.
Vivek Khetan's user avatar
1 vote

Coefficients from Logistic Regression using Scikit-Learn

Can you post some sample code of what you're trying to do? I can match predict_proba exactly when using the coefficients and intercept. ...
Tophat's user avatar
  • 2,430
1 vote

Sklearn - Override random_state=None by default

For scikit-learn can set np.random.seed(1), for example, and as long as nothing in your script is modifying the seed nondeterministically then you should get ...
Imran's user avatar
  • 2,381
1 vote

Do models without parameters exist?

Consider the case of the majority rule. In the majority rule you go over the training set, check which concept value is the majority and returns it for every sample. There are no parameters, there is ...
DaL's user avatar
  • 2,643
1 vote
Accepted

Can the learning rate be considered both a parameter AND a hyper-parameter?

My understanding is that $\eta$ is set before the training starts to a large value but then, as the training progresses and the function gets closer and closer to a local minimum, the learning rate is ...
Neil Slater's user avatar
1 vote

How can I know how to interpret the output coefficients (`coefs_`) from the model sklearn.svm.LinearSVC()?

y = -(coef_0 / coef_1) x - intercept/coef_1 is coef_0 x + coef_1 y + intercept = 0, which is the border line separating the blob ...
Jihyun's user avatar
  • 111
1 vote

Regression model with variable number of parameters in dataset?

If the old variables and the new variables are highly correlated then you could do a more advanced form of imputation and make a model for each new input that predicts the new input given the old ...
Ryan Zotti's user avatar
  • 4,149

Only top scored, non community-wiki answers of a minimum length are eligible