Skip to main content
57 votes

What is the relationship between the accuracy and the loss in deep learning?

There is no relationship between these two metrics. Loss can be seen as a distance between the true values of the problem and the values predicted by the model. The larger the loss, the larger the ...
Jérémy Blain's user avatar
45 votes
Accepted

What's the difference between Sklearn F1 score 'micro' and 'weighted' for a multi class classification problem?

F1Score is a metric to evaluate predictors performance using the formula F1 = 2 * (precision * recall) / (precision + recall) where recall = TP/(TP+FN) and precision = TP/(TP+FP) and ...
TitoOrt's user avatar
  • 1,882
22 votes
Accepted

How to interpret classification report of scikit-learn?

The classification report is about key metrics in a classification problem. You'll have precision, recall, f1-score and support for each class you're trying to find. The recall means "how many of ...
LaSul's user avatar
  • 471
13 votes
Accepted

Balanced Accuracy vs. F1 Score

One major difference is that the F1-score does not care at all about how many negative examples you classified or how many negative examples are in the dataset at all; instead, the balanced accuracy ...
A Kareem's user avatar
  • 853
12 votes

What is the relationship between the accuracy and the loss in deep learning?

Actually, accuracy is a metric that can be applied to classification tasks only. It describes just what percentage of your test data are classified correctly. For example, you have binary ...
DmytroSytro's user avatar
12 votes

MAD vs RMSE vs MAE vs MSLE vs R²: When to use which?

Well actually these can give you different insights into your models errors. If $y$ is your target, $p$ your prediction and $e = p - y$ the errors: Mean Error: $ME = mean(e)$ In (-∞,∞), the closer ...
Ludecan's user avatar
  • 261
12 votes
Accepted

Cosine similarity vs The Levenshtein distance

As mentioned in other answers, traditionally cosine is used to measure similarity between vectors whereas Levenshtein is used as a string similarity measure, i.e. measuring the distance between ...
Erwan's user avatar
  • 25.6k
11 votes
Accepted

What is Continuous Ranked Probability Score (CRPS)?

CRPS is in a sense just the mean square error (MSE) of your predicted cumulative density function (CDF) and the true CDF. The CRPS generalizes the MAE (Mean Absolute Error) to the case of ...
Pluviophile's user avatar
  • 3,958
10 votes

Why do we use a Gaussian kernel as a similarity metric?

Let's be precise. "Distance" has lots of meanings in data science, I think you're talking about Euclidean distance. The Gaussian kernel is a non-linear function of Euclidean distance. The kernel ...
SmallChess's user avatar
  • 3,570
9 votes

What is the relationship between the accuracy and the loss in deep learning?

The other answers give good definitions of accuracy and loss. To answer your second question, consider this example: We have a problem of classifying images from a balanced dataset as containing ...
rlms's user avatar
  • 191
7 votes

What is the relationship between the accuracy and the loss in deep learning?

Someone says that accuracy has no relationship to the loss, but from a theoretical perspective, there IS a relationship. Accuracy is $1 - (error\ rate)$ and the error rate can be seen as the ...
aest's user avatar
  • 71
7 votes
Accepted

F1_score(average='micro') is equal to calculating accuracy for multiclasification

In classification tasks for which every test case is guaranteed to be assigned to exactly one class, micro-F is equivalent to accuracy. The above answer is from: https://stackoverflow.com/questions/...
Saandeep Sreerambatla's user avatar
7 votes

Can the F1 score be equal to zero?

F1 will never be zero, but very near to zero for a bad classifier. If TP or TN is zero then there isn't any need to check F1.
Gaurav Koradiya's user avatar
6 votes
Accepted

Interpretability of RMSE and R squared scores on cross validation

And one last thing, is my result on X_train indicative that my features are informative enough to learn the target? or is the R² train score somehow biased? High scoring fits on training data does ...
Dan Scally's user avatar
  • 1,764
6 votes
Accepted

XGBoost custom objective for regression in R

I have a suggestion. Indeed the methodology is the right one but the problem comes from the definition of your functions. Since they are not the right ones, they then give the wrong Grad and Hess. The ...
Kipédène Coulibaly's user avatar
5 votes

Log loss vs accuracy for deciding between different learning rates?

According to me, it is not correct to co-relate loss with accuracy. Loss is used to optimize the hypothesis such that we can get best weights whereas accuracy is used to identify how well model ...
vipin bansal's user avatar
  • 1,272
5 votes

Inception Score (IS) and Fréchet Inception Distance (FID), which one is better for GAN evaluation?

Here is the original paper proposing FID. Here is an excerpt from Jason Brownlee's https://machinelearningmastery.com/how-to-implement-the-frechet-inception-distance-fid-from-scratch/ along with a ...
Noah Weber's user avatar
  • 5,709
5 votes

Can the F1 score be equal to zero?

It's a mistake on Wikipedia. $F_{1}$ as the harmonic mean is defined only at positive real numbers. $PRE$ or $REC$ could be equal 0 in case $TP=0$. Which provides to undefined result $F_1=\frac{0}{0}...
fuwiak's user avatar
  • 1,373
5 votes
Accepted

Confusion between precision and recall

First of all, precision and recall are not specific to image classification; they are relevant wherever there are two distinct "positive" and "negative" classes (for example, when ...
Itamar Mushkin's user avatar
4 votes

can accuracy rise while precision and recall drop?

The explanation is simple, assume you have the following values: ...
Dani Mesejo's user avatar
  • 2,226
4 votes
Accepted

Linear Regression - Metrics to consider?

Aditya already mentioned rightly so in the comments RMSE, MSE or MAE is preferred while evaluating a linear regression model. So to answer your questions I will provide with the pros and cons of using ...
moksha's user avatar
  • 146
4 votes
Accepted

Is R2 score a reasonable regression measure on huge datasets?

The coefficient of determination $r^2$ is defined in terms of variance: it is the proportion of variance in the dependent variable that is explained by the independent variable. Variance is a property ...
Pieter's user avatar
  • 961
4 votes
Accepted

F - measure derivation (harmonic mean of precision and recall)

why the equality of both partial derivatives correspond to these hypothesis. I would rather understand when one partial derivative equals the other partial derivative multiplied by minus one. ...
KT.'s user avatar
  • 2,121
4 votes

How to compute f1 in TensorFlow

TF Addons computes the F1 score and more generally the FBeta Score
skeller88's user avatar
  • 151
4 votes

Cosine similarity vs The Levenshtein distance

I think the answers you got are technically correct, but don't address the big picture. In the data science world, cosine similarity is mainly used for documents which have been encoded by an ...
Wayne's user avatar
  • 316
4 votes

What is AUC - ROC Curve?

Related concepts, but not the same. ROC-receiver operating curve AUC area under the curve Thank this post for explanation : Abbreviations AUC = Area Under the Curve. AUROC = Area Under the ...
Noah Weber's user avatar
  • 5,709
4 votes
Accepted

How to select 'cutoff' of classifier probability

What you're looking for is something along the line of an ROC curve: Using the threshold as a decision parameter, you can observe the trade-off between FPR (False Positive Rate: how many of the ...
Itamar Mushkin's user avatar
4 votes
Accepted

Once a predictive model is in production, how it can be evaluated?

This is in fact a very good question. The answer is simple, but depends on the case. In general, what we do after pushing a model to production we apply an audit process. Let me explain: in reality ...
Bashar Haddad's user avatar
3 votes

Why is the F-measure preferred for classification tasks?

The Fı-score is preferred to simple classification accuracy in order to counter the problem of imbalanced datasets; if the thing you are looking for occurs only rarely anyway then a naive classifier ...
Gaius's user avatar
  • 328
3 votes

Good performance metrics for multiclass classification problem besides accuracy?

If you can use python I suggest PyCM module. A vast variety of performance evaluation parameters is in access by this module and also you can use its documentation if you want to implement it by ...
Alireza Zolanvari's user avatar

Only top scored, non community-wiki answers of a minimum length are eligible