Hot answers tagged

46 votes

What is the relationship between the accuracy and the loss in deep learning?

There is no relationship between these two metrics. Loss can be seen as a distance between the true values of the problem and the values predicted by the model. Greater the loss is, more huge is the ...
40 votes
Accepted

What's the difference between Sklearn F1 score 'micro' and 'weighted' for a multi class classification problem?

F1Score is a metric to evaluate predictors performance using the formula F1 = 2 * (precision * recall) / (precision + recall) where recall = TP/(TP+FN) and precision = TP/(TP+FP) and ...
  • 1,792
22 votes
Accepted

How to interpret classification report of scikit-learn?

The classification report is about key metrics in a classification problem. You'll have precision, recall, f1-score and support for each class you're trying to find. The recall means "how many of ...
  • 441
11 votes
Accepted

Balanced Accuracy vs. F1 Score

One major difference is that the F1-score does not care at all about how many negative examples you classified or how many negative examples are in the dataset at all; instead, the balanced accuracy ...
  • 803
10 votes

Why do we use a Gaussian kernel as a similarity metric?

Let's be precise. "Distance" has lots of meanings in data science, I think you're talking about Euclidean distance. The Gaussian kernel is a non-linear function of Euclidean distance. The kernel ...
  • 3,470
10 votes

What is the relationship between the accuracy and the loss in deep learning?

Actually, accuracy is a metric that can be applied to classification tasks only. It describes just what percentage of your test data are classified correctly. For example, you have binary ...
9 votes

MAD vs RMSE vs MAE vs MSLE vs R²: When to use which?

Well actually these can give you different insights into your models errors. If $y$ is your target, $p$ your prediction and $e = p - y$ the errors: Mean Error: $ME = mean(e)$ In (-∞,∞), the closer ...
  • 231
9 votes
Accepted

What is Continuous Ranked Probability Score (CRPS)?

CRPS is in a sense just the mean square error (MSE) of your predicted cumulative density function (CDF) and the true CDF. The CRPS generalizes the MAE (Mean Absolute Error) to the case of ...
  • 2,786
8 votes

What is the relationship between the accuracy and the loss in deep learning?

The other answers give good definitions of accuracy and loss. To answer your second question, consider this example: We have a problem of classifying images from a balanced dataset as containing ...
  • 181
8 votes
Accepted

Cosine similarity vs The Levenshtein distance

As mentioned in other answers, traditionally cosine is used to measure similarity between vectors whereas Levenshtein is used as a string similarity measure, i.e. measuring the distance between ...
  • 24.4k
7 votes

Can the F1 score be equal to zero?

F1 will never be zero, but very near to zero for a bad classifier. If TP or TN is zero then there isn't any need to check F1.
6 votes
Accepted

Interpretability of RMSE and R squared scores on cross validation

And one last thing, is my result on X_train indicative that my features are informative enough to learn the target? or is the R² train score somehow biased? High scoring fits on training data does ...
  • 1,714
6 votes
Accepted

F1_score(average='micro') is equal to calculating accuracy for multiclasification

In classification tasks for which every test case is guaranteed to be assigned to exactly one class, micro-F is equivalent to accuracy. The above answer is from: https://stackoverflow.com/questions/...
5 votes

Log loss vs accuracy for deciding between different learning rates?

According to me, it is not correct to co-relate loss with accuracy. Loss is used to optimize the hypothesis such that we can get best weights whereas accuracy is used to identify how well model ...
  • 1,234
5 votes

Inception Score (IS) and Fréchet Inception Distance (FID), which one is better for GAN evaluation?

Here is the original paper proposing FID. Here is an excerpt from Jason Brownlee's https://machinelearningmastery.com/how-to-implement-the-frechet-inception-distance-fid-from-scratch/ along with a ...
  • 5,459
5 votes

Can the F1 score be equal to zero?

It's a mistake on Wikipedia. $F_{1}$ as the harmonic mean is defined only at positive real numbers. $PRE$ or $REC$ could be equal 0 in case $TP=0$. Which provides to undefined result $F_1=\frac{0}{0}...
  • 1,357
5 votes
Accepted

Confusion between precision and recall

First of all, precision and recall are not specific to image classification; they are relevant wherever there are two distinct "positive" and "negative" classes (for example, when ...
4 votes
Accepted

Is R2 score a reasonable regression measure on huge datasets?

The coefficient of determination $r^2$ is defined in terms of variance: it is the proportion of variance in the dependent variable that is explained by the independent variable. Variance is a property ...
  • 961
4 votes
Accepted

F - measure derivation (harmonic mean of precision and recall)

why the equality of both partial derivatives correspond to these hypothesis. I would rather understand when one partial derivative equals the other partial derivative multiplied by minus one. ...
  • 2,101
4 votes

can accuracy rise while precision and recall drop?

The explanation is simple, assume you have the following values: ...
  • 2,186
4 votes
Accepted

Linear Regression - Metrics to consider?

Aditya already mentioned rightly so in the comments RMSE, MSE or MAE is preferred while evaluating a linear regression model. So to answer your questions I will provide with the pros and cons of using ...
  • 146
4 votes

What is the relationship between the accuracy and the loss in deep learning?

Someone says that accuracy has no relationship to the loss, but from a theoretical perspective, there IS a relationship. Accuracy is $1 - (error\ rate)$ and the error rate can be seen as the ...
  • 41
4 votes

How to compute f1 in TensorFlow

TF Addons computes the F1 score and more generally the FBeta Score
  • 151
4 votes

What is AUC - ROC Curve?

Related concepts, but not the same. ROC-receiver operating curve AUC area under the curve Thank this post for explanation : Abbreviations AUC = Area Under the Curve. AUROC = Area Under the ...
  • 5,459
4 votes
Accepted

How to select 'cutoff' of classifier probability

What you're looking for is something along the line of an ROC curve: Using the threshold as a decision parameter, you can observe the trade-off between FPR (False Positive Rate: how many of the ...
4 votes
Accepted

XGBoost custom objective for regression in R

I have a suggestion. Indeed the methodology is the right one but the problem comes from the definition of your functions. Since they are not the right ones, they then give the wrong Grad and Hess. The ...
4 votes
Accepted

Once a predictive model is in production, how it can be evaluated?

This is in fact a very good question. The answer is simple, but depends on the case. In general, what we do after pushing a model to production we apply an audit process. Let me explain: in reality ...
3 votes

Why is the F-measure preferred for classification tasks?

The Fı-score is preferred to simple classification accuracy in order to counter the problem of imbalanced datasets; if the thing you are looking for occurs only rarely anyway then a naive classifier ...
  • 328
3 votes

Good performance metrics for multiclass classification problem besides accuracy?

If you can use python I suggest PyCM module. A vast variety of performance evaluation parameters is in access by this module and also you can use its documentation if you want to implement it by ...
3 votes
Accepted

Can macro F1 score be greater than micro F1 score?

Yes, it can. Example with Precision Class A: 1 TP and 1 FP Class B: 10 TP and 90 FP Class C: 1 TP and 1 FP Class D: 1 TP and 1 FP Here, $P_A = P_C = P_D = 0.5$, $P_B = 0.1$ Macro-F1 is: $P_M = \...

Only top scored, non community-wiki answers of a minimum length are eligible