Skip to main content
13 votes
Accepted

Balanced Accuracy vs. F1 Score

One major difference is that the F1-score does not care at all about how many negative examples you classified or how many negative examples are in the dataset at all; instead, the balanced accuracy ...
A Kareem's user avatar
  • 853
8 votes

How F1 score is good with unbalanced dataset

F1-score The formula for F1-score is: \begin{align*} F1=2 ∗ \frac{\text{precision∗recall}}{\text{precision+recall}} \end{align*} F1-score can be interpreted as a weighted average or harmonic mean of ...
Pluviophile's user avatar
  • 3,928
7 votes

Is it possible to make F1_Score differentiable and use it directly as a Loss function?

Yes there is, let's take $F_1$ score base definition, with : $$ F_1 = 2 \times \frac{precision \times recall} {precision + recall} \\ F_1 = \frac{2 \times TP} {2 \times TP + FP + FN} $$ And this is ...
Thomas FEL's user avatar
7 votes

Can the F1 score be equal to zero?

F1 will never be zero, but very near to zero for a bad classifier. If TP or TN is zero then there isn't any need to check F1.
Gaurav Koradiya's user avatar
6 votes
Accepted

Is an $F_1$ score of 0.1 always bad?

From a credit scoring point of view : a $F_1$ score of $0.1$ seems pretty bad but not impossible with an unbalanced data-set. It might be enough for your needs (once you weight your errors by the cost)...
Lucas Morin's user avatar
  • 2,289
5 votes

Can the F1 score be equal to zero?

It's a mistake on Wikipedia. $F_{1}$ as the harmonic mean is defined only at positive real numbers. $PRE$ or $REC$ could be equal 0 in case $TP=0$. Which provides to undefined result $F_1=\frac{0}{0}...
fuwiak's user avatar
  • 1,373
4 votes

How to compute f1 in TensorFlow

TF Addons computes the F1 score and more generally the FBeta Score
skeller88's user avatar
  • 151
4 votes

Accuracy is lower than f1-score for imbalanced data

I'll try to answer this with a couple examples: Say we have 100 instances (55 negative, 45 positive). Let's say we predict 1/45 positives and 55/55 negatives correctly. Then our accuracy is 0.56 but ...
Ben's user avatar
  • 2,572
4 votes

Is an $F_1$ score of 0.1 always bad?

(1) For the sake of keeping it short in your case: yes 0.1 is bad. To avoid philosophical discussions let's just assume you have to get this higher. (2) It definitely makes sense since your dataset is ...
Noah Weber's user avatar
  • 5,699
3 votes

scikit-learn classification report's f1 accuracy?

I give you that this is a weird way of displaying the data, but the accuracy is the only field that don't fit the schema. For example: ...
Oliver Foster's user avatar
3 votes
Accepted

What is the appropriate statistical significance test for multi-class classification?

Generalisation of Mcnemars is called Cochran–Mantel–Haenszel test. There is an implementation in R, but I suppose porting to Python should not be too hard. You can find the R version here.
Noah Weber's user avatar
  • 5,699
3 votes
Accepted

How to compute f1 in TensorFlow

In tf 2.0+: ...
RonsenbergVI's user avatar
  • 1,034
3 votes
Accepted

problem with using f1 score with a multi class and imbalanced dataset - (lstm , keras)

The problem is simple: recall, precision and F1-score work only with binary classification. If you try with a example manually you will see that the definitions that you're using for precision and ...
Erwan's user avatar
  • 25.5k
3 votes
Accepted

What cost optimisation problem is solved by F score?

There isn't a direct and perfect correspondence between the optimal cost-based objective and the optimal F1 score. Here's an example to show that, though it's perhaps somewhat unsatisfactory because ...
Ben Reiniger's user avatar
  • 12k
2 votes
Accepted

How to correctly calculate average F1 score, precision and recall of a Named Entity Recognition system?

There is a quite detailed comparison with references here: https://towardsdatascience.com/a-tale-of-two-macro-f1s-8811ddcf8f04 Basically the two definitions are used and both can be considered valid. ...
Erwan's user avatar
  • 25.5k
2 votes

Multiclass imbalanced classification

You can weight the loss of each class with a suitable value which is inversely proportional to the class size. One example would be to use: (Total number of data points)/(number of data points of the ...
user1825567's user avatar
  • 1,396
2 votes

Can the F1 score be equal to zero?

It can't be exactly zero. We need exactly one (only one) of precision. Or recall to be zero to make f1 = zero, but both have "tp" as the numerator. ...
10xAI's user avatar
  • 5,634
2 votes

Is it possible to make F1_Score differentiable and use it directly as a Loss function?

Following on Thomas, on the relation between the Bray-Curtis distance and the F1 score and the calculation of the first and second-order derivatives: If one defines the Bray Curtis distance between ...
jane's user avatar
  • 21
2 votes
Accepted

Model to choose with Cross Validation or not?

Selecting the correct scoring metric depends on the business problem you are trying to solve. I would research the differences between f1 micro and macro and determine which scoring metric ultimately ...
Oliver Foster's user avatar
2 votes
Accepted

Warning when plotting confusion matrix with all sample of one class

If you insist on giving no positive examples it will have the same problem. As the error message states, the tool wants to compute the recall. The recall metric is given by: $$recall = \frac{tp}{tp+fn}...
rapaio's user avatar
  • 4,773
2 votes
Accepted

Multiclass Classification and log_loss

Interpretability of log loss Log loss isn't necessarily between the range [0; 1] - it only expects input to be in this range. Take a look at this example: $$ y_{pred} = 0.1 \\ y_{true} = 1.0 \\ log\...
YuseqYaseq's user avatar
2 votes
Accepted

When is Recall@k useful for a classifier with softmax-like output?

If your use case is producing probabilities of 3 classes, you should use multiclass precision/recall/f1, rather than the @k versions. ...
Karl's user avatar
  • 756
1 vote
Accepted

Too high performances on a classification problem

It is normal to get 100% accuracy in this case. I mean it is inevitable to get 100% accuracy. If you train your algorithm even with 10% of your data you will again get 100% accuracy. The abnormal ...
Shahriyar Mammadli's user avatar
1 vote

Too high performances on a classification problem

As noted by Peter, you should split your data into test and train sets before fitting the TfidfVectorizer to the training set (not to all the data). Pandas read_json() is not reading your data from ...
Michael Long's user avatar
1 vote
Accepted

how print f1-score with scikit´s accuracy_score or accuracy of confusion_matrix?

Use f1_score instead of the classification report: ...
Simon Larsson's user avatar
1 vote
Accepted

Selecting threshold for F1 Score

Ideally, the threshold should be selected on your training set. Your holdout set is just there to double confirm that whatever has worked on your training set will generalize to images outside of the ...
Vincent Yong's user avatar
1 vote
Accepted

Is it conscientious to use a threshold for a model output in order to play on the recall and precision?

Good question. Using a threshold is perfectly fine and is not "manual overfitting". It is not manual because this is a step that can (and should) be done automatically. It is not overfitting as it ...
Valentin Calomme's user avatar
1 vote

F1 score graph skewed

The way I see it is because some reason your x limits is different, you could try: plt.xlim(0,120)
Carlos Mougan's user avatar
1 vote
Accepted

Confusion matrix of UNET image sgemenation model

If your masks have only one channel and in the case of binary segmentation, you can easily compute a confusion matrix from one image thanks to: TP: ground truth and predicted pixel are of class 1 (...
Nicolas's user avatar
  • 244
1 vote

Can the F1 score be equal to zero?

If we go by the formula, it can actually be zero when when at least one of precision or recall is zero (regardless of the other one being zero or undefined). Look at the formulas for precision, recall,...
Diamond's user avatar
  • 111

Only top scored, non community-wiki answers of a minimum length are eligible