Questions tagged [f1score]

The tag has no usage guidance.

Filter by
Sorted by
Tagged with
1
vote
1answer
41 views

Perfect scores for multiclass classification

I am working on a multiclass classification problem with 3 (1, 2, 3) classes being perfectly distributed. (70 instances of each class resulting in (210, 8) dataframe). Now my data has all the 3 ...
0
votes
0answers
13 views

Question answering bot: EM>F1, does it make sense?

I am fine-tuning a Question Answering bot starting from a pre-trained model from HuggingFace repo. The dataset I am using for the fine-tuning has a lot of empty answers. So, after the fine tuning, ...
1
vote
1answer
17 views

How to measure multi-label multi-class accuracy

I have a model that has multi-label multi-class targets Example Age Height Weight Mark Distance Red Yellow Green Blue Black White 14 160 62 78 103 0 1 1 1 1 0 56 177 90 99 363 1 1 0 0 0 0 32 179 ...
2
votes
1answer
25 views

Is it ok for precision and recall metrics if a small minority of samples are both false positives and true positives?

I am working on a multi-label classification NN using genomic data. there are 10 samples and 2 ground truth labels (age and gender) for every sample. I use a sigmoid activation at the final layer and ...
0
votes
0answers
13 views

How can I Determine a Treshold According to the Precision and Recall?

I am gettin these precision and recall values from my classifier and I want to determine a treshold for the test data. How can I determine that treshold? Is these values enough or something else is ...
1
vote
1answer
23 views

How to improve my f1 score in stories analyze

I got an assignment to build a model that identify the gender of the text writer. The assignment score will determine by my model f1_score, to get the maximum points, T need it will be at least 0.7. I'...
1
vote
1answer
28 views

Making an ensemble model for high F1 score

I presently have 2 algorithms that have a numerical output. Using a threshold of 0.9, I get the classification output. Let's say they are: P (high precision, low recall) R (high recall, low precision)...
0
votes
1answer
22 views

Accuracy on Validation and Test set, Overfit?

Just a quick question, I am building a ML model right now however I am receiving very similar (72.2 and 72.4 for example)% for both Accuracy and F1-Score on my Validation Dataset and my unseen Test ...
0
votes
0answers
15 views

Model selection for multiple groups to maximize overall f1 score

Model 1 Model 2 .. Model m Group 1 tp11/act1/pred11 .. .. Group 2 tp21/act2/pred21 .. .. .. .. .. .. .. Group n tpn1/actn/predn1 .. tpnm/actn/prednm I have a multiple groups model selection for ...
1
vote
1answer
43 views

How to explain a relationship between Accuracy and F1 Score / F-Measure?

I am building a CNN model for pitch estimation using a song recording. Pitch estimation is done by inputting spectrogram to CNN model and make the CNN predict pitch sequence (250 pitch values per ...
1
vote
1answer
50 views

How to compute f1_score for multiclass multilabel classification

I have used one hot encoder [1,0,0][0,1,0][0,0,1] for my functional classification model. The predicted probabilities for test data ...
0
votes
2answers
82 views

Warning when plotting confusion matrix with all sample of one class

I have two arrays: the first one with all the correct labels (they are all set to zero since each sample belong to the same class) and another one with all the labels predicted by my neural network. ...
0
votes
1answer
30 views

Which F1-score is used for the semantic segmentation tasks?

I read some papers about state-of-the-art semantic segmentation models and in all of them, authors use for comparison F1-score metric, but they did not write whether they use the "micro" or &...
2
votes
1answer
338 views

scikit-learn classification report's f1 accuracy?

When I run scikit-learn classification_report() on my 2-class y and yhat, I get the ...
1
vote
1answer
146 views

What is the appropriate statistical significance test for multi-class classification?

I have a multi-class classification problem. I am primarily using macro-average F1 measure to evaluate the performance of models and want to verify if the results are statistically significant. I have ...
1
vote
3answers
369 views

Too high performances on a classification problem

I have a .json file as dataset of the type: and I am working on a classification problem in which I have to predict 4 classes, which are rhe semantic. I have worked through the problem, and after ...
5
votes
4answers
177 views

Is an $F_1$ score of 0.1 always bad?

I'm currently building a model to predict early mortgage delinquency (60+ days delinquent within 2 years of origination) for loans originating in 2018Q1. I will eventually train out-of-time (on loans ...
-1
votes
1answer
47 views

Model to choose with Cross Validation or not?

I made different tests on an imbalanced dataset and got these results: Model 1 = train test validation split + Cross Validation(cv=10) --> f1'micro' 0,95 Model 2 = train test split + smote method ...
1
vote
1answer
53 views

how print f1-score with scikit´s accuracy_score or accuracy of confusion_matrix?

I would like to print the f1-score. I got confused about the wording f1-accuracy score and accuracy score. What is the difference of these 2 scikit-learn metrics and how can I print the f1-score out ...
1
vote
1answer
315 views

Multiclass Classification and log_loss

I hope I can make this clear with few lines of code/explanation. I've a 16K list of texts, labelled over 30 different classes that were ran through different classifiers; my Prediction and the Ground ...
5
votes
2answers
769 views

Accuracy is lower than f1-score for imbalanced data

For a binary classification, I have a dataset with 55% negative label and 45% positive labels. The results of the classifier shows that the accuracy is lower than the f1-score. Does that mean that the ...
1
vote
1answer
104 views

Selecting threshold for F1 Score

When selecting a probability threshold to maximize the F1 score prior to deploying a model (based on the precision-recall curve), should the threshold be selected based on the training or holdout ...
0
votes
1answer
25 views

How to calculate the evaluation metrics (i.e., F1 score) in leave one subject out cv when a subject belongs to single class only

I have dataset of 10 subjects. the dataset has 4 classess. 0,1,2 and 3. The distribution of classes are not same. For example subject 1 does not have 1,2 and 3. It belongs to zeros class. currently ...
1
vote
1answer
28 views

Is it conscientious to use a threshold for a model output in order to play on the recall and precision?

I have just finished reading an article about the F1 score, recall and precision. Everything was clear except the fact that the author, in his example (see https://towardsdatascience.com/beyond-...
3
votes
1answer
25 views

F1 score graph skewed

The following code ...
7
votes
1answer
2k views

Balanced Accuracy vs. F1 Score

I've read plenty of online posts with clear explanations about the difference between accuracy and F1 score in a binary classification context. However, when I came across the concept of balanced ...
1
vote
0answers
482 views

How can I calculate the F1 score using Mask RCNN?

I customized the "https://github.com/matterport/Mask_RCNN.git" repository to train with my own data set, for object detection, ignoring the mask segmentation part. Now I am evaluating my results, I ...
0
votes
1answer
219 views

Confusion matrix of UNET image sgemenation model

I have used Unet model for image segmentation. I have used RGB images and corresponding image masks and at output i got corresponding region of interest. Now i want to find confusion matrix of this ...
1
vote
0answers
427 views

Calculating the F score of Object Detection of Mask RCNN

I am using Detectron2 Mask RCNN for an object detection problem. The images consist of cells that are very close to each other. I can not use mAP as a performance measure since the annotations are a ...
10
votes
4answers
6k views

Can the F1 score be equal to zero?

As it is mentioned in the F1 score Wikipedia, 'F1 score reaches its best value at 1 (perfect precision and recall) and worst at 0'. What is the worst condition that was mentioned? Even if we ...
-1
votes
1answer
70 views

What is evaluation metric for two sets? [closed]

I've two sets one is ground truth and other is output of my machine learning models. Assume my groundtruth set is A={1,2,3,4,5} and output of machine learning model is B={3,4,5,6,7,8}. One way I can ...
1
vote
0answers
19 views

Monotonicity of Jaccard and Dice in multilabel datasets

I understand that Jaccard and Dice follow a monotonic relation on binary datasets because the two are related as $J = {S \over {(2 - S)}}$, and I guess this would be the case when micro-average is ...
0
votes
1answer
131 views

Different definitions of Macro F1 score, which one is used in Scikit-learn?

In this article Macro F1 and Macro F1 two different definitions of the F1 used in the literature are demonstrated. The first F1 score is computed such as: F1 scores are computed for each class ...
3
votes
1answer
47 views

Multiclass imbalanced classification

I have a dataset with the target variable having 3 classes. Value counts of Target variable are as follows: ...
1
vote
1answer
875 views

Macro F1 result higher than accuracy for imbalanced dataset

In one of the research papers on fake news detection, the authors collected a fake news binary dataset (fake vs. real news) consists of 16,817 real articles and <...
2
votes
1answer
716 views

How to correctly calculate average F1 score, precision and recall of a Named Entity Recognition system?

My Named Entity Recognition (NER) pipeline built with Apache uimaFIT and DKPro recognizes named entities (called datatypes for now) in texts (e.g. persons, locations, organizations and many more). I ...
4
votes
1answer
75 views

Visualizing F-score differences in information extraction

I have several corpora and NLP systems (including a few merge ensembles of output of these systems combined in unions and intersections) with which I have extracted the annotation span sets {(begin, ...
3
votes
2answers
1k views

Is it possible to make F1_Score differentiable and use it directly as a Loss function?

One of the metrics that is widely used in binary classification is the F1 score: $F_1 = 2\cdot \frac{recall \cdot precision}{recall+precision}$ The problem of the F1-score is that it is not ...
4
votes
4answers
12k views

How to compute f1 in TensorFlow

I have a code that computes the accuracy, but now I would like to compute the F1 score. ...