Skip to main content
Share Your Experience: Take the 2024 Developer Survey

Questions tagged [f1score]

The tag has no usage guidance.

Filter by
Sorted by
Tagged with
1 vote
1 answer
40 views

Does f1 score evaluate only the model or does it also enable us to observe and evaluate the data?

I have a dataset. This dataset consists of the data that the actual picture that needs to be drawn, that is, the 100-point graded paper, and the similarity between 100 and 0 points graded pictures ...
doqukan's user avatar
  • 11
0 votes
0 answers
25 views

Why the f1 score on validation dataset significantly higher than f1 score on testing dataset?

I'm using a TensorFlow model that look likes this: ...
Furno's user avatar
  • 1
0 votes
0 answers
19 views

PR-AUC vs F1 vs Balanced Accuracy

I'm trying to create a Random Forest Classifier for selecting ~ 700 features. I have a highly imbalanced dataset to select features from. There are significantly fewer positive cases (1%) compared ...
user155775's user avatar
4 votes
1 answer
107 views

What cost optimisation problem is solved by F score?

I know the general expression of the F1-score: $$F1 = \frac{precision * recall}{precision + recall}$$ And its $F_{beta}$ variants (see: https://en.wikipedia.org/wiki/F-score): $$F_{beta} = (1+\beta^2) ...
Lucas Morin's user avatar
  • 2,254
0 votes
0 answers
10 views

Which analog of F1 score metrics can I use in this case?

I am training a cnn segmentation model and I need some analog of F1 score So, we have GT as red rectangles (called "red") and Pred as blue rectangles (called "blue"). It is clear ...
sixtytrees's user avatar
0 votes
0 answers
41 views

Finding Accuracy, Recall, Precision, and F1 from Matlab Confusion Matrix

I'm working on a project to find the highest accuracy between KNN and a Decision Tree for Classification using Matlab. How to calculate the Accuracy, Recall, Precision, and F1 from the output below? ...
willow's user avatar
  • 1
0 votes
1 answer
68 views

When is Recall@k useful for a classifier with softmax-like output?

If a 3-class classifier returns a length-3 vector of probabilities, e.g. [0.1, 0.85, 0.05] for classes A, B, and C respectively (strongly indicating B), does it ...
Alex Shroyer's user avatar
1 vote
0 answers
27 views

Correctness of derivation for binary F1 variance for F1 confidence intervals

I'm developing a python library for confidence intervals for common accuracy metrics, with both analytic and bootstrap computations. Following this paper, I implemented the Macro and Micro F1 scores ...
Jacob G's user avatar
  • 121
0 votes
1 answer
277 views

How to explain relative difference between macro-AUC and macro-F1 in a multiclass classification problem?

I recently published a paper in which the result of a supervised model is the following. All the metrics are macro-averaged. I have been asked to comment on the gap between the AUC and the other ...
Eric Yamga's user avatar
1 vote
1 answer
32 views

4/96 imbalanced but all metrics above .95

I'm working with some severely imbalanced dataset where my 1 class represents 4% of the data in a binary classification problem. I have about 10M rows and developed a model that outputs +.95 in ...
Marc's user avatar
  • 222
1 vote
1 answer
132 views

Feature Engineer each class separately in Binary Classification

I have an imbalanced tabular dataset, my problem is a binary classification. The dataset is strongly imbalanced so I have performed oversampling, but it did not solve the issue, you can find the ...
bechirjamoussi's user avatar
0 votes
1 answer
18 views

output F1-score instead of Accuracy

I have the code below outputting the accuracy. How can I output the F1-score instead? ...
Pedro Silvestre's user avatar
0 votes
1 answer
220 views

Scikit learn ComplementNB is outputting NaN for scores

I have an unbalanced binary dataset with 23 features, 92000 rows are labeled 0, and 207,000 rows are labeled 1. I trained models on this dataset such as GaussianNB, DecisionTreeClassifier, and a few ...
Sharhad Bashar's user avatar
-1 votes
1 answer
29 views

Difference between the different measurement metric [closed]

Can someone explain what each of these mean? both in simple terms and in terms of TP, TN, FP, FN? Also are there any other common metrics that I am missing? F-measure or F-score Recall Precision ...
Sharhad's user avatar
0 votes
1 answer
715 views

Is it correct to train and validate the model on F1-score metrics?

I am trying to do experiments on multiple data sets. Some are more imbalanced than others. Now, in order to assure fair reporting, we compute F1-Score on test data. In most machine learning models, we ...
Ahmad's user avatar
  • 148
0 votes
0 answers
117 views

Overfitting? Is it ok, if I've met my desired threshold?

I've trained a lightgbm classification model, selected features, and tuned the hyperparameters all to obtain a model that appears to work well. When I've come to evaluate it on an out of bag selection ...
Lewis Morris's user avatar
5 votes
1 answer
8k views

How F1 score is good with unbalanced dataset

I have read around on this site that it's recommended to use F1 score if the dataset is imbalanced and if you want to seek a balance between recall and precession. Could you please explain how F1 can ...
Avv's user avatar
  • 231
0 votes
2 answers
7k views

NameError: name 'model' is not defined Keras with f1_score

I'm having a problem with my Keras model, in the .compile() I use accuracy, loss, precision, recall and AUC, but also I need f1_score, due to Keras doesn´t include f1_score, I tried to calculate by ...
megasaw's user avatar
  • 13
0 votes
1 answer
4k views

problem with using f1 score with a multi class and imbalanced dataset - (lstm , keras)

I'm trying to use f1 score because my dataset is imbalanced. I already tried this code but the problem is that val_f1_score is always equal to 1. I don't know if I did it correctly or not. my X_train ...
Soroosh Moghimi's user avatar
1 vote
1 answer
800 views

Perfect scores for multiclass classification

I am working on a multiclass classification problem with 3 (1, 2, 3) classes being perfectly distributed. (70 instances of each class resulting in (210, 8) dataframe). Now my data has all the 3 ...
spectre's user avatar
  • 2,105
0 votes
2 answers
1k views

Question answering bot: EM>F1, does it make sense?

I am fine-tuning a Question Answering bot starting from a pre-trained model from HuggingFace repo. The dataset I am using for the fine-tuning has a lot of empty answers. So, after the fine tuning, ...
SilentCloud's user avatar
1 vote
1 answer
355 views

How to measure multi-label multi-class accuracy

I have a model that has multi-label multi-class targets Example Age Height Weight Mark Distance Red Yellow Green Blue Black White 14 160 62 78 103 0 1 1 1 1 0 56 177 90 99 363 1 1 0 0 0 0 32 179 ...
asmgx's user avatar
  • 549
0 votes
0 answers
17 views

How can I Determine a Treshold According to the Precision and Recall?

I am gettin these precision and recall values from my classifier and I want to determine a treshold for the test data. How can I determine that treshold? Is these values enough or something else is ...
TarabydaVllasıCafcaflıAtArabsı's user avatar
1 vote
1 answer
47 views

How to improve my f1 score in stories analyze

I got an assignment to build a model that identify the gender of the text writer. The assignment score will determine by my model f1_score, to get the maximum points, T need it will be at least 0.7. I'...
Bar Benezri's user avatar
2 votes
1 answer
189 views

Making an ensemble model for high F1 score

I presently have 2 algorithms that have a numerical output. Using a threshold of 0.9, I get the classification output. Let's say they are: P (high precision, low recall) R (high recall, low precision)...
Kanishk Mair's user avatar
1 vote
2 answers
1k views

Accuracy on Validation and Test set, Overfit?

Just a quick question: I am building an ML model right now, however, I am receiving very similar (72.2 and 72.4 for example)% for both Accuracy and F1-Score on my Validation Dataset and my unseen Test ...
Dylan Maguire's user avatar
1 vote
1 answer
997 views

How to explain a relationship between Accuracy and F1 Score / F-Measure?

I am building a CNN model for pitch estimation using a song recording. Pitch estimation is done by inputting spectrogram to CNN model and make the CNN predict pitch sequence (250 pitch values per ...
Dionisius Pratama's user avatar
1 vote
1 answer
643 views

How to compute f1_score for multiclass multilabel classification

I have used one hot encoder [1,0,0][0,1,0][0,0,1] for my functional classification model. The predicted probabilities for test data ...
Kyv's user avatar
  • 151
0 votes
2 answers
1k views

Warning when plotting confusion matrix with all sample of one class

I have two arrays: the first one with all the correct labels (they are all set to zero since each sample belong to the same class) and another one with all the labels predicted by my neural network. ...
Fabio's user avatar
  • 53
0 votes
1 answer
184 views

Which F1-score is used for the semantic segmentation tasks?

I read some papers about state-of-the-art semantic segmentation models and in all of them, authors use for comparison F1-score metric, but they did not write whether they use the "micro" or &...
Panicum's user avatar
  • 111
3 votes
1 answer
2k views

scikit-learn classification report's f1 accuracy?

When I run scikit-learn classification_report() on my 2-class y and yhat, I get the ...
stackoverflowuser2010's user avatar
2 votes
1 answer
1k views

What is the appropriate statistical significance test for multi-class classification?

I have a multi-class classification problem. I am primarily using macro-average F1 measure to evaluate the performance of models and want to verify if the results are statistically significant. I have ...
Ashwin Geet D'Sa's user avatar
1 vote
3 answers
581 views

Too high performances on a classification problem

I have a .json file as dataset of the type: and I am working on a classification problem in which I have to predict 4 classes, which are rhe semantic. I have worked through the problem, and after ...
J.D.'s user avatar
  • 871
6 votes
4 answers
1k views

Is an $F_1$ score of 0.1 always bad?

I'm currently building a model to predict early mortgage delinquency (60+ days delinquent within 2 years of origination) for loans originating in 2018Q1. I will eventually train out-of-time (on loans ...
antsatsui's user avatar
  • 233
-1 votes
1 answer
67 views

Model to choose with Cross Validation or not?

I made different tests on an imbalanced dataset and got these results: Model 1 = train test validation split + Cross Validation(cv=10) --> f1'micro' 0,95 Model 2 = train test split + smote method ...
martin's user avatar
  • 329
1 vote
1 answer
530 views

how print f1-score with scikit´s accuracy_score or accuracy of confusion_matrix?

I would like to print the f1-score. I got confused about the wording f1-accuracy score and accuracy score. What is the difference of these 2 scikit-learn metrics and how can I print the f1-score out ...
martin's user avatar
  • 329
2 votes
1 answer
3k views

Multiclass Classification and log_loss

I hope I can make this clear with few lines of code/explanation. I've a 16K list of texts, labelled over 30 different classes that were ran through different classifiers; my Prediction and the Ground ...
Andrea Moro's user avatar
6 votes
3 answers
3k views

Accuracy is lower than f1-score for imbalanced data

For a binary classification, I have a dataset with 55% negative label and 45% positive labels. The results of the classifier shows that the accuracy is lower than the f1-score. Does that mean that the ...
ds_newbie's user avatar
0 votes
1 answer
502 views

Selecting threshold for F1 Score

When selecting a probability threshold to maximize the F1 score prior to deploying a model (based on the precision-recall curve), should the threshold be selected based on the training or holdout ...
thereandhere1's user avatar
0 votes
1 answer
72 views

How to calculate the evaluation metrics (i.e., F1 score) in leave one subject out cv when a subject belongs to single class only

I have dataset of 10 subjects. the dataset has 4 classess. 0,1,2 and 3. The distribution of classes are not same. For example subject 1 does not have 1,2 and 3. It belongs to zeros class. currently ...
ash 's user avatar
  • 1
1 vote
1 answer
34 views

Is it conscientious to use a threshold for a model output in order to play on the recall and precision?

I have just finished reading an article about the F1 score, recall and precision. Everything was clear except the fact that the author, in his example (see https://towardsdatascience.com/beyond-...
Kaharon's user avatar
  • 113
3 votes
1 answer
123 views

F1 score graph skewed

The following code ...
E199504's user avatar
  • 605
13 votes
1 answer
9k views

Balanced Accuracy vs. F1 Score

I've read plenty of online posts with clear explanations about the difference between accuracy and F1 score in a binary classification context. However, when I came across the concept of balanced ...
Ric S's user avatar
  • 267
1 vote
0 answers
1k views

How can I calculate the F1 score using Mask RCNN?

I customized the "https://github.com/matterport/Mask_RCNN.git" repository to train with my own data set, for object detection, ignoring the mask segmentation part. Now I am evaluating my results, I ...
Williana Sousa's user avatar
0 votes
1 answer
976 views

Confusion matrix of UNET image sgemenation model

I have used Unet model for image segmentation. I have used RGB images and corresponding image masks and at output i got corresponding region of interest. Now i want to find confusion matrix of this ...
Manvir Kaur's user avatar
1 vote
0 answers
904 views

Calculating the F score of Object Detection of Mask RCNN

I am using Detectron2 Mask RCNN for an object detection problem. The images consist of cells that are very close to each other. I can not use mAP as a performance measure since the annotations are a ...
ahsan mukhtar's user avatar
12 votes
4 answers
16k views

Can the F1 score be equal to zero?

As it is mentioned in the F1 score Wikipedia, 'F1 score reaches its best value at 1 (perfect precision and recall) and worst at 0'. What is the worst condition that was mentioned? Even if we ...
akhil penta's user avatar
-1 votes
1 answer
114 views

What is evaluation metric for two sets? [closed]

I've two sets one is ground truth and other is output of my machine learning models. Assume my groundtruth set is A={1,2,3,4,5} and output of machine learning model is B={3,4,5,6,7,8}. One way I can ...
Shahrear Bin Amin's user avatar
1 vote
0 answers
26 views

Monotonicity of Jaccard and Dice in multilabel datasets

I understand that Jaccard and Dice follow a monotonic relation on binary datasets because the two are related as $J = {S \over {(2 - S)}}$, and I guess this would be the case when micro-average is ...
VSR's user avatar
  • 11
0 votes
1 answer
334 views

Different definitions of Macro F1 score, which one is used in Scikit-learn?

In this article Macro F1 and Macro F1 two different definitions of the F1 used in the literature are demonstrated. The first F1 score is computed such as: F1 scores are computed for each class ...
Felix Z.'s user avatar
  • 172