Skip to main content

Questions tagged [confusion-matrix]

A confusion matrix is a special contingency table used to evaluate the predictive accuracy of a classifier. Predicted classes are listed in rows and actual classes in columns, with counts of respective cases in each cell.

Filter by
Sorted by
Tagged with
21 votes
3 answers
78k views

How to get predictions with predict_generator on streaming test data in Keras?

In the Keras blog on training convnets from scratch, the code shows only the network running on training and validation data. What about test data? Is the validation data the same as test data (I ...
pseudomonas's user avatar
  • 1,042
16 votes
1 answer
81k views

Train Accuracy vs Test Accuracy vs Confusion matrix

After I developed my predictive model using Random Forest I get the following metrics: ...
Pedro Alves's user avatar
14 votes
3 answers
13k views

How can I make big confusion matrices easier to read?

I have recently published a dataset (link) with 369 classes. I ran a couple of experiments on them to get a feeling for how difficult the classification task is. Usually, I like it if there are ...
Martin Thoma's user avatar
12 votes
4 answers
16k views

Can the F1 score be equal to zero?

As it is mentioned in the F1 score Wikipedia, 'F1 score reaches its best value at 1 (perfect precision and recall) and worst at 0'. What is the worst condition that was mentioned? Even if we ...
akhil penta's user avatar
11 votes
3 answers
9k views

Inverse Relationship Between Precision and Recall

I made some search to learn precision and recall and I saw some graphs represents inverse relationship between precision and recall and I started to think about it to clarify subject. I wonder the ...
tkarahan's user avatar
  • 442
10 votes
2 answers
8k views

How to get an aggregate confusion matrix from n different classifications

I want to test the accuracy of a methodology. I ran it ~400 times, and I got a different classification for each run. I also have the ground truth, i.e., the real classification to test against. For ...
gc5's user avatar
  • 879
9 votes
9 answers
59k views

Confusion Matrix - Get Items FP/FN/TP/TN - Python

After run my python code: print(confusion_matrix(x_test, x_pred)) I get this: [100 32 211 21] My question is how can I get the following list: True positive = ...
John_Rodgers's user avatar
6 votes
4 answers
807 views

Confusion matrix logic

Can someone explain me the logic behind the confusion matrix? True Positive (TP): prediction is POSITIVE, actual outcome is POSITIVE, result is 'True Positive' - No questions. False Negative (FN): ...
Tauno's user avatar
  • 799
6 votes
3 answers
3k views

Accuracy is lower than f1-score for imbalanced data

For a binary classification, I have a dataset with 55% negative label and 45% positive labels. The results of the classifier shows that the accuracy is lower than the f1-score. Does that mean that the ...
ds_newbie's user avatar
6 votes
1 answer
241 views

Kappa From Combined Confusion Matrices

I am trying to evaluate and compare several different machine learning models built with different parameters (i.e. downsampling, outlier removal) and different classifiers (i.e. Bayes Net, SVM, ...
Robert Bixler's user avatar
5 votes
1 answer
34k views

Confusion Matrix three classes python

I want to calculate: True_Positive, False_Positive, False_Negative, True_Negative for three categories. I used to have two classes ...
FUN_'s user avatar
  • 51
5 votes
2 answers
5k views

True positives and true negatives, F1 score: multi class classification

I have 4 classes for an application of classification of animal kingdom: 1 --> invertibrates; 2 --> vertibrates; 3--> mammal; 4 ---> ambhibian. Given a mixture of images the objective is to identify ...
Sm1's user avatar
  • 541
5 votes
3 answers
6k views

Precision and Recall if not binary

I have to calculate precision and recall for a university project to measure the quality of the classification output (with sklearn). Say this would be my results: y_true = [0, 1, 2, 1, 1] y_pred = [...
solaire's user avatar
  • 153
5 votes
1 answer
158 views

Suitable metric choice for imbalanced multi-class dataset (classes have equal importance)

What type of metrics I should use to evaluate my classification models, given that I have two imbalanced multi-class datasets (21 and 16 classes, respectively) where all classes have equal importance? ...
Dave's user avatar
  • 248
5 votes
1 answer
1k views

Dealing with unbalanced error rate in confusion matrix

Here is the confusion matrix I got when I was playing with Forest Type Cover Kaggle dataset : Link. In the matrix, light color and higher numbers represent higher error rates, so as you can see, ...
Chenxiong Yi's user avatar
4 votes
2 answers
598 views

How to make sense of confusion matrix

Consider a binary classification problem with 0 labels denoting normal and 1 abnormal or rare. The number of instances with <...
Srishti M's user avatar
  • 471
4 votes
1 answer
10k views

The Area Under an ROC Curve (AUC) vs Confusion Matrix for classifier evaluation?

When should I use The Area Under an ROC Curve (AUC) or the Confusion Matrix for classifier evaluation? The clasifier evaluation is for example the prediction of customers for possible future sales.
Rene B.'s user avatar
  • 369
4 votes
2 answers
880 views

What is done first, cross validation or grid search?

When I have the data set to train a model with SVM, which procedure is performed first, cross validation or grid search? I have read this in a couple of books but I don't know in what order all this ...
SRG's user avatar
  • 43
4 votes
1 answer
556 views

How to create a confusion matrix for k-means with two features?

I have the need to do a confusion matrix for data run through k-means with two features. I am aware that this is a clustering algorithm and not a classification algorithm but I have seen some ...
Colin Crook's user avatar
4 votes
1 answer
277 views

Error rate of a class from confusion matrix

My professor gives a multiclass confusion matrix and asks for the error rate of a certain class. Unfortunately, the professor refuses to give a definition. I think the closest value to an error rate ...
Pippo's user avatar
  • 143
4 votes
1 answer
132 views

Can Expectation Maximization estimate truth and confusion matrix from multiple noisy sources?

Suppose we have $m$ sources, each of which noisily observe the same set of $n$ independent events from the outcome set $\{A,B,C\}$. Each source has a confusion matrix, for example for source $i$: $$...
Brendan Hill's user avatar
4 votes
2 answers
448 views

Does it make sense to build a ROC for a decision tree where there are multiple threshold you can adjust?

I understand building a ROC curve when the output is a probability, say, from a logistic regression model. You can build a ROC curve by varying the cutoff threshold. But what about decision trees of ...
Paul's user avatar
  • 171
4 votes
1 answer
483 views

How to best estimate the coefficients of a confusion matrix in case of strong class imbalance?

I have a trained binary classifier (forget about how this was trained and think of it as a magical black box) and I would like to measure its classification performance (e.g. compute a confusion ...
Davide Fiocco's user avatar
3 votes
1 answer
600 views

How to define confusion matrix for classification?

Below is the dataset where the response variable is play with two labels (yes, and no): ...
Xuan Dung's user avatar
  • 153
3 votes
1 answer
1k views

Should I oversample my validation data to get better F1 score and PRC?

I am currently working with a dataset that is imbalanced, about 30k rows * 14 features (just for you know), and 99.5% of the data is labeled 0. Since the model is strongly imbalanced I decided to use ...
Frank Xu's user avatar
3 votes
2 answers
6k views

How to read confusion matrix from multiclass dataset?

I have a dataset with multi class for classification. After train and test, tried to plot with confusion matrix. And I found it really different with dataset with simple label true false or yes no. So ...
yozawiratama's user avatar
3 votes
3 answers
1k views

How are scores calculated for each class of binary classification

The formula for Precision is TP / TP + FP, but how to apply it individually for each class of a binary classification problem, For example here the precision, recall and f1 scores are calculated for ...
Jainam Shroff's user avatar
3 votes
1 answer
1k views

How to create a confusion matrix for one node of a decision tree?

I am doing past papers for my data science exam and was curious about one of the questions. They ask us to create a confusion matrix by hand for one node of a decision tree. I understand how to create ...
Matthew Coudert's user avatar
3 votes
1 answer
36k views

Continuous variable not supported in confusion matrix

I am using linear regression algorithm for a data set. And trying to compute confusion matrix between y_pred and y_test. I am getting ...
Harshith's user avatar
  • 283
3 votes
2 answers
1k views

Precision, recall and importance of them in the imbalance problem

I have a test dataset. The dataset is an imbalanced dataset. The total training instances for the dataset is 543 among them minority class(yes) is 75 and the majority class(No) is 468. The class of ...
Encipher's user avatar
  • 361
3 votes
1 answer
3k views

Python plot for confusion matrix similar to confusion wheel?

I have a confusion matrix with 7 classes and would like to represent the matrix in a graph. Something similar like a confusion wheel. Mainly I need to show, the correct observations in each class and ...
chmodsss's user avatar
  • 1,964
3 votes
1 answer
1k views

Confusion matrix in sklearn

If you look at this: ...
luky's user avatar
  • 133
3 votes
2 answers
922 views

What does the color coding and normalized values in confusion matrix actually specify?

I am unable to infer anything about the model from the following confusion matrix. What is the color coding actually specifying? For example, when predicted label is 1 and true label is 1, the value ...
Amanda's user avatar
  • 153
3 votes
1 answer
444 views

how can I replicate working of Multi Label Binarizer from sklearn package in R? [closed]

I want to achieve same working of MultiLabelBinarizer from sklearn.preprocessing package in R. I have list of labels for each example (for Predicated and Actual) like below. ...
Nitin Desai's user avatar
3 votes
1 answer
1k views

Confusion matrix in multilabel classification of an object in more than one class simultaneously

Regarding a classification problem where for example given an image which depicts a human and we are trying to predict their stance and their behavior. For example Human 1: 'Sitting' and 'Eating' in ...
Dimimal13's user avatar
3 votes
3 answers
3k views

How do I calculate the range of a F1-score from a confusion matrix of 3 class,A,B,C

Is there any support function to calculate the average F1-score range?
Bhabesh Roy's user avatar
2 votes
2 answers
1k views

Some simple questions about confusion matrix and metrics in general

I will first tell you about the context then ask my questions. The model detects hate speech and the training and testing datasets are imbalanced (NLP). My questions: Is this considered a good model? ...
Maxi's user avatar
  • 89
2 votes
2 answers
318 views

Confusion matrix to check results

I am a new user in StackExchange and a new learner of Data Science. I am working on better understanding how to estimate the results collected, specifically fake users extracted from a dataset running ...
user105599's user avatar
2 votes
1 answer
1k views

Is it possible to find a model that minimises both false positive and false negative?

Is it possible to come up with a model that minimises both false positive and false negative? Minimising can be done to a point, such as the Bayes error threshold.
vkj's user avatar
  • 31
2 votes
2 answers
1k views

How to balance sensitivity(sn) and specificity(sp) of an Artificial Neural Network model?

I have been working on a binary classification problem of protein sequences. I have used a feed-forward neural network with two hidden layers. I have the training and validation accuracy/loss curves ...
Suresh Pokharel's user avatar
2 votes
3 answers
12k views

Calculating RMSE AND R-squared from the confusion matrix

I have my confusion matrix as C.mat 8263 20 39 2 3826 14 43 7 4431 My predicted class labels are Ypred and actual labels are Ytest. Ypred ...
sam venu's user avatar
2 votes
2 answers
502 views

Confusion matrix. "How close I am to the diagonal?". Is there such metric?

I have a question regarding confusion matrices. To start, we discuss the case of multi-class classification so the confusion matrix has dimension, for example 4 times 4, for classification task with 4 ...
Alex P's user avatar
  • 31
2 votes
2 answers
814 views

What is the purpose of a confusion matrix in a classification problem?

I am studying machine learning. After some research I understood that a typical workflow for a classification problem (after having prepared the data) is the following: Split data in test, train and ...
Federico Gentile's user avatar
2 votes
1 answer
1k views

How to calculate the different metrics for multi class classification

My confusion matrix has the following structure: (Predicted) C= ( actual) [TN FP FN TP] How can I calculate the Mathews ...
Sm1's user avatar
  • 541
2 votes
1 answer
140 views

Confusion regarding confusion matrix

I am confused on how to represent the confusion matrix -- where to put the FP and FN. Link1 and Link2 show different confusion matrix for binary classification. The rows represent the actual and ...
Sm1's user avatar
  • 541
2 votes
2 answers
643 views

Is confusion matrix possible in one column

I am performing anomaly detection using K-Means. I am working with only one column, plotting those values and then within this column I am adding some anomalies. My question is if it is possible to ...
E199504's user avatar
  • 605
2 votes
4 answers
2k views

What is sensitivity in confusion matrix?

A confusion matrix is a table that is often used to describe the performance of a classification model (or "classifier") on a set of test data for which the true values are known. The confusion matrix ...
user avatar
2 votes
1 answer
356 views

How to decide optimal threshold for my classification model from FPR, TPR and threshold

I am building my model in Python to classify customer in buyer/ non-buyer category. I used mutiple agorithms for this problem and then after evaluation selecting the best out of all. sklearn package ...
CodeMaster GoGo's user avatar
2 votes
1 answer
766 views

Is it possible to combine two confusion matrices?

Assume I have two different algorithms that tests whether a given image contains a goat or not. I apply these two algorithms to two different datasets and obtain two confusion matrices. Now I want to ...
Levent's user avatar
  • 123
2 votes
1 answer
114 views

Why rejection of a true null hypothesis is called type I error?

I’m comparing two confusion matrices: https://en.wikipedia.org/wiki/Confusion_matrix#Table_of_confusion https://en.wikipedia.org/wiki/Type_I_and_type_II_errors The 2nd is rotated, the Decision is on ...
belz's user avatar
  • 31