Questions tagged [confusion-matrix]
A confusion matrix is a special contingency table used to evaluate the predictive accuracy of a classifier. Predicted classes are listed in rows and actual classes in columns, with counts of respective cases in each cell.
178
questions
21
votes
3
answers
78k
views
How to get predictions with predict_generator on streaming test data in Keras?
In the Keras blog on training convnets from scratch, the code shows only the network running on training and validation data. What about test data? Is the validation data the same as test data (I ...
- 1,042
16
votes
1
answer
77k
views
Train Accuracy vs Test Accuracy vs Confusion matrix
After I developed my predictive model using Random Forest I get the following metrics:
...
- 367
14
votes
3
answers
12k
views
How can I make big confusion matrices easier to read?
I have recently published a dataset (link) with 369 classes. I ran a couple of experiments on them to get a feeling for how difficult the classification task is. Usually, I like it if there are ...
- 18.8k
12
votes
4
answers
15k
views
Can the F1 score be equal to zero?
As it is mentioned in the F1 score Wikipedia, 'F1 score reaches its best value at 1 (perfect precision and recall) and worst at 0'.
What is the worst condition that was mentioned?
Even if we ...
- 221
11
votes
3
answers
9k
views
Inverse Relationship Between Precision and Recall
I made some search to learn precision and recall and I saw some graphs represents inverse relationship between precision and recall and I started to think about it to clarify subject. I wonder the ...
- 422
10
votes
2
answers
7k
views
How to get an aggregate confusion matrix from n different classifications
I want to test the accuracy of a methodology. I ran it ~400 times, and I got a different classification for each run. I also have the ground truth, i.e., the real classification to test against.
For ...
- 879
9
votes
9
answers
56k
views
Confusion Matrix - Get Items FP/FN/TP/TN - Python
After run my python code:
print(confusion_matrix(x_test, x_pred))
I get this:
[100 32
211 21]
My question is how can I get the following list:
True positive = ...
- 157
6
votes
4
answers
776
views
Confusion matrix logic
Can someone explain me the logic behind the confusion matrix?
True Positive (TP): prediction is POSITIVE, actual outcome is POSITIVE, result is 'True Positive' - No questions.
False Negative (FN): ...
- 769
6
votes
3
answers
2k
views
Accuracy is lower than f1-score for imbalanced data
For a binary classification, I have a dataset with 55% negative label and 45% positive labels.
The results of the classifier shows that the accuracy is lower than the f1-score.
Does that mean that the ...
- 61
6
votes
1
answer
236
views
Kappa From Combined Confusion Matrices
I am trying to evaluate and compare several different machine learning models built with different parameters (i.e. downsampling, outlier removal) and different classifiers (i.e. Bayes Net, SVM, ...
- 161
5
votes
1
answer
33k
views
Confusion Matrix three classes python
I want to calculate:
True_Positive, False_Positive, False_Negative, True_Negative
for three categories. I used to have two classes ...
- 51
5
votes
3
answers
6k
views
Precision and Recall if not binary
I have to calculate precision and recall for a university project to measure the quality of the classification output (with sklearn).
Say this would be my results:
y_true = [0, 1, 2, 1, 1]
y_pred = [...
- 153
5
votes
1
answer
127
views
Suitable metric choice for imbalanced multi-class dataset (classes have equal importance)
What type of metrics I should use to evaluate my classification models, given that I have two imbalanced multi-class datasets (21 and 16 classes, respectively) where all classes have equal importance?
...
- 238
5
votes
1
answer
1k
views
Dealing with unbalanced error rate in confusion matrix
Here is the confusion matrix I got when I was playing with Forest Type Cover Kaggle dataset : Link.
In the matrix, light color and higher numbers represent higher error rates, so as you can see, ...
- 51
4
votes
2
answers
5k
views
True positives and true negatives, F1 score: multi class classification
I have 4 classes for an application of classification of animal kingdom: 1 --> invertibrates; 2 --> vertibrates; 3--> mammal; 4 ---> ambhibian. Given a mixture of images the objective is to identify ...
- 521
4
votes
2
answers
585
views
How to make sense of confusion matrix
Consider a binary classification problem with 0 labels denoting normal and 1 abnormal or rare. The number of instances with <...
- 469
4
votes
1
answer
9k
views
The Area Under an ROC Curve (AUC) vs Confusion Matrix for classifier evaluation?
When should I use The Area Under an ROC Curve (AUC) or the Confusion Matrix for classifier evaluation? The clasifier evaluation is for example the prediction of customers for possible future sales.
- 369
4
votes
2
answers
769
views
What is done first, cross validation or grid search?
When I have the data set to train a model with SVM, which procedure is performed first, cross validation or grid search? I have read this in a couple of books but I don't know in what order all this ...
- 43
4
votes
1
answer
510
views
How to create a confusion matrix for k-means with two features?
I have the need to do a confusion matrix for data run through k-means with two features. I am aware that this is a clustering algorithm and not a classification algorithm but I have seen some ...
- 53
4
votes
1
answer
249
views
Error rate of a class from confusion matrix
My professor gives a multiclass confusion matrix and asks for the error rate of a certain class. Unfortunately, the professor refuses to give a definition.
I think the closest value to an error rate ...
- 143
4
votes
1
answer
131
views
Can Expectation Maximization estimate truth and confusion matrix from multiple noisy sources?
Suppose we have $m$ sources, each of which noisily observe the same set of $n$ independent events from the outcome set $\{A,B,C\}$. Each source has a confusion matrix, for example for source $i$:
$$...
- 155
4
votes
2
answers
297
views
Does it make sense to build a ROC for a decision tree where there are multiple threshold you can adjust?
I understand building a ROC curve when the output is a probability, say, from a logistic regression model. You can build a ROC curve by varying the cutoff threshold.
But what about decision trees of ...
- 171
4
votes
1
answer
465
views
How to best estimate the coefficients of a confusion matrix in case of strong class imbalance?
I have a trained binary classifier (forget about how this was trained and think of it as a magical black box) and I would like to measure its classification performance (e.g. compute a confusion ...
- 421
3
votes
1
answer
595
views
How to define confusion matrix for classification?
Below is the dataset where the response variable is play with two labels (yes, and no):
...
- 153
3
votes
1
answer
1k
views
Should I oversample my validation data to get better F1 score and PRC?
I am currently working with a dataset that is imbalanced, about 30k rows * 14 features (just for you know), and 99.5% of the data is labeled 0. Since the model is strongly imbalanced I decided to use ...
- 31
3
votes
2
answers
1k
views
How to read confusion matrix from multiclass dataset?
I have a dataset with multi class for classification. After train and test, tried to plot with confusion matrix. And I found it really different with dataset with simple label true false or yes no. So ...
- 131
3
votes
3
answers
878
views
How are scores calculated for each class of binary classification
The formula for Precision is TP / TP + FP, but how to apply it individually for each class of a binary classification problem,
For example here the precision, recall and f1 scores are calculated for ...
3
votes
1
answer
879
views
How to create a confusion matrix for one node of a decision tree?
I am doing past papers for my data science exam and was curious about one of the questions. They ask us to create a confusion matrix by hand for one node of a decision tree.
I understand how to create ...
3
votes
1
answer
34k
views
Continuous variable not supported in confusion matrix
I am using linear regression algorithm for a data set. And trying to compute confusion matrix between y_pred and y_test. I am getting ...
- 253
3
votes
2
answers
437
views
Precision, recall and importance of them in the imbalance problem
I have a test dataset. The dataset is an imbalanced dataset. The total training instances for the dataset is 543 among them minority class(yes) is 75 and the majority class(No) is 468. The class of ...
- 359
3
votes
1
answer
3k
views
Python plot for confusion matrix similar to confusion wheel?
I have a confusion matrix with 7 classes and would like to represent the matrix in a graph. Something similar like a confusion wheel.
Mainly I need to show, the correct observations in each class and ...
- 1,954
3
votes
1
answer
920
views
3
votes
2
answers
850
views
What does the color coding and normalized values in confusion matrix actually specify?
I am unable to infer anything about the model from the following confusion matrix. What is the color coding actually specifying?
For example, when predicted label is 1 and true label is 1, the value ...
- 153
3
votes
1
answer
407
views
how can I replicate working of Multi Label Binarizer from sklearn package in R? [closed]
I want to achieve same working of MultiLabelBinarizer from sklearn.preprocessing package in R. I have list of labels for each example (for Predicated and Actual) like below.
...
- 61
3
votes
1
answer
1k
views
Confusion matrix in multilabel classification of an object in more than one class simultaneously
Regarding a classification problem where for example given an image which depicts a human and we are trying to predict their stance and their behavior. For example Human 1: 'Sitting' and 'Eating' in ...
- 85
3
votes
3
answers
3k
views
How do I calculate the range of a F1-score from a confusion matrix of 3 class,A,B,C
Is there any support function to calculate the average F1-score range?
- 31
2
votes
2
answers
487
views
Some simple questions about confusion matrix and metrics in general
I will first tell you about the context then ask my questions.
The model detects hate speech and the training and testing datasets are imbalanced (NLP).
My questions:
Is this considered a good model?
...
- 89
2
votes
2
answers
300
views
Confusion matrix to check results
I am a new user in StackExchange and a new learner of Data Science. I am working on better understanding how to estimate the results collected, specifically fake users extracted from a dataset running ...
- 155
2
votes
1
answer
1k
views
Is it possible to find a model that minimises both false positive and false negative?
Is it possible to come up with a model that minimises both false positive and false negative?
Minimising can be done to a point, such as the Bayes error threshold.
- 31
2
votes
2
answers
898
views
How to balance sensitivity(sn) and specificity(sp) of an Artificial Neural Network model?
I have been working on a binary classification problem of protein sequences. I have used a feed-forward neural network with two hidden layers. I have the training and validation accuracy/loss curves ...
- 165
2
votes
3
answers
11k
views
Calculating RMSE AND R-squared from the confusion matrix
I have my confusion matrix as C.mat
8263 20 39
2 3826 14
43 7 4431
My predicted class labels are Ypred and actual labels are Ytest. Ypred ...
- 21
2
votes
2
answers
492
views
Confusion matrix. "How close I am to the diagonal?". Is there such metric?
I have a question regarding confusion matrices.
To start, we discuss the case of multi-class classification so the confusion matrix has dimension, for example 4 times 4, for classification task with 4 ...
- 31
2
votes
2
answers
654
views
What is the purpose of a confusion matrix in a classification problem?
I am studying machine learning. After some research I understood that a typical workflow for a classification problem (after having prepared the data) is the following:
Split data in test, train and ...
- 147
2
votes
1
answer
964
views
How to calculate the different metrics for multi class classification
My confusion matrix has the following structure:
(Predicted)
C= ( actual) [TN FP
FN TP]
How can I calculate the Mathews ...
- 521
2
votes
1
answer
133
views
Confusion regarding confusion matrix
I am confused on how to represent the confusion matrix -- where to put the FP and FN. Link1 and Link2 show different confusion matrix for binary classification. The rows represent the actual and ...
- 521
2
votes
2
answers
557
views
Is confusion matrix possible in one column
I am performing anomaly detection using K-Means. I am working with only one column, plotting those values and then within this column I am adding some anomalies.
My question is
if it is possible to ...
- 605
2
votes
4
answers
2k
views
What is sensitivity in confusion matrix?
A confusion matrix is a table that is often used to describe the performance of a classification model (or "classifier") on a set of test data for which the true values are known. The confusion matrix ...
user87246
2
votes
1
answer
327
views
How to decide optimal threshold for my classification model from FPR, TPR and threshold
I am building my model in Python to classify customer in buyer/ non-buyer category. I used mutiple agorithms for this problem and then after evaluation selecting the best out of all.
sklearn package ...
- 768
2
votes
1
answer
637
views
Is it possible to combine two confusion matrices?
Assume I have two different algorithms that tests whether a given image contains a goat or not. I apply these two algorithms to two different datasets and obtain two confusion matrices.
Now I want to ...
- 123
2
votes
1
answer
1k
views
Calibrating probability thresholds for multiclass classification
I have built a network for the classification of three classes. The network consists of a CNN followed by two fully-connected layers. The CNN consists of convolutional layers, followed by batch ...
- 236