Questions tagged [confusion-matrix]

A confusion matrix is a special contingency table used to evaluate the predictive accuracy of a classifier. Predicted classes are listed in rows and actual classes in columns, with counts of respective cases in each cell.

178 questions
Filter by
Sorted by
Tagged with
78k views

How to get predictions with predict_generator on streaming test data in Keras?

In the Keras blog on training convnets from scratch, the code shows only the network running on training and validation data. What about test data? Is the validation data the same as test data (I ...
• 1,042
77k views

Train Accuracy vs Test Accuracy vs Confusion matrix

After I developed my predictive model using Random Forest I get the following metrics: ...
• 367
12k views

How can I make big confusion matrices easier to read?

I have recently published a dataset (link) with 369 classes. I ran a couple of experiments on them to get a feeling for how difficult the classification task is. Usually, I like it if there are ...
• 18.8k
15k views

Can the F1 score be equal to zero?

As it is mentioned in the F1 score Wikipedia, 'F1 score reaches its best value at 1 (perfect precision and recall) and worst at 0'. What is the worst condition that was mentioned? Even if we ...
• 221
9k views

Inverse Relationship Between Precision and Recall

I made some search to learn precision and recall and I saw some graphs represents inverse relationship between precision and recall and I started to think about it to clarify subject. I wonder the ...
• 422
7k views

How to get an aggregate confusion matrix from n different classifications

I want to test the accuracy of a methodology. I ran it ~400 times, and I got a different classification for each run. I also have the ground truth, i.e., the real classification to test against. For ...
• 879
56k views

Confusion Matrix - Get Items FP/FN/TP/TN - Python

After run my python code: print(confusion_matrix(x_test, x_pred)) I get this: [100 32 211 21] My question is how can I get the following list: True positive = ...
• 157
776 views

Confusion matrix logic

Can someone explain me the logic behind the confusion matrix? True Positive (TP): prediction is POSITIVE, actual outcome is POSITIVE, result is 'True Positive' - No questions. False Negative (FN): ...
• 769
2k views

Accuracy is lower than f1-score for imbalanced data

For a binary classification, I have a dataset with 55% negative label and 45% positive labels. The results of the classifier shows that the accuracy is lower than the f1-score. Does that mean that the ...
236 views

Kappa From Combined Confusion Matrices

I am trying to evaluate and compare several different machine learning models built with different parameters (i.e. downsampling, outlier removal) and different classifiers (i.e. Bayes Net, SVM, ...
33k views

Confusion Matrix three classes python

I want to calculate: True_Positive, False_Positive, False_Negative, True_Negative for three categories. I used to have two classes ...
• 51
6k views

Precision and Recall if not binary

I have to calculate precision and recall for a university project to measure the quality of the classification output (with sklearn). Say this would be my results: y_true = [0, 1, 2, 1, 1] y_pred = [...
• 153
127 views

Suitable metric choice for imbalanced multi-class dataset (classes have equal importance)

What type of metrics I should use to evaluate my classification models, given that I have two imbalanced multi-class datasets (21 and 16 classes, respectively) where all classes have equal importance? ...
• 238
1k views

Dealing with unbalanced error rate in confusion matrix

Here is the confusion matrix I got when I was playing with Forest Type Cover Kaggle dataset : Link. In the matrix, light color and higher numbers represent higher error rates, so as you can see, ...
5k views

True positives and true negatives, F1 score: multi class classification

I have 4 classes for an application of classification of animal kingdom: 1 --> invertibrates; 2 --> vertibrates; 3--> mammal; 4 ---> ambhibian. Given a mixture of images the objective is to identify ...
• 521
585 views

How to make sense of confusion matrix

Consider a binary classification problem with 0 labels denoting normal and 1 abnormal or rare. The number of instances with <...
• 469
9k views

The Area Under an ROC Curve (AUC) vs Confusion Matrix for classifier evaluation?

When should I use The Area Under an ROC Curve (AUC) or the Confusion Matrix for classifier evaluation? The clasifier evaluation is for example the prediction of customers for possible future sales.
• 369
769 views

What is done first, cross validation or grid search?

When I have the data set to train a model with SVM, which procedure is performed first, cross validation or grid search? I have read this in a couple of books but I don't know in what order all this ...
• 43
510 views

How to create a confusion matrix for k-means with two features?

I have the need to do a confusion matrix for data run through k-means with two features. I am aware that this is a clustering algorithm and not a classification algorithm but I have seen some ...
249 views

Error rate of a class from confusion matrix

My professor gives a multiclass confusion matrix and asks for the error rate of a certain class. Unfortunately, the professor refuses to give a definition. I think the closest value to an error rate ...
• 143
131 views

Can Expectation Maximization estimate truth and confusion matrix from multiple noisy sources?

Suppose we have $m$ sources, each of which noisily observe the same set of $n$ independent events from the outcome set $\{A,B,C\}$. Each source has a confusion matrix, for example for source $i$: ...
• 155
297 views

Does it make sense to build a ROC for a decision tree where there are multiple threshold you can adjust?

I understand building a ROC curve when the output is a probability, say, from a logistic regression model. You can build a ROC curve by varying the cutoff threshold. But what about decision trees of ...
• 171
465 views

How to best estimate the coefficients of a confusion matrix in case of strong class imbalance?

I have a trained binary classifier (forget about how this was trained and think of it as a magical black box) and I would like to measure its classification performance (e.g. compute a confusion ...
595 views

How to define confusion matrix for classification?

Below is the dataset where the response variable is play with two labels (yes, and no): ...
• 153
1k views

Should I oversample my validation data to get better F1 score and PRC?

I am currently working with a dataset that is imbalanced, about 30k rows * 14 features (just for you know), and 99.5% of the data is labeled 0. Since the model is strongly imbalanced I decided to use ...
• 31
1k views

How to read confusion matrix from multiclass dataset?

I have a dataset with multi class for classification. After train and test, tried to plot with confusion matrix. And I found it really different with dataset with simple label true false or yes no. So ...
• 131
878 views

How are scores calculated for each class of binary classification

The formula for Precision is TP / TP + FP, but how to apply it individually for each class of a binary classification problem, For example here the precision, recall and f1 scores are calculated for ...
879 views

How to create a confusion matrix for one node of a decision tree?

I am doing past papers for my data science exam and was curious about one of the questions. They ask us to create a confusion matrix by hand for one node of a decision tree. I understand how to create ...
34k views

Continuous variable not supported in confusion matrix

I am using linear regression algorithm for a data set. And trying to compute confusion matrix between y_pred and y_test. I am getting ...
• 253
437 views

Precision, recall and importance of them in the imbalance problem

I have a test dataset. The dataset is an imbalanced dataset. The total training instances for the dataset is 543 among them minority class(yes) is 75 and the majority class(No) is 468. The class of ...
• 359
3k views

Python plot for confusion matrix similar to confusion wheel?

I have a confusion matrix with 7 classes and would like to represent the matrix in a graph. Something similar like a confusion wheel. Mainly I need to show, the correct observations in each class and ...
• 1,954
920 views

Confusion matrix in sklearn

If you look at this: ...
• 133
850 views

What does the color coding and normalized values in confusion matrix actually specify?

I am unable to infer anything about the model from the following confusion matrix. What is the color coding actually specifying? For example, when predicted label is 1 and true label is 1, the value ...
• 153
407 views

how can I replicate working of Multi Label Binarizer from sklearn package in R? [closed]

I want to achieve same working of MultiLabelBinarizer from sklearn.preprocessing package in R. I have list of labels for each example (for Predicated and Actual) like below. ...
1k views

Confusion matrix in multilabel classification of an object in more than one class simultaneously

Regarding a classification problem where for example given an image which depicts a human and we are trying to predict their stance and their behavior. For example Human 1: 'Sitting' and 'Eating' in ...
3k views

How do I calculate the range of a F1-score from a confusion matrix of 3 class,A,B,C

Is there any support function to calculate the average F1-score range?
487 views

Some simple questions about confusion matrix and metrics in general

I will first tell you about the context then ask my questions. The model detects hate speech and the training and testing datasets are imbalanced (NLP). My questions: Is this considered a good model? ...
• 89
300 views

Confusion matrix to check results

I am a new user in StackExchange and a new learner of Data Science. I am working on better understanding how to estimate the results collected, specifically fake users extracted from a dataset running ...
• 155
1k views

Is it possible to find a model that minimises both false positive and false negative?

Is it possible to come up with a model that minimises both false positive and false negative? Minimising can be done to a point, such as the Bayes error threshold.
• 31
898 views

How to balance sensitivity(sn) and specificity(sp) of an Artificial Neural Network model?

I have been working on a binary classification problem of protein sequences. I have used a feed-forward neural network with two hidden layers. I have the training and validation accuracy/loss curves ...
11k views

Calculating RMSE AND R-squared from the confusion matrix

I have my confusion matrix as C.mat 8263 20 39 2 3826 14 43 7 4431 My predicted class labels are Ypred and actual labels are Ytest. Ypred ...
• 21
492 views

Confusion matrix. "How close I am to the diagonal?". Is there such metric?

I have a question regarding confusion matrices. To start, we discuss the case of multi-class classification so the confusion matrix has dimension, for example 4 times 4, for classification task with 4 ...
• 31
654 views

What is the purpose of a confusion matrix in a classification problem?

I am studying machine learning. After some research I understood that a typical workflow for a classification problem (after having prepared the data) is the following: Split data in test, train and ...
964 views

How to calculate the different metrics for multi class classification

My confusion matrix has the following structure: (Predicted) C= ( actual) [TN FP FN TP] How can I calculate the Mathews ...
• 521
133 views

Confusion regarding confusion matrix

I am confused on how to represent the confusion matrix -- where to put the FP and FN. Link1 and Link2 show different confusion matrix for binary classification. The rows represent the actual and ...
• 521
557 views

Is confusion matrix possible in one column

I am performing anomaly detection using K-Means. I am working with only one column, plotting those values and then within this column I am adding some anomalies. My question is if it is possible to ...
• 605
2k views

What is sensitivity in confusion matrix?

A confusion matrix is a table that is often used to describe the performance of a classification model (or "classifier") on a set of test data for which the true values are known. The confusion matrix ...
327 views

How to decide optimal threshold for my classification model from FPR, TPR and threshold

I am building my model in Python to classify customer in buyer/ non-buyer category. I used mutiple agorithms for this problem and then after evaluation selecting the best out of all. sklearn package ...
637 views

Is it possible to combine two confusion matrices?

Assume I have two different algorithms that tests whether a given image contains a goat or not. I apply these two algorithms to two different datasets and obtain two confusion matrices. Now I want to ...
• 123