25 votes
Accepted

Train Accuracy vs Test Accuracy vs Confusion matrix

Definitions Accuracy: The amount of correct classifications / the total amount of classifications. The train accuracy: The accuracy of a model on examples it was constructed on. The test accuracy ...
  • 1,344
17 votes
Accepted

How to get predictions with predict_generator on streaming test data in Keras?

To get a confusion matrix from the test data you should go througt two steps: Make predictions for the test data For example, use model.predict_generator to ...
  • 961
13 votes

Confusion Matrix three classes python

Multi-class Confusion Matrix is very well established in literature; you could find it easily on your own. Anyhow, Scikit-learn can do it easily like: ...
  • 4,146
11 votes
Accepted

Confusion Matrix - Get Items FP/FN/TP/TN - Python

Considering you have two lists y_actual and y_pred ( I assume you made a typo error on x_test and x_pred as in your code), you can pass the two lists to this function to parse them ...
  • 747
8 votes
Accepted

How can I make big confusion matrices easier to read?

You can apply a technique I described in my masters thesis (page 48ff) and called Confusion Matrix Ordering (CMO): Order the columns/rows in such a way, that most errors are along the diagonal. Split ...
  • 18.2k
7 votes
Accepted

Inverse Relationship Between Precision and Recall

If we decrease the false negative (select more positives), recall always increases, but precision may increase or decrease. Generally, for models better than random, precision and recall have an ...
  • 8,877
7 votes

Can the F1 score be equal to zero?

F1 will never be zero, but very near to zero for a bad classifier. If TP or TN is zero then there isn't any need to check F1.
6 votes
Accepted

The Area Under an ROC Curve (AUC) vs Confusion Matrix for classifier evaluation?

A confusion matrix can be used to measure the performance of a particular classifier with a fixed threshold. Given a set of input cases, the classifier scores each one, and score above the threshold ...
6 votes

Should I oversample my validation data to get better F1 score and PRC?

what you encounter are real-world problems rarely taught in classes. For training, I would test SKLearn's class_weight = "balanced" or class_weight={0:0.995, 1:0.005}. It's a very robust ...
  • 1,007
5 votes

Performance Measures

Maybe not the answer you would like to hear, but I have to admit that I find wikipedia page on the topic definitively well done... https://en.wikipedia.org/wiki/Precision_and_recall
5 votes
Accepted

How to make sense of confusion matrix

Adding to the answer above, The labeling totally depends on how you define it. You can define 0 as negative or as positive. However, for the sake of understanding and ease of readability, keep it ...
  • 684
5 votes

Is it possible to find a model that minimises both false positive and false negative?

Yes and No! depending on what do you mean by minimization. When you say minimizing $f$ and $g$ according to something, you are actually looking for a point which minimizes both. It does not mean that ...
5 votes
Accepted

Continuous variable not supported in confusion matrix

The confusion matrix is used to tell you how many predictions were classified correctly or incorrectly. You are looking at a regression model, which gives you a continous output (not classification). ...
  • 14.3k
5 votes
Accepted

True positives and true negatives, F1 score: multi class classification

In a multiclass problem there is one score for each class, counting any other class as a negative. For example for class 1: TP instances are gold standard class 1 predicted as class 1 FN instances ...
  • 24.4k
5 votes

Confusion matrix - determine the values of FP FN TP and TN

You can! The trick is that you actually know two other critical variables: the number of positive and negative examples (P and N). You can then use them to algebraically solve for the confusion ...
  • 2,472
5 votes

Can the F1 score be equal to zero?

It's a mistake on Wikipedia. $F_{1}$ as the harmonic mean is defined only at positive real numbers. $PRE$ or $REC$ could be equal 0 in case $TP=0$. Which provides to undefined result $F_1=\frac{0}{0}...
  • 1,357
5 votes
Accepted

How to create a confusion matrix for one node of a decision tree?

CM is about True Vs Predicted. Since only one node is in the discussion which means we will have only one column of "Predicted" but all the possible rows of "True" Let's assume 100 ...
  • 5,404
4 votes

Python plot for confusion matrix similar to confusion wheel?

I would strongly recommend against using a confusion wheel to visualize your confusion matrix. As impressive and fancy as they look, confusion wheels are visually complicated and unintuitive to read. ...
4 votes

Confusion Matrix - Get Items FP/FN/TP/TN - Python

Create a method that does the printing for you: ...
  • 652
4 votes

Precision and Recall if not binary

sklearn.metrics.classification_report provides precision and recall for all classes along with F-score and support. It might prove to be helpful in your case of 3 classes.
  • 684
4 votes

Confusion matrix logic

A confusion matrix is a table that is often used to describe the performance of a classification model. The figure you have provided presents a binary case, but it is also used with more than 2 ...
  • 2,150
4 votes

Confusion matrix logic

Seems like you understand the meaning of the confusion matrix, but not the logic used to name its entries! Here are my 5 cents: The names are all of this kind: ...
4 votes

Accuracy is lower than f1-score for imbalanced data

I'll try to answer this with a couple examples: Say we have 100 instances (55 negative, 45 positive). Let's say we predict 1/45 positives and 55/55 negatives correctly. Then our accuracy is 0.56 but ...
  • 2,472
4 votes
Accepted

Confusion matrix to check results

A confusion matrix is indeed a very useful way to analyze the results of your experiment. It provides the exact number (or percentage) of instances with true class X predicted as class Y for all the ...
  • 24.4k
4 votes
Accepted

How to create a confusion matrix for k-means with two features?

The question doesn't mention it clearly but apparently the goal is to detect outliers, in this case defined as instances with "anything over 200 in the Total TCP column". So every instance ...
  • 24.4k
3 votes

How to get predictions with predict_generator on streaming test data in Keras?

Here is some code I tried and worked for me: ...
  • 31
3 votes

Confusion matrix logic

Please find the below: False Negative (FN): prediction is NEGATIVE, actual outcome is POSITIVE, result is 'False Negative' - Why is that? Shouldn't it be 'False Positive'? Answer : The predictive ...
  • 1,431
3 votes

Inverse Relationship Between Precision and Recall

Thanks for clear statement of the problem. The point is that if you want to decrease false negatives, you should sufficiently lower the threshold of your decision function. If the false negatives are ...
  • 1,217
3 votes

Inverse Relationship Between Precision and Recall

You are correct @Tolga, both can increase at the same time. Consider the following data: Prediction | True Class 1.0 | 0 0.5 | 1 0.0 | 0 If ...
  • 1,902
3 votes
Accepted

Confusion matrix - determine the values of FP FN TP and TN

You should modify the code to produce the confusion matrix itself. But assuming that's impossible for some reason... A bit of linear algebra helps here. @n1k31t4 is right that given only accuracy, ...
  • 10.6k

Only top scored, non community-wiki answers of a minimum length are eligible