25
votes
Accepted
Train Accuracy vs Test Accuracy vs Confusion matrix
Definitions
Accuracy: The amount of correct classifications / the total amount
of classifications.
The train accuracy: The accuracy of a model on examples it was constructed on.
The test accuracy ...
- 1,354
17
votes
Accepted
How to get predictions with predict_generator on streaming test data in Keras?
To get a confusion matrix from the test data you should go througt two steps:
Make predictions for the test data
For example, use model.predict_generator to ...
- 961
13
votes
Confusion Matrix three classes python
Multi-class Confusion Matrix is very well established in literature; you could find it easily on your own. Anyhow, Scikit-learn can do it easily like:
...
- 4,207
11
votes
Accepted
Confusion Matrix - Get Items FP/FN/TP/TN - Python
Considering you have two lists y_actual and y_pred ( I assume you made a typo error on x_test and x_pred as in your code), you can pass the two lists to this function to parse them
...
- 777
8
votes
Accepted
How can I make big confusion matrices easier to read?
You can apply a technique I described in my masters thesis (page 48ff) and called Confusion Matrix Ordering (CMO):
Order the columns/rows in such a way, that most errors are along the diagonal.
Split ...
- 18.8k
7
votes
Should I oversample my validation data to get better F1 score and PRC?
what you encounter are real-world problems rarely taught in classes.
For training, I would test SKLearn's class_weight = "balanced" or class_weight={0:0.995, 1:0.005}. It's a very robust ...
- 1,057
7
votes
Accepted
Inverse Relationship Between Precision and Recall
If we decrease the false negative (select more positives), recall always increases, but precision may increase or decrease. Generally, for models better than random, precision and recall have an ...
- 9,227
7
votes
Accepted
True positives and true negatives, F1 score: multi class classification
In a multiclass problem there is one score for each class, counting any other class as a negative.
For example for class 1:
TP instances are gold standard class 1 predicted as class 1
FN instances ...
- 25k
7
votes
Can the F1 score be equal to zero?
F1 will never be zero, but very near to zero for a bad classifier. If TP or TN is zero then there isn't any need to check F1.
- 267
6
votes
Accepted
The Area Under an ROC Curve (AUC) vs Confusion Matrix for classifier evaluation?
A confusion matrix can be used to measure the performance of a particular classifier with a fixed threshold. Given a set of input cases, the classifier scores each one, and score above the threshold ...
- 1,241
5
votes
Performance Measures
Maybe not the answer you would like to hear, but I have to admit that I find wikipedia page on the topic definitively well done...
https://en.wikipedia.org/wiki/Precision_and_recall
- 185
5
votes
Accepted
How to make sense of confusion matrix
Adding to the answer above,
The labeling totally depends on how you define it. You can define 0 as negative or as positive. However, for the sake of understanding and ease of readability, keep it ...
- 704
5
votes
Is it possible to find a model that minimises both false positive and false negative?
Yes and No! depending on what do you mean by minimization. When you say minimizing $f$ and $g$ according to something, you are actually looking for a point which minimizes both. It does not mean that ...
- 6,532
5
votes
Accepted
Continuous variable not supported in confusion matrix
The confusion matrix is used to tell you how many predictions were classified correctly or incorrectly. You are looking at a regression model, which gives you a continous output (not classification).
...
- 14.7k
5
votes
Confusion matrix - determine the values of FP FN TP and TN
You can!
The trick is that you actually know two other critical variables: the number of positive and negative examples (P and N). You can then use them to algebraically solve for the confusion ...
- 2,522
5
votes
Can the F1 score be equal to zero?
It's a mistake on Wikipedia.
$F_{1}$ as the harmonic mean is defined only at positive real numbers. $PRE$ or $REC$ could be equal 0 in case $TP=0$. Which provides to undefined result $F_1=\frac{0}{0}...
- 1,365
5
votes
Accepted
How to create a confusion matrix for one node of a decision tree?
CM is about True Vs Predicted.
Since only one node is in the discussion which means we will have only one column of "Predicted" but all the possible rows of "True"
Let's assume 100 ...
- 5,524
4
votes
Python plot for confusion matrix similar to confusion wheel?
I would strongly recommend against using a confusion wheel to visualize your confusion matrix. As impressive and fancy as they look, confusion wheels are visually complicated and unintuitive to read. ...
4
votes
Confusion Matrix - Get Items FP/FN/TP/TN - Python
Create a method that does the printing for you:
...
- 652
4
votes
Precision and Recall if not binary
sklearn.metrics.classification_report provides precision and recall for all classes along with F-score and support. It might prove to be helpful in your case of 3 classes.
- 704
4
votes
Confusion matrix logic
A confusion matrix is a table that is often used to describe the performance of a classification model. The figure you have provided presents a binary case, but it is also used with more than 2 ...
- 2,190
4
votes
Confusion matrix logic
Seems like you understand the meaning of the confusion matrix, but not the logic used to name its entries!
Here are my 5 cents:
The names are all of this kind:
...
4
votes
Inverse Relationship Between Precision and Recall
Thanks for clear statement of the problem. The point is that if you want to decrease false negatives, you should sufficiently lower the threshold of your decision function. If the false negatives are ...
- 1,237
4
votes
Accuracy is lower than f1-score for imbalanced data
I'll try to answer this with a couple examples:
Say we have 100 instances (55 negative, 45 positive). Let's say we predict 1/45 positives and 55/55 negatives correctly. Then our accuracy is 0.56 but ...
- 2,522
4
votes
Accepted
Confusion matrix to check results
A confusion matrix is indeed a very useful way to analyze the results of your experiment. It provides the exact number (or percentage) of instances with true class X predicted as class Y for all the ...
- 25k
4
votes
Accepted
How to create a confusion matrix for k-means with two features?
The question doesn't mention it clearly but apparently the goal is to detect outliers, in this case defined as instances with "anything over 200 in the Total TCP column". So every instance ...
- 25k
4
votes
How to read confusion matrix from multiclass dataset?
For each element in the test set, we know its actual class (true class) and the class our classifier predicts (predicted class).
In a binary classification problem, we have only two classes: positive ...
- 23.7k
3
votes
How to get predictions with predict_generator on streaming test data in Keras?
Here is some code I tried and worked for me:
...
- 31
3
votes
Confusion matrix logic
Please find the below:
False Negative (FN): prediction is NEGATIVE, actual outcome is POSITIVE, result is 'False Negative' - Why is that? Shouldn't it be 'False Positive'?
Answer : The predictive ...
- 1,441
3
votes
Inverse Relationship Between Precision and Recall
You are correct @Tolga, both can increase at the same time. Consider the following data:
Prediction | True Class
1.0 | 0
0.5 | 1
0.0 | 0
If ...
- 1,922
Only top scored, non community-wiki answers of a minimum length are eligible