1
$\begingroup$

This is my first classification report of a logistic regression

This is my second classification report of a logistic regression

I need some help to interpret the 2 classification reports of the same logistic regression. The only difference between them is the size of test_size. Even though my second classification report has a much higher f1-score, because of its low test_size (0.1), it is an inferior algorithm than my first classification report?

I am not wholly sure how to interpret the relationship between the f1-score and support values. But from what I understand, the support values should all be close to each other and f1-score be as close to 1 as possible. In that case does that make the algorithm I developed a bad one then?

$\endgroup$

2 Answers 2

1
$\begingroup$

You are misusing the idea of "test" dataset in machine learning. A test dataset should only be used once. You are re-using the test dataset and changing the modeling choices based on that re-use. This is an example of data leakage which will lower the generalization performance.

Additionally, the number of samples for the P group (3 and 21 respectively) are too small draw any meaningful conclusions. A small change with that few samples will greatly change the results, as evidence in those two reports. There needs to be much larger sample size to meaningfully apply machine learning techniques.

$\endgroup$
0
$\begingroup$

Support is the number of occurrences of each label in the ground truth. For example in the results with test size=0.1, class P has only 3 samples. Based on this, if the support values are not close to each other, it only means that your data is unbalanced. This documentation might be helpful.

$\endgroup$
1
  • $\begingroup$ So in the test size=0.1 example, for class P, does a support value of 3 means that there are only 3 samples of P in the test sample for my classification algorithm? In other words, if I were to increase test size, the amount of class P in the test samples available for my classification algorithm to train on will also increase? $\endgroup$ Commented Aug 6, 2020 at 1:45

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.