0
$\begingroup$

I have dataset of 10 subjects. the dataset has 4 classess. 0,1,2 and 3. The distribution of classes are not same. For example subject 1 does not have 1,2 and 3. It belongs to zeros class. currently I am evaluating the model using leave one subject out.In such scenario how to calculate the evaluation metrics like F1 score. I tried the sklearn classification report but for some subjects there are only one class. shall I consider the macro F1 score?

$\endgroup$

1 Answer 1

0
$\begingroup$

You should not use summary statistics if you only have 10 data points. With that few datapoints, the summary statistics estimates will not be robust.

$\endgroup$
1
  • $\begingroup$ Thanks for the reply. actually, there are 10 subjects. Each subject includes few thousands images. Therefore the total samples are around 16000 images. but the problem is some subjects does not have all the classess (i.e., for subject one there are 2000 images all belongs to 0 class) in such cases how to measure evaluation metrics. $\endgroup$
    – ash
    Jun 15, 2020 at 14:35

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.