I have dataset of 10 subjects. the dataset has 4 classess. 0,1,2 and 3. The distribution of classes are not same. For example subject 1 does not have 1,2 and 3. It belongs to zeros class. currently I am evaluating the model using leave one subject out.In such scenario how to calculate the evaluation metrics like F1 score. I tried the sklearn classification report but for some subjects there are only one class. shall I consider the macro F1 score?
1 Answer
$\begingroup$
$\endgroup$
1
You should not use summary statistics if you only have 10 data points. With that few datapoints, the summary statistics estimates will not be robust.
-
$\begingroup$ Thanks for the reply. actually, there are 10 subjects. Each subject includes few thousands images. Therefore the total samples are around 16000 images. but the problem is some subjects does not have all the classess (i.e., for subject one there are 2000 images all belongs to 0 class) in such cases how to measure evaluation metrics. $\endgroup$– ashJun 15, 2020 at 14:35