I am working on sign language recognition system using HOG and KNN. I have 26 classes of 180 images per class. The dataset was split into 1/3(67%) for tanning and 2/3(33%) testing after feature extraction with HOG. Model achieved recognition accuracy of 95% on testing dataset. But I am not understating the confusion matrix and classification generated. I believed 1/3 (33%) of each class should be 60 images for testing per class. But result the confusion matrix and classification report generated are shared below. Very confusing report. Kindly help. I can see TP of 65 more than class images.
$\begingroup$
$\endgroup$
4
-
$\begingroup$ is it a multiclass problem or at least you mistakenly approached it that way? i.e. is it possible that an image belongs to more than one class? It might be the mistake. you need to share your code $\endgroup$– Kasra ManshaeiJun 11, 2021 at 8:22
-
$\begingroup$ It is a multiclass problem, some images have great similarities though. Could that be the reason? If yes, way out of the problem, please. @KasraManshaei $\endgroup$– seyiniaJun 11, 2021 at 9:35
-
$\begingroup$ Sorry I meant multilabel. if one image has two labels then the support of that sample will be 2. so the total support is due to number of labels and not samples. it means you may have 65 "labels to classify" instead of 60 which is number of samples. the rest i can tell u only if you provide a sample code $\endgroup$– Kasra ManshaeiJun 11, 2021 at 9:46
-
$\begingroup$ I just checked all my classes. Each class is 180 images. I spilt the whole dataset using test data split @KasraManshaei $\endgroup$– seyiniaJun 11, 2021 at 10:26
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
1
This looks completely normal to me: your dataset has 26x180=4680 instances, so the test set should have 4680x0.33=1544.4 instances. According to the classification report it contains 1545 instances, which is consistent with this calculation.
It's important to understand that by default the dataset is split between training and test set randomly across all the instances, without taking their class into account. This means that by chance some of the classes can have a bit more or a bit less than 33% instances in the test set. This is what can be observed in the classification report and it's not a problem.
Sometimes this can be an issue when there are classes which have very few instances in total. In this case one should use stratified sampling in order to apply the proportion to each class independently.
-
$\begingroup$ Thanks, I find your response useful. However, this problem can be solved by manually separate testing dataset from training dataset. Thanks is zillion time @Erwan $\endgroup$– seyiniaJun 11, 2021 at 20:49