0
$\begingroup$

I am working on a multi-class classifier for data set with 240K samples and ~1880 classes, the most populated class is 4% of the dataset and large number of classes are less then 1% of the samples.

I am trying to find a method to decide for which class I have enough information to correctly predict or to reject the prediction due to classifier performance for that particular class.

Also, are there any good methods to deal with that kind of situations of high multi class with very low sample size?

$\endgroup$
2
$\begingroup$

Yeah. There have been scenarios like that. Imagenet has more than 1000 classes. Same could be said for certain cifar datasets.

If deep learning, go with one-hot encoding for the labels.

Stick with something like multi-class AUC as metric instead of believing on Validation accuracy.

Go with a keras DNN, then you could try advanced architectures, with the last Dense net to have 1880 neurons each representing one label. Also stick with categorical_crossentropy for loss.

Hope this helps.

| improve this answer | |
$\endgroup$
2
$\begingroup$

I think it highly depends on the specific task. I mean if you say you have less then 1% of the images for one class it could be still enough. For example 0.5% of images in one class would still be 1200 images per class which is quiet okay. Also there might be classes which are pretty similiar to each other, so it will be harder and maybe more images are needed to distinct these to classes, while other classes are easy to seperate.

If you ask for methods to overcome this problem there are several. One for example is over- or undersampling. So drop images of classes with higher percentage in dataset, or duplicate images of classes with lower percentage. This is to get all classes into a balanced ratio.

Also agressive augmentation could be used, e.g. flipping, rotating, contrast adaption, shearing, zooming, etc..., just remember: do not destroy the semantic of images.

If you want to test the classifier performance you can still use validation accuracy, but you have to build your validation set with balanced number of classes and these images should be representative for each class! Also the sample size in the validation set must be high enough.

| improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.