Dear DataScience Community, I am working on class imbalance tabular data with high-dimension inputs. The tabular data is derived from the satellite data pixels, and I have inflated the train data dimension with derived indices calculated from available data. The accuracy measured with the F1 score is very low for the classes with low data counts. I've attached the sample data size.
I used MLP, CatBoostClassifier, and SVM for the task, with CatBoostClassifier giving me the best result. To address the data classification accuracy, I implemented oversampling, undersampling(losing a large chunk of features of classes with higher count), SMOTE, GSMOTE, and an algorithmic approach where I assigned weight during the model training part in CatBoostClassifier with
class_weights parameter. The best result is when I used the algorithmic approach assigning the
Still, the class with higher samples has higher F1 scores, and the performance of my model is not good in the undersampled classes. Gone through a lot of literature but didn't come to a valid conclusion on a better approach to deal with this level of imbalance and on classification model and evaluation metric selection too.
Next, I am planning to divide the data into chunks and train on a different model, and use a voting approach to classify and update the status after completion.
So, I would love to hear how the datascience community has tackled the situation to deal with this kind of data imbalance. Also, idea of any deep learning method suitable for the challenge?