1
$\begingroup$

I have an imbalanced dataset (2:1 ratio) with about 60 patients and 80 features.

I performed Recursive Feature Elimination (RFE) and stratified cross validation to reduce the features to 15 and I get an AUC of 0.9 with Logistic regression and/or SVM. I don't fully trust the AUC I got because I think it will not generalize correctly because of such a small positive class. So, I was thinking on oversampling (K-means + PCA) the minority class and re-run the RFE approach, would this help? Thanks.

My question is more or less the same as this one: Why will the accuracy of a highly unbalanced dataset reduce after oversampling? but I do use AUC.

$\endgroup$
0
$\begingroup$

The bigger issue might be the small n. With 60 samples and 2:1 ratio, you only have 20 samples in the minority class. Generalization, no matter what machine learning technique is used, will be limited with just 20 samples.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.