1
$\begingroup$

I'm working on a disease binary classification problem.

0 = healthy , 1 = not healthy

The disease is a movement disorder that appears on the patient while moving a specific movement. I applied leave-one-out cross-validation to train on all patients except one to test on and so on.

The problem is, some patients don't show class not healthy at all in their datasets. Thus, the evaluation metrics f1-score, precision & recall went down sharply because of those patients.

I tried to use SMOTE oversampling, but it didn't work because it will generate new samples from the data of other patients. Additionally, I tried class-weights, but it didn't work as well, because there is not a minor class in the test set to give it a higher priority.

How can I solve this problem?

$\endgroup$
0
$\begingroup$

Give cross-validation or bootstrapping a shot! Also, for metrics, look at per class performance and macro averaged.

$\endgroup$
1
  • 1
    $\begingroup$ How about stratifying the train-test set? $\endgroup$ – Julio Jesus Sep 1 '20 at 21:52

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.