0
$\begingroup$

I have a problem where I am to construct a classification variable Yes/No based on another feature's value. We are interested in the Yes class in this case. I am told to use 10-fold cross validation.

The data set contains 384 observations with only 6 Yes classifications. Obviously it is going to be nearly impossible to get any sort of usable model to predict Yes class without leaking the variable the class is based on into the model, but I don't want to do that.

I am using 100% of the dataset as I will import a different dataset to predict on.

When I try to run models in Orange, my evaluation results don't show an AUC for either yes or no, just the average, I have a warning saying some scores can't be computed? Because they are too low/high, number of folds? How I am to interpret this?

My ultimate question is, would this ever be practical in the real world? Would you even try to use this data for prediction of the Yes class?

I have added screenshots below of my test widget as well as my lift curves

enter image description here

enter image description here

enter image description here

enter image description here

enter image description here

Improve this question
$\endgroup$
3
  • $\begingroup$ With just six observations of the minority class, I fear you lack sufficient data to do much of anything. $\endgroup$
    – Dave
    Jul 28, 2022 at 0:50
  • 1
    $\begingroup$ ". . . it is going to be nearly impossible to get any sort of usable model to predict Yes class without leaking the variable the class is based on into the model . . . ". If you already know the variable that predicts the target class, why train a model at all? $\endgroup$
    – zachdj
    Jul 28, 2022 at 0:52
  • 1
    $\begingroup$ @zachdj brings up a good point. It sounds like you know how the classes are generated. $\endgroup$
    – Dave
    Aug 2, 2022 at 14:35

0

Reset to default

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.