0
$\begingroup$

My dataset consists of 150 patients where 50 are controls/healthy (negative) and 100 are sick (positive). If I want my model to have high sensitivity at hight specificity, in other words to have low false positive rates, should I correct my model by applying weights to it? Because usually the positive class is the minority class and I see why you need to correct for it but should I in my case?

Thanks

$\endgroup$
1
$\begingroup$

There are two things that might help you:

1) If you use a classifier $f$ that returns a value $f(x)\in[0,1]$ between 0 and 1 instead of a direct class assignment, then you can use a threshold $\theta\in[0,1]$:

$f(x)=\begin{cases} 1,& \text{if } f(x)\geq\theta\\ 0, & \text{else } \end{cases}$

The higher the value of $\theta$, the more you want to be sure to classify somebody as positive, which means that there will be fewer false positives, but at the same time you will have a fewer sensitivity (recall) because there will be more false negatives.

2) if your classifier always predicts the majority class, you could try oversample the minority class (using SMOTE or other oversampling techniques) to make the training data set balanced.

| improve this answer | |
$\endgroup$
  • $\begingroup$ Hi, I always use probabilities as outputs that is why I asked for high sensitivity at low specificity. My question is should I use weights or not? $\endgroup$ – Luis Pinto Feb 10 at 17:34
  • $\begingroup$ @LuisPinto There are a lot of similar questions here. Most suggest to undersample the majority class or oversample the minority class: link Some suggest to use weights. Classifiers such as SVM allow to set some class weights: link Which classifier do you use? $\endgroup$ – methus Feb 11 at 22:50
  • $\begingroup$ I am using SVM or Logistic regression. All the posts about weights assume that the positive class is the minority which is not my case. Also, I don't want to undersample since the minority class only has 50 patients and that would reduce my dataset to 100 patients which I think it is not enough to train a good model $\endgroup$ – Luis Pinto Feb 12 at 0:47

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.