0
$\begingroup$

I am developing binary classification models to predict a medical condition in my dataset. My results show that both Logistic Regression and Linear SVM consistently outperformed other ML algorithms (SVM, NB, MLP and DT), as can be seen in the following screenshot:

enter image description here

Observing recent research, I found multiple studies and reviews that talk about the phenomenon of machine learning not being superior to logistic regression for clinical prediction models, such as this systematic review of 71 studies: https://pubmed.ncbi.nlm.nih.gov/30763612/.

I would like to understand what it means for LR to outperform other more complex ML algorithms? Does it just indicate the my classes are linearly separable?

Improve this question
$\endgroup$
2
  • $\begingroup$ Does LR indicate Linear Regression or Logistic Regression? Binary classification is normally performed with Logistic Reg. And so, not linear. "Does it just indicate that my classes are linearly separable?" Maybe, no. Note that AUC Linear SVM = AUC LR = 0.79. Do you already check all steps of the graph again? $\endgroup$
    – Student
    Mar 17 at 4:24
  • $\begingroup$ @Student LR indicates Logistic Regression. $\endgroup$
    – sums22
    Mar 20 at 9:42

1 Answer 1

Reset to default
0
$\begingroup$

Clinical trial data is typically collected from a sample population and often has a limited size and number of features. Complex models applied to such data are more likely to overfit, whereas simpler models like LR are less prone to overfitting and can generalize better. Furthermore, the features and the outcome variable may exhibit a predominantly linear relationship, making LR a suitable choice for these types of datasets. The underlying linearity assumption between the features and the outcome variable also contributes to LR's robustness to noise and outliers, which is another factor behind its better performance.

On the other hand, SVM is sensitive to hyperparameter selection. Tuning SVM can be particularly challenging when dealing with small datasets, such as clinical data. It is worth noting that SVM can model non-linear relationships using kernel functions (non-linear functions); however, this capability does not guarantee improved performance if the data's underlying relationship is inherently linear. In some cases, a linear kernel might perform well, but this is not a certainty.

It is important to consider various aspects before drawing any concrete conclusions. Performance may vary depending on the specific problem, the nature of the datasets, and of course the choice of hyperparameters.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.