2
$\begingroup$

We know that neural networks and other learning methods can have better performance relative to logistic regression in some non-linear classification problems. But, it is known too that logistic regression can separate classes with a line that can be curvy if only we add more predictors that are a square, cube, etc of the given predictors (still considered a linear decision boundary).

So my question is, can we theoretically solve any classification problem through logistic regression, or are there limitations that I do not realize that make other non-linear learning methods mandatory to certain classification problems?

$\endgroup$
0
$\begingroup$

There are several assumptions that are needed to be satisfied in order for logistic regression to work. From personal experience the two most important assumptions:

  1. Features need to be not (or a little) correlated with each other, because if they are correlated then the change of one feature indicates also the change of the other that's a problem when you try to solve the optimization problem
  2. May some features not be ordinal

You can also check here for more information.

Other learning methods such as RandomForest require no such assumptions and may work better for some datasets. But keep in mind the no free lunch theorem, most of the times you need to get your hands dirty and test several learning models before you decide which works best for your data.

$\endgroup$
1
  • $\begingroup$ Thank you very much. $\endgroup$
    – joelpires
    Jan 7 '20 at 19:26

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.