4
$\begingroup$

Is there a fundamental difference between building a set on N logistic regressions in 1 vs all fashion, as compared to training a single mutlinomial logistic regression? Put another way, are there any optimization techniques that treat the 1 to N classes logistic regression problem in a way that's markedly different from N independent regressions?

Intuitively is seems like the answer ought to be yes, since there should be a lot of information sharing between various problems if two classes are similar. But since I'm not entirely well versed on how common 1 to N solvers actually work, I can't tell if I'm right or these problems are treated in ways that fundamentally the same.

I think that we can see that there could be a difference between the two models, but I'm not entirely sure. Googling the matter revealed a few arcane discussing about the subject, but I was unable to find an authoritative discussion of the matter.

$\endgroup$
1
$\begingroup$

Quote from Alan, Agresti. "Categorical data analysis." A John Wiley and Sons, Inc. Publication, Hoboken, New Jersey, USA (2002), page 273:

The separate-fitting estimates differ from the ML estimates for simultaneous fitting of the J-1 logits. They are less efficient, tending to have larger standard errors. However, Begg and Gray 1984 showed that the efficiency loss is minor when the response category having highest prevalence is the baseline.

| improve this answer | |
$\endgroup$
  • $\begingroup$ Thanks for that. I've actually alraedy come across that quote. That's what I meant by "arcane". This is an assertion. I'm trying to understand what's the differences between the too approaches and when they might be expressed. $\endgroup$ – Uri Merhav Nov 11 '15 at 20:54
  • $\begingroup$ @UriMerhav That would have been nice to say so in the question. $\endgroup$ – Franck Dernoncourt Dec 11 '15 at 2:55

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.