Difference between multinomia logistic regression and N 1 vs all binary

Question

Is there a fundamental difference between building a set on N logistic regressions in 1 vs all fashion, as compared to training a single mutlinomial logistic regression? Put another way, are there any optimization techniques that treat the 1 to N classes logistic regression problem in a way that's markedly different from N independent regressions?

Intuitively is seems like the answer ought to be yes, since there should be a lot of information sharing between various problems if two classes are similar. But since I'm not entirely well versed on how common 1 to N solvers actually work, I can't tell if I'm right or these problems are treated in ways that fundamentally the same.

I think that we can see that there could be a difference between the two models, but I'm not entirely sure. Googling the matter revealed a few arcane discussing about the subject, but I was unable to find an authoritative discussion of the matter.

Answer 1

Quote from Alan, Agresti. "Categorical data analysis." A John Wiley and Sons, Inc. Publication, Hoboken, New Jersey, USA (2002), page 273:

The separate-fitting estimates differ from the ML estimates for simultaneous fitting of the J-1 logits. They are less efficient, tending to have larger standard errors. However, Begg and Gray 1984 showed that the efficiency loss is minor when the response category having highest prevalence is the baseline.