We know that there are many different types of classification algorithms. But among the different categories of classification algorithms, which algorithms are suitable for binary classification and which are suitable for more classes, and why?
If you want to be highly literal, logistic regression is excellent for binary classes but completely inappropriate for $3+$ classes. No worries: there is multinomial logistic regression, the theory of which mimics binary logistic regression (one might consider logistic regression to be a special case of multinomial logistic regression). Depending on the sophistication of my audience, I might be comfortable referring to "logistic regression" and leaving it to them to realize that I mean "multinomial" logistic regression when there are $3+$ categories and "binary" logistic regression when there are $2$ categories.
Random forest can do the binary case but also the multiclass case. Ditto for k-nearest neighbors, support vector machines, and neural networks.
I cannot think of a model for binary classes that lacks a multiclass analogue.
The answer is that there is no one answer. The choice of machine learning algorithms - whether for binary or multi-class classification - very much depends upon your data and your application:
- how big is your training set?
- how balanced is the training set?
- how many dependent variables does it contain?
- what is the separability of your two classes on those variables?
- what kind of accuracy/recall do you need?
- Is the result driving life or death decisions, or helping you decide what to eat for afternoon snack?
- Are you doing this once, or will you be developing multiple models?
For example, if you have a simple, linearly separable problem with a small training set, then a linear discriminant solution (Fisher) makes the most sense, but if you have a complex separability boundary with a small training data, then LR, SVMs or decision trees may be better, or if you have a large amount of data you can use neural nets. You should also consider how much time you want to invest in the model. Generally neural nets are a fairly big investment in time, but simpler models, such as LR and KNN can be implemented in Excel.
That said, more information on the data and the application might allow us to provide better suggestions.