Logistic regression if 3 categories in outcome variable

Question

Logistic regression is generally performed if there are 2 categories in outcome variables. I just tried it for iris dataset with species as y variable which has 3 categories. I used following code:

import pandas as pd
import matplotlib.pylab as plt

from sklearn.linear_model import LogisticRegression
clf = LogisticRegression()

from sklearn import datasets
iris = datasets.load_iris() 

clf.fit(iris.data, iris.target)
logcoefdf = pd.DataFrame(data=clf.coef_, 
     columns=["SL", "SW", "PL", "PW"], 
     index=['setosa','versicolor','virginica'])
print(logcoefdf)
logcoefdf.plot.bar()
plt.show()

The printout and plot of coefficients is as follows:

                  SL        SW        PL        PW
setosa      0.414988  1.461297 -2.262141 -1.029095
versicolor  0.416640 -1.600833  0.577658 -1.385538
virginica  -1.707525 -1.534268  2.470972  2.555382

(I have labelled rows by names of species but I am not sure if this is correct).

From above output I get following plot:

What is the interpretation of these results? Does it mean that petal length (PL) is lowest in setosa and highest in virginica group? And both sepal width and petal width are less in versicolor species? Thanks for your insight.

Edit: If I use only 2 categories of iris dataset, I get only one set of coefficients:

clf.fit(iris.data[0:100,:], iris.target[0:100])
print(clf.coef_)

Output:

[[-0.40731745 -1.46092371  2.24004724  1.00841492]]

Is it that Logistic regression is being performed for all possible combinations of categories, i.e. setosa vs versicolor, versicolor vs virginica and virginica vs setosa?

Answer 1

Logistic regression can work on multi-class. Frankly it is not big different from binary classifier. The question is how does it find the coefficients for each class which you had displayed. By default to find the coefficients for a single class, it goes with one vs rest combination. So setosa vs remaining all classes. It gets coefficients iteratively by minimizing cross entropy. So ultimately you will end up with a matrix having a size of n_classes vs features.

Coming to the interpretation part of the question. To interpret the importance of a single feature, you can do it as - Given the coefficients of every other feature to be same (in comparing a class with another) including intercept, the class that has the feature with highest coefficient has greater chance of engulfing the new point. This being said the features having high petal length and width and low sepal length and width contribute good for a data point to belong to class virginica. But it does not make much sense to interpret each coefficient individually. Hope I made some sense.