Logistic regression is generally performed if there are 2 categories in outcome variables. I just tried it for iris dataset with species as y variable which has 3 categories. I used following code:
import pandas as pd
import matplotlib.pylab as plt
from sklearn.linear_model import LogisticRegression
clf = LogisticRegression()
from sklearn import datasets
iris = datasets.load_iris()
clf.fit(iris.data, iris.target)
logcoefdf = pd.DataFrame(data=clf.coef_,
columns=["SL", "SW", "PL", "PW"],
index=['setosa','versicolor','virginica'])
print(logcoefdf)
logcoefdf.plot.bar()
plt.show()
The printout and plot of coefficients is as follows:
SL SW PL PW
setosa 0.414988 1.461297 -2.262141 -1.029095
versicolor 0.416640 -1.600833 0.577658 -1.385538
virginica -1.707525 -1.534268 2.470972 2.555382
(I have labelled rows by names of species but I am not sure if this is correct).
From above output I get following plot:
What is the interpretation of these results? Does it mean that petal length (PL) is lowest in setosa and highest in virginica group? And both sepal width and petal width are less in versicolor species? Thanks for your insight.
Edit: If I use only 2 categories of iris dataset, I get only one set of coefficients:
clf.fit(iris.data[0:100,:], iris.target[0:100])
print(clf.coef_)
Output:
[[-0.40731745 -1.46092371 2.24004724 1.00841492]]
Is it that Logistic regression is being performed for all possible combinations of categories, i.e. setosa vs versicolor, versicolor vs virginica and virginica vs setosa?