Python : How to use Multinomial Logistic Regression using SKlearn

Question

I have a test dataset and train dataset as below. I have provided sample data with min records, but my data has more than 1000's of record. Here if you see E is my target variable which I need to predict using an algorithm. It has only four categories like 1, 2, 3, 4. It can take only any of these values.

Training Dataset:

A    B    C    D    E
1    20   30   1    1
2    22   12   33   2
3    45   65   77   3
12   43   55   65   4
11   25   30   1    1
22   23   19   31   2
31   41   11   70   3
1    48   23   60   4

Test Dataset:

A    B    C    D    E
11   21   12   11
1    2    3    4
5    6    7    8 
99   87   65   34 
11   21   24   12

Since E has only 4 categories, I thought of predicting this using Multinomial Logistic Regression (1 vs Rest Logic). I am trying to implement it using Python.

I know the logic that we need to set these targets in a variable and use an algorithm to predict any of these values:

output = [1,2,3,4]

But I am stuck at a point on how to use it using python (sklearn) to loop through these values and what algorithm should I use to predict the output values? Any help would be greatly appreciated.

Answer 1

Put the training data into two numpy arrays:

import numpy as np

# data from columns A - D
Xtrain = np.array([[1,    20,   30,   1],
                   [2,    22,   12,   33],
                   [3,    45,   65,   77],
                   [12,   43,   55,   65],
                   [11,   25,   30,   1],
                   [22,   23,   19,   31],
                   [31,   41,   11,   70],
                   [1,    48,   23,   60]])

# data from column E
ytrain = np.array([1, 2, 3, 4, 1, 2, 3, 4])

Then train a logistic regression model:

from sklearn.linear_model import LogisticRegression

lr = LogisticRegression().fit(Xtrain, ytrain)

Make predictions (on the training data):

yhat = lr.predict(Xtrain)

=> results in "1, 4, 3, 4, 1, 2, 3, 4".. so it's got 7 right and 1 wrong.

Calculate accuracy:

from sklearn.metrics import accuracy_score

accuracy_score(ytrain, yhat)

=> results in 87.5% accuracy

To make predictions for new data, just create another numpy array containing your test data and call lr.predict on it.

You might also want to look into parameter tuning to improve your score. For example the LogisticRegression class has some parameters that control regularization - tuning them with methods found in sklearn.grid_search might improve your score.