3
$\begingroup$

I have built a logistic regression model using Python anaconda and was surprised to see that the number of model coefficients turned out to be proportional to the training sample size i.e.

My training data is:

print('Training data is type %s and shape %s' % (type(os_X_train), os_X_train.shape))

and outputs:

Training data is type <class 'pandas.core.frame.DataFrame'> and shape (174146, 11)

Then the model is:

logreg = LogisticRegression(penalty='l2',solver='lbfgs',max_iter=1000)
model = make_pipeline(preprocess, logreg)
model.fit(os_X_train, os_y_train)
print(logreg.coef_.shape)
print("Model coefficients: ", logreg.intercept_, logreg.coef_)

This outputs:

(1, 153024)
Model coefficients:  [12.02830778] [[ 0.42926969  0.14192505 -1.89354062 ...  0.008847    0.00884372 -8.15123962]]

To my understanding the number of model coefficients should be the number of columns for the predictor variables or features plus one the intercept, or?

$\endgroup$
4
  • $\begingroup$ It is correct what you are saying. Why don't you just change the dimension of your traning set to see that the number of coefficents is not changing in proportion to the number of rows? $\endgroup$
    – 3nomis
    Feb 4, 2019 at 12:00
  • 2
    $\begingroup$ what type of preprocessing are you doing to your os_X_train? $\endgroup$ Feb 10, 2019 at 11:09
  • 3
    $\begingroup$ Variable $.coef\_.shape$ denotes (class number, feature counts), so there must be a problem with $make\_pipeline$. Probably, comparing the properties of $model$ and $logreg$ objects would help to crack the problem. $\endgroup$
    – Esmailian
    Mar 6, 2019 at 14:07
  • $\begingroup$ Be aware that in a multiclass classification task assuming you use One versus Rest approach you will have n_classes * (n columns + 1) coefficients $\endgroup$
    – Multivac
    Sep 2, 2020 at 15:50

1 Answer 1

1
$\begingroup$

You are correct that the number of parameters depends on the number of features and it of the number of observations. You are seeing this error because you and the computer do not agree on what your input data mean.

Sklearn assumes your data to be $n\times p$, where each row represents an observation, and each column represents a variable. I think you’re doing the transpose if that, $p \times n$, where each column represents an observation. As you have more and more observations, you have more and more columns, telling sklearn that there are additional features.

If you transpose your predictor data, the functions should behave as expected.

This isn’t unique to logistic regression. You would have this happen with any model. Say you trained a k-NN on 80 observations of 6 features. Then you test on 20 observations of those 6 features. Python should come back like, “You gave me 80 features for training and now only 20 for testing. What gives?”

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.