How many coefficients does the Logistic regression model has as a function of the number of features?

Question

I have built a logistic regression model using Python anaconda and was surprised to see that the number of model coefficients turned out to be proportional to the training sample size i.e.

My training data is:

print('Training data is type %s and shape %s' % (type(os_X_train), os_X_train.shape))

and outputs:

Training data is type <class 'pandas.core.frame.DataFrame'> and shape (174146, 11)

Then the model is:

logreg = LogisticRegression(penalty='l2',solver='lbfgs',max_iter=1000)
model = make_pipeline(preprocess, logreg)
model.fit(os_X_train, os_y_train)
print(logreg.coef_.shape)
print("Model coefficients: ", logreg.intercept_, logreg.coef_)

This outputs:

(1, 153024)
Model coefficients:  [12.02830778] [[ 0.42926969  0.14192505 -1.89354062 ...  0.008847    0.00884372 -8.15123962]]

To my understanding the number of model coefficients should be the number of columns for the predictor variables or features plus one the intercept, or?

Answer 1

You are correct that the number of parameters depends on the number of features and it of the number of observations. You are seeing this error because you and the computer do not agree on what your input data mean.

Sklearn assumes your data to be $n\times p$, where each row represents an observation, and each column represents a variable. I think you’re doing the transpose if that, $p \times n$, where each column represents an observation. As you have more and more observations, you have more and more columns, telling sklearn that there are additional features.

If you transpose your predictor data, the functions should behave as expected.

This isn’t unique to logistic regression. You would have this happen with any model. Say you trained a k-NN on 80 observations of 6 features. Then you test on 20 observations of those 6 features. Python should come back like, “You gave me 80 features for training and now only 20 for testing. What gives?”