# SKLearn - Understanding Discrepancy Between LogisticRegressionCV classification_report and scores_

Cross-posting from Stack Overflow:

I'm running into a weird situation where my sklearn LogisticRegressionCV model is apparently getting 100% accuracy (the lack of shuffling is intentional). However, when I ask the model to report its accuracy scores, the accuracy scores per inverse regularization are much lower than 100%. What am I doing wrong?

The model:


decoder = LogisticRegressionCV(
penalty='l1',                       # want sparse parameters
cv=KFold(n_splits=5, shuffle=False),
Cs=np.logspace(-1, 2, 3),
random_state=0,                     # reproducibility
dual=False,                         # Prefer dual=False when n_samples > n_features.
solver='liblinear',                 # for l1 penalty, must use liblinear or saga
max_iter=1e5,                       # give plenty of time to get good performance
fit_intercept=False,
class_weight='balanced',
).fit(X, y)

y_hat = decoder.predict(X)

y_hat_probs = decoder.predict_proba(X)

print(classification_report(y, y_hat))
print(confusion_matrix(y, y_hat))


outputs

              precision    recall  f1-score   support

0.0       1.00      1.00      1.00       374
1.0       1.00      1.00      1.00       105

accuracy                           1.00       479
macro avg       1.00      1.00      1.00       479
weighted avg       1.00      1.00      1.00       479

[[374   0]
[  0 105]]


Asking for decoder.scores_ yields values nowhere near 100%:

{1.0: array([[0.35416667, 0.60416667, 0.61458333],
[0.64583333, 0.64583333, 0.65625   ],
[0.59375   , 0.65625   , 0.64583333],
[0.54166667, 0.57291667, 0.59375   ],
[0.17894737, 0.61052632, 0.65263158]])}


Why is there this discrepancy?

• When making the kfold, the model have access not to all the data frame whereas when making the predictions, since you are training and evaluating in the same set, the model is just "remembering" the data – Julio Jesus Dec 10 '20 at 23:57

## 1 Answer

You are training and evaluate on the same data.

You should split data to train set(to train) and test set(to evaluate) to get proper results. Kinda:


decoder.fit(X_train, y_train)
y_hat = decoder.predict(X_test)

print(classification_report(y_test, y_hat))
print(confusion_matrix(y_test, y_hat))