# Why results of statsmodel logreg is different from scikit-learn logreg?

I am trying to do a binary classification. I have only 6 input variables and one output variables. Label 1 is 1554 records and Label 0 is 3558 records.

As you can see below, the metrics that I get from these two are different. I am not sure what other info I can share more about this issue. I just tried to do the classification through different methods

# code for statsmodel logreg

model = smm.Logit(y_train, X_train_std)    #std indicates standardized inputs
result=model.fit()
result.summary()
y_pred = result.predict(X_test_std)
y_pred[y_pred > 0.5] = 1
y_pred[y_pred < 0.5] = 0
cm = confusion_matrix(y_test, y_pred)
print(cm)
print("Accuracy is ", accuracy_score(y_test, y_pred))
print(classification_report(y_test, y_pred))
print("ACU score is ",roc_auc_score(y_test, y_pred))
print("Recall score is",recall_score(y_test,y_pred))
print("Precision score is",precision_score(y_test,y_pred))
print("F1 score is",f1_score(y_test,y_pred))


# code for scikit-learn

#log reg optimized parameters
op_param_grid = {'C': [0.01],'class_weight':['balanced'],'penalty': ['l1'], 'solver': ['saga'],'max_iter':[200]}
logreg=LogisticRegression(random_state=41)
logreg_cv=GridSearchCV(logreg,op_param_grid,cv=10,scoring='f1')
logreg_cv.fit(X_train_std,y_train)


May I know why is this happening and which one should I rely upon?

Is it not right to expect the same results on both approach? Do they work differently?

Can anyone help?

• You need to tell in detail what you did and/or post code. – Peter Jan 11 at 15:22
• Hi @Peter - Please find the updated post – The Great Jan 11 at 15:31

As far as I can tell, you use a "normal" Logit in one approach and a Logit with L1 penalty in the other case (penalty': ['l1']), which is called "Lasso". In Statsmodels L1 penalty would be implemented like stated in the docs.
Lasso and "normal" Logit are two different approaches. In the first case, (some) parameters are shrunken and can be set to zero, while with normal Logit, no parameter is shrunken. Lasso is often used to improve fit with noisy $$x$$, to do feature selection, or in case of high dimensional data (lot of $$x$$). Compare Chapters 4.3 and 6.2 in Introduction to Statistical Learning.
• Because when I tried it with just logistic regression (), meaning only default paramters, I still get different results – The Great Jan 12 at 0:58
• @TheGreat You can pass to sklearn penalty: 'none' or follow Peter's link to implement L1 penalty in statsmodels. But also you will probably need to address the class weights you specify in sklearn. (And why run grid search over a single point?) – Ben Reiniger Jan 12 at 3:29