1
$\begingroup$

I am trying to do a binary classification. I have only 6 input variables and one output variables. Label 1 is 1554 records and Label 0 is 3558 records.

As you can see below, the metrics that I get from these two are different. I am not sure what other info I can share more about this issue. I just tried to do the classification through different methods

# code for statsmodel logreg

model = smm.Logit(y_train, X_train_std)    #std indicates standardized inputs
result=model.fit()
result.summary()
y_pred = result.predict(X_test_std)
y_pred[y_pred > 0.5] = 1
y_pred[y_pred < 0.5] = 0
cm = confusion_matrix(y_test, y_pred)
print(cm)
print("Accuracy is ", accuracy_score(y_test, y_pred))
print(classification_report(y_test, y_pred))
print("ACU score is ",roc_auc_score(y_test, y_pred))
print("Recall score is",recall_score(y_test,y_pred))
print("Precision score is",precision_score(y_test,y_pred))
print("F1 score is",f1_score(y_test,y_pred))

# code for scikit-learn

#log reg optimized parameters
op_param_grid = {'C': [0.01],'class_weight':['balanced'],'penalty': ['l1'], 'solver': ['saga'],'max_iter':[200]}
logreg=LogisticRegression(random_state=41)
logreg_cv=GridSearchCV(logreg,op_param_grid,cv=10,scoring='f1')
logreg_cv.fit(X_train_std,y_train)

May I know why is this happening and which one should I rely upon?

enter image description here

Is it not right to expect the same results on both approach? Do they work differently?

Can anyone help?

$\endgroup$
  • $\begingroup$ You need to tell in detail what you did and/or post code. $\endgroup$ – Peter Jan 11 at 15:22
  • 1
    $\begingroup$ Hi @Peter - Please find the updated post $\endgroup$ – The Great Jan 11 at 15:31
3
$\begingroup$

As far as I can tell, you use a "normal" Logit in one approach and a Logit with L1 penalty in the other case (penalty': ['l1']), which is called "Lasso". In Statsmodels L1 penalty would be implemented like stated in the docs.

Lasso and "normal" Logit are two different approaches. In the first case, (some) parameters are shrunken and can be set to zero, while with normal Logit, no parameter is shrunken. Lasso is often used to improve fit with noisy $x$, to do feature selection, or in case of high dimensional data (lot of $x$). Compare Chapters 4.3 and 6.2 in Introduction to Statistical Learning.

ISLR comes with Python labs. This can give you a good starting point how to apply one or the other method in a proper way.

| improve this answer | |
$\endgroup$
  • $\begingroup$ Hi, thanks for the response .upvoted... Leaving my code aside, is it even possible to get the same results provided the parameters are same? I mean even if I don't key in parameters for Scikit log reg, it will pick the default ones... So am I right to expect that results could be same if I use default parameters? Have you tried it $\endgroup$ – The Great Jan 12 at 0:25
  • $\begingroup$ Because when I tried it with just logistic regression (), meaning only default paramters, I still get different results $\endgroup$ – The Great Jan 12 at 0:58
  • $\begingroup$ @TheGreat You can pass to sklearn penalty: 'none' or follow Peter's link to implement L1 penalty in statsmodels. But also you will probably need to address the class weights you specify in sklearn. (And why run grid search over a single point?) $\endgroup$ – Ben Reiniger Jan 12 at 3:29
  • $\begingroup$ regd the gridsearch, those are the best hyparameters chosen after gridsearch in previous runs. I just retain them as is.. $\endgroup$ – The Great Jan 12 at 4:39
  • $\begingroup$ Have a look here: stats.stackexchange.com/questions/203740/… $\endgroup$ – Peter Jan 12 at 13:05

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.