To my understanding, the SGD classifier, and Logistic regression seems similar. An SGD classifier with loss = 'log' implements Logistic regression and loss = 'hinge' implements Linear SVM. I also understand that logistic regression uses gradient descent as the optimization function and SGD uses Stochastic gradient descent which converges much faster. But which of the two algorithms to use in which scenarios? Also, how are SGD and Logistic regression similar and how are they different?


1 Answer 1


Welcome to SE:Data Science.

SGD is a optimization method, while Logistic Regression (LR) is a machine learning algorithm/model. You can think of that a machine learning model defines a loss function, and the optimization method minimizes/maximizes it.

Some machine learning libraries could make users confused about the two concepts. For instance, in scikit-learn there is a model called SGDClassifier which might mislead some user to think that SGD is a classifier. But no, that's a linear classifier optimized by the SGD.

In general, SGD can be used for a wide range of machine learning algorithms, not only LR or linear models. And LR can use other optimizers like L-BFGS, conjugate gradient or Newton-like methods.

  • 2
    $\begingroup$ So, if I write clf = SGDClassifier( class_weight='balanced', alpha=i, penalty='l2', loss='hinge', random_state=42) it is an implementation of Linear SVM and if I write clf = SGDClassifier( class_weight='balanced', alpha=i, penalty='l2', loss='log', random_state=42) . It is an implementation of Logisitic regression. Am I right ? $\endgroup$ Commented Sep 7, 2018 at 18:37
  • 1
    $\begingroup$ @AkashDubey Yes $\endgroup$
    – user12075
    Commented Sep 7, 2018 at 21:18
  • $\begingroup$ @user12075 how does SGDClassifer compare to sklearn.linear_model.LogisticRegression(solver='sag', multi_class='multinomial') ? $\endgroup$ Commented Nov 24, 2022 at 14:18

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.