What is the difference between sklearn's LogisticRegression classifier and its SGDClassifier? I understand that the SGD is an optimization method, while Logistic Regression (LR) is a machine learning algorithm/model. I also understand that SGDClassifier is a linear classifier that is optimized by SGD (per this answer: per this answer but how do these two models below differ?
SGDClassifier(loss='log') and LogisticRegression(solver='sag') ?
Logistic regression has different solvers {‘newton-cg’, ‘lbfgs’, ‘liblinear’, ‘sag’, ‘saga’}, which SGD Classifier does not have, you can read the difference in the articles that sklearn offers.
SGD Classifier is a generalized model that uses gradient descent. In it you can specify the learning rate, the number of iterations and other parameters. There are also many identical parameters, for example l1, l2 regularization.
If you select loss='log', then indeed the model will turn into a logistic regression model. However, the biggest difference is that the SGD Classifier can be trained by batch - using the partial_fit() method. For example, if you want to do online training, active training, or training on big data. That is, you can configure the learning process more flexibly and track metrics for each epoch, for example. In this case, the training of the model will be similar to the training of a neural network. Moreover, you can create a neural network with 1 layer and 1 neuron and take the logistic loss function for training on tensorflow or pytorch or another framework. And you will get logistic regression again.
If you use only the fit function, then there should not be a big difference in the use of these models, except for the one introduced by various LogisticRegression solvers, which are not in SGD Classifier.
