I think it's important to clarify the different terms used here.
An estimator refers to the type of model that is being used (e.g. logistic regression, linear regression, support vector machines).
The loss function refers to the function you use to quantify how much your model's predictions differ from the target values they're trying to predict (e.g. log loss, mean squared error, cross entropy loss)
An optimisation technique is a method for updating the parameters of your model to maximise its performance with respect to some loss function - often referred to as minimising the loss. (e.g Stochastic gradient descent, maximum likelihood estimation).
In the case of SGDClassifier()
you are first picking an optimisation method - stochastic gradient descent, and then picking a loss function to optimise, by specifying the loss
argument. With this information, scikit-learn then 'autocompletes' the model (estimator) to be used.
The page you linked to answers part of your question about which models correspond to which loss functions:
- Perceptron = Perceptron
- Least-squares = Linear regression
- Hinge = SVM classification
- Epsilon-insensitive = SVM Regression
As for Huber and modified Huber, I believe those are also equivalent to variations of SVM (see here).
Note that the SGDClassifier()/SGDRegressor()
classes will not always give you the same results as the model-specific scikit-learn (e.g. LogisticRegressor()
), since this former will use Stochastic Gradient Descent for optimisation, while the latter will usually have a different default optimisation method.