Difference between SVM and GD/SGD?

Question

My colleague mentioned that a data science project is using SGD classifier.

So I started reading about GD/SGD and came across a nice article about Text classification using SVM and GD.

In the end of the article the author mentions:

We begin with a bunch of documents to train on. We then represent each documents as numerical vectors, and you can choose to split them into n-grams and weigh these n-grams with TF-IDF. Now that we have a numerical representation, we can give the program training data and make it guess each time. Based on how wrong the program was, the program adjusts to its mistakes to make better prediction the next time using Gradient Descent. It keeps track of its learning using SVMs and make predictions with the SVM model by “plotting” the new datapoint in the graph and seeing if it is on the right or left by the decision boundary.

The article does not mention about SGD, but from internet research I understand that SGD is a variation of GD where-in the parameter is tweaked immediately at each iteration, rather than at the end of the entire training set.

So to me it appears that SVM is the actual classifier and GD (or SGD) is the function that tells the classifier how correctly it has done the prediction based on which the SVM classifier improves itself.

Coming back to my original question: Is my understanding right that SGD is not the classifer?

Answer 1

Yes,gradient descent (GD) and Stochastic Gradient Descent(SGD) are Optimization functions.Objective of a Optimization function is to give the best combinations of parameters for our algorithm which minimize the error.Support Vector Machine is the Classification algorithm and GD/SGD are the optimization functions use to find best optimal hyperplane(separator).

Gradient Descent’s cost-function(total error of the training set) iterates over ALL training samples

Stochastic Gradient Descent’s cost-function only accounts for ONE training sample, chosen at random