Logistic Regression vs SVM

Question

Following Andrew Ng's machine learning course, he explains how we can modify logistic regression to obtain SVM algorithm. First he replaces (sort of approximating) cross entropy loss with hinge loss as shown in the image below:

Then he removes the $\frac{1}{m}$ coefficient and divides the whole cost function by the regularization parameter $\lambda$, which leads to parameter $C$ behind the sigma ($C=\frac{1}{\lambda}$). Resulting cost function is shown in image below which is the minimization objective in SVM:

Later he shows that by choosing a very large value for parameter $C$, SVM will be a large margin classifier. Large value for parameter $C$ is the same as choosing a small value for parameter $\lambda$ in logistic regression. So, is logistic regression with a small value for $\lambda$ also a large margin classifier?

Answer 1

Logistic Regression is a large margin loss not classifier. (in literal sense, meaning there is a margin with the loss function not the classification boundary

There are two terms in the softmax loss, true values minus the exp or log of the observations (roughly and depending on the variant of loss) this just implies that distance of observed example from its true decision boundary needs to beat the log sum of the distances from all of the decision boundaries.

Because the softmax function is a probability distribution, the largest the log softmax can be is 0, so the log softmax returns a negative value (i.e. a penalty) that approaches 0 as the probability of the true class under the softmax function approaches 1.

Answer 2

I understood why logistic regression is not a large margin classifier by reading this medium post.