# Why not LinearClassifer when RidgeClassifier works?

Recently I came across this model called RidgeClassifier, It coverts the predicted value (y) to {-1,1} and then uses the Ridge Regression. During prediction if the value of y is < 0 then it is predicted as -1 and if the value of y is > 0 then it is predicted as +1.

So, when RidgeClassifier works well for pretty well for some datasets, Why dont/cant we have something called LinearClassifer which can use LinearRegression during training and the same model for prediction?

• In other words, why can't we use a regression model for a classification task, by just pretending the dependent variable is continuous? Nov 19 at 11:21
• What stops you from doing this method without a ridge regression penalty?
– Dave
Nov 19 at 11:23
• I think this why no classification with regression? explains that....? Why cant we make the same argument for RidgeClassifier, expecting it does not work, But then how does it work @Dave Nov 19 at 12:57

My stance aligns with what Frank Harrell wrote in the link you gave, so I will not defend the methodology you propose; I am a fan of working with probability outputs. However, nothing stops you from doing the modeling with an OLS regression. After all, the parameters in ridge regression are (typically) calculated as $$\hat\beta_{ridge} = (X^TX +\lambda I_{p\times p})^{-1}X^Ty$$. If you do the special case of this where $$\lambda = 0$$, you get the usual OLS equation.

If you are determined to predict on a continuum and then convert to discrete classes, linear regression is just as viable as ridge regression.

Example in R

set.seed(2021)
N <- 100
x <- runif(N,-2, 2)
z <- x
pr <- 1/(1 + exp(-z))
y <- rbinom(N, 1, pr)
y_alt <- 2*y - 1 # convert to +/- 1
L <- lm(y_alt ~ x)
preds <- predict(L)


Now take preds > 0 as y_alt = +1 and preds < 0 as y_alt = -1.