0
$\begingroup$

Why we need a solver like bfgs in LogisticRegression unlike LinearRegression? Don't we have a close form like LinearRegression?

$\endgroup$

1 Answer 1

1
$\begingroup$

In Linear Regression, you're trying to predict a continuous value, like predicting the price of a house based on its size and location. In this case, we can find a simple equation to calculate the best line that fits the data, and we can solve it mathematically to find the exact answer.

But in Logistic Regression, you're trying to predict a probability, like the chance of someone having a disease based on their age, height, and weight. Instead of a straight line, we use a special function called the logistic function (or sigmoid function) to map the features to probabilities between 0 and 1.

The problem is that there is no simple mathematical equation to find the best parameters (weights) for this logistic function. We need to use an optimization algorithm, like BFGS, to help us find the best parameters.

The BFGS algorithm is like a smart detective that searches for the best parameters by trying different values and improving them step by step. It starts with some initial guesses and checks how well they fit the data. Then, it adjusts the parameters in a way that makes the predictions better. It keeps doing this until it finds the best set of parameters that give the most accurate predictions.

$\endgroup$
1
  • $\begingroup$ So, the difference is that there is sigmoid function added to linear model. So, where the complexity is that makes us use optimization function? Thanks $\endgroup$ May 29, 2023 at 7:23

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.