Why we need a solver like bfgs in LogisticRegression unlike LinearRegression? Don't we have a close form like LinearRegression?
In Linear Regression, you're trying to predict a continuous value, like predicting the price of a house based on its size and location. In this case, we can find a simple equation to calculate the best line that fits the data, and we can solve it mathematically to find the exact answer.
But in Logistic Regression, you're trying to predict a probability, like the chance of someone having a disease based on their age, height, and weight. Instead of a straight line, we use a special function called the logistic function (or sigmoid function) to map the features to probabilities between 0 and 1.
The problem is that there is no simple mathematical equation to find the best parameters (weights) for this logistic function. We need to use an optimization algorithm, like BFGS, to help us find the best parameters.
The BFGS algorithm is like a smart detective that searches for the best parameters by trying different values and improving them step by step. It starts with some initial guesses and checks how well they fit the data. Then, it adjusts the parameters in a way that makes the predictions better. It keeps doing this until it finds the best set of parameters that give the most accurate predictions.