-4
$\begingroup$

Here is my understanding of the relation between MLE & Gradient Descent in Logistic Regression. Please correct me if I'm wrong:

1) MLE estimates optimal parameters by taking the partial derivative of the log-likelihood function wrt. each parameter & equating it to 0. Gradient Descent just like MLE gives us the optimal parameters by taking the partial derivative of the loss function wrt. each parameter. GD also uses hyperparameters like learning rate & step size in the process of obtaining parameter values.

2) MLE is analytical/exact whereas GD is numerical/approximate. Unfortunately, MLE in Logistic Regression cannot give an exact solution, unless we have a small sample of data with one or two independent variables. So in Logistic Regression, we use both methods to find the best values of parameters. We take the negative of the log-likelihood function that we obtain through MLE as a loss function & use it in GD to get parameter values.

$\endgroup$

1 Answer 1

1
$\begingroup$

2) is Wrong. MLE stands for Maximum likelihood estimation. Not an analytical method.

$\endgroup$
1
  • $\begingroup$ Thank you. Are the rest of my points alright? $\endgroup$
    – Apoorva
    May 20 at 17:59

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.