# Finding logistic loss/negative log likelihood - binary logistic regression classification

I am new to ML and data science and am struggling with a simple problem. In my problem, I am given a series of datapoints $$X_i$$ where $$X_i = (x_{i1}, x_{i2})$$ with each data point having a label $$y_i$$ where $$y_i \in [-1, 1]$$.

My first task that I must complete the following: Given a weight vector $$w$$, write a function to compute the logistic loss (also known as the negative log likelihood) for a given dataset.

I also am tasked with building a function that computes the gradient of the logistic loss evaluated at $$w$$, and both of these functions in tandem will be used to run gradient descent on the given dataset. For now, I am strictly concerned with developing the first function.

I understand log likelihood to be $$\sum_{i=1}^n y_i \log p(x_i) + (1 − y_i) \log (1 − p(x_i))$$ for a binary classifier, but I am unsure of how to write a function that computes the negative log likelihood. Specifically, how do we calculate $$p(x_i)$$ and how does a given weight vector $$w$$ factor into things?

• It looks like $p$ is the regression equation, so $p(x_i)=\hat w_0+\hat w_1x_i$. Now how does your log likelihood look? // Negative log likelihood is exactly what it sounds like. Just take $-1$ times the log likelihood.
– Dave
Nov 23, 2021 at 0:30

Your dataset has two features $$x_{i1}$$ and $$x_{i2}$$ per data point, $$i$$. Hence your weight vector would look like [$$w_0$$,$$w_1$$,$$w_2$$] where $$w_0$$ is the bias and the next two elements are weights corresponding to two features respectively.

Therefore, the predicted output would be $$p(x_i) = w_0 + (w_1*x_{i1}) + (w_2*x_{i2})$$

The class labels are -1 and 1. Hence it is a binary classification problem. It would be nice to have labels 0 and 1 for using standard formula for log-likelihood. So consider changing -1's to 0's. Then apply the formula you suggested to calculate log-likelihood.

Else, if you do not prefer changing labels, and keep labels [-1,1] as such, modify the formula as below:

$$log{-}likelihood = \sum_{i=1}^n max(y,0)*\log p(x_i) + min(1-y,1)*(1-\log p(x_i))$$

The first term in the formula should become zero when label $$y$$ is -1 and the second term should become zero when the label $$y$$ is 1.

Once log-likelihood is obtained, just negate it ($$-1 * log{-}likelihood$$) to obtain the negative-log-likelihood.

Hope it helps.