# Is this scheme correct for logistic regression with stochastic gradient descent

I am implementing logistic regression with stochastic gradient descent, but it is not working as expected. I've tried many epochs and different learning rates $$\alpha$$ but the probability of belonging to the correct class oscillates around $$0.5$$. I have got two classes denoted as $$0,1$$ and two features. Is this scheme correct?:

Initialize weights $$w = [w_0, w_1, w_2]$$ to some random number (I choose 1) and loop:

1. Choose random training sample $$x_i = [1, x_1, x_2]$$.
2. Calculate the probability of it belonging to class $$1,$$ $$P(\hat{y} = 1) = \frac{1}{1+e^{-w \cdot x_i}}$$
3. Update the weights according to $$w = w + \alpha x_i(y_i - P(\hat{y} = 1))$$

This scheme makes sense to me since the update to the weights would be big if the probability of belonging to class $$1$$ was low, if the correct class was $$1$$ but I can't verify it more than that.

• Instead of showing the formula you intended to implement, please show the implementation itself (a minimal code example that people can run). – Mathias Müller Feb 16 '20 at 11:55