What is the meaning for the coef and intercept
A (binary) logistic regression algorithm tries to determine whether the data $x$ belongs to class 0 or class 1 by the value
$f(x)=\omega x+b$. If the value $f(x)>0$, the algorithm believe $x$ is more likely to be in class 1; while if $f(x)<0$, then $x$ is more likely to be in class 0.
In your code
alg.coef_ is the $\omega$ above, and
alg.intercept_ is the $b$ above.
Why everything is predicted to be in class 0
If you run your code, you should see the result
alg.coef_=-0.354, alg.intercept_=0.307. Therefore your
alg is calculating $f(x)=-0.354x+0.307$. Now if you plug in your
data, i.e. $x=$1, 2, 1.5 and 2.5 respectively, you should get the value $f(x)=$-0.047,-0.401,-0.224 and -0.579 respectively.
As you see, all the four $f(x)$ values are smaller than 0, therefore your
alg determines that they all belong be class 0.
But still why? Why the result is not what I expected?
Here is were regularization comes in. In the context of logistic regression, the learning algorithm regularizes that the learned parameter, $w$, should not be too large. To be specific, by default it regularizes the $l_2$ norm norm of $w$. In your case $w$ is a scalar, then $l_2$ norm is its absolute value.
In general, regularization is used to prevent overfitting. But that's another broad topic.
If you want your algorithm to behave as you expected (classify 1 and 1.5 to class 1, and classify 2 and 2.5 to class 0) , there are 2 ways.
- You tell you program to use extremely weak regularization.
That can be achieved by passing a large value to the
C parameter in
alg = LogisticRegression() (default value is 1.0, see detail), e.g.
alg = LogisticRegression(C=1000)
In this case your algorithm should return
alg.coef_=-9.894, alg.intercept_=17.156, and it can classify your four data points correctly. Notice that the absotely value of
alg.coef_ is now much larger than before.
data = [x[:-1] for x in points]*100
target = [int(x[-1]) for x in points]*100
Then you can get
alg.coef_=-4.438, alg.intercept_=7.594 without setting
C. In this case the same default regularization strength is still applied, but the additional data provide stronger evidence for your algorithm to believe that the
alg.coef_ should be allowed larger (in absolutely value).