# Logistic Regression can't fit my data

I'm trying fit my data , but I couldn't fit it.Data set

0.50,0
0.75,0
1.00,0
1.25,0
1.50,0
1.75,0
1.75,1
2.00,0
2.25,1
2.50,0
2.75,1
3.00,0
3.25,1
3.50,0
4.00,1
4.25,1
4.50,1
4.75,1
5.00,1
5.50,1


and my code

data = np.loadtxt('dat', delimiter=',',dtype=None);

x=data[:,0:1];
y=data[:,1].reshape(x.size/x[0].size,1);
a=np.ones(shape=(y.size,x[0].size+1));
a[:,1:2]=x;
q=np.ones(shape=(a.shape[1],1));
alpha=0.003

for i in range(500000):
h=1/(1+np.exp(-np.dot(a,q)))
for j in range(q.size):
q[j][0]=q[j][0]-alpha*np.sum((h-y)*a[:,j]);
plt.axis((-1,10,-1,5))
plt.plot(x,y,'x',x,h);
plt.show();


So I tried different learning rates(alpha),tried different number of iterations but my fitting data is looking like this but it's should looks like this enter link description here

What am I missing? Is there any logical error or something like that? Thanks for your deal.

The problem is in the following line:

q[j][0]=q[j][0]-alpha*np.sum((h-y)*a[:,j]);


(h-y) has shape (20, 1), a[:,j] has shape (20), multiplying them results in a shape (20, 20), which is wrong.

Try a[:,j:j+1] instead of a[:,j] and it'll start working:

q[j][0]=q[j][0]-alpha*np.sum((h-y)*a[:,j:j+1]);


This gives the following plot:

You could adjust the shapes of your variables a bit to make it easier:

x=data[:,:1]
y=data[:,1]
a=np.hstack([np.ones((len(x), 1)), x])
q=np.ones(a.shape[1])
alpha=0.003

for i in range(10000):
h=1/(1+np.exp(-np.dot(a,q)))
for j in range(len(q)):
q[j]=q[j]-alpha*np.sum((h-y)*a[:,j])

plt.plot(x,y,'x',x,h)
plt.show()


Also note that in Python you don't need ; at the end of each line.

I don't write a lot of Python code but it appears you are hard coding alpha, which would explain your results. I suggest reading up on the math and intuition behind logistic regression, along with how model parameters are estimated in general.(https://en.wikipedia.org/wiki/Maximum_likelihood_estimation).

For guidance on implementation this article does a good job of going through the steps and has the benefit of being in Python: http://aimotion.blogspot.com/2011/11/machine-learning-with-python-logistic.html.

While I highly suggest you learn how to write it from scratch first, you may want to check out R which has a very easy to use base implementation of logistic regression.

df = read.csv("~/data.csv", header = T, stringsAsFactors = F)
glm_train = glm(classifier ~ value, data = df, family = "binomial")

library(ggplot2)
binomial_smooth = function(...) {
geom_smooth(method = "glm", method.args = list(family = "binomial"), ...)
}
ggplot(df, aes(x = value, y = classifier)) + geom_point() + binomial_smooth()

summary(glm_train)
# Deviance Residuals:
#   Min        1Q    Median        3Q       Max
# -1.70557  -0.57357  -0.04654   0.45470   1.82008
#
# Coefficients:
#   Estimate Std. Error z value Pr(>|z|)
# (Intercept)  -4.0777     1.7610  -2.316   0.0206 *
#   x             1.5046     0.6287   2.393   0.0167 *
#   ---
#   Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1