3
$\begingroup$

I'm trying fit my data , but I couldn't fit it.Data set

0.50,0
0.75,0
1.00,0
1.25,0
1.50,0
1.75,0
1.75,1
2.00,0
2.25,1
2.50,0
2.75,1
3.00,0
3.25,1
3.50,0
4.00,1
4.25,1
4.50,1
4.75,1
5.00,1
5.50,1

and my code

data = np.loadtxt('dat', delimiter=',',dtype=None);

x=data[:,0:1];
y=data[:,1].reshape(x.size/x[0].size,1);
a=np.ones(shape=(y.size,x[0].size+1));
a[:,1:2]=x;
q=np.ones(shape=(a.shape[1],1));
alpha=0.003

for i in range(500000):
    h=1/(1+np.exp(-np.dot(a,q)))
    for j in range(q.size):
        q[j][0]=q[j][0]-alpha*np.sum((h-y)*a[:,j]);
plt.axis((-1,10,-1,5))
plt.plot(x,y,'x',x,h);
plt.show();

So I tried different learning rates(alpha),tried different number of iterations but my fitting data is looking like this enter image description here but it's should looks like this enter link description here

What am I missing? Is there any logical error or something like that? Thanks for your deal.

$\endgroup$
4
$\begingroup$

The problem is in the following line:

q[j][0]=q[j][0]-alpha*np.sum((h-y)*a[:,j]);

(h-y) has shape (20, 1), a[:,j] has shape (20), multiplying them results in a shape (20, 20), which is wrong.

Try a[:,j:j+1] instead of a[:,j] and it'll start working:

q[j][0]=q[j][0]-alpha*np.sum((h-y)*a[:,j:j+1]);

This gives the following plot:

Result

You could adjust the shapes of your variables a bit to make it easier:

x=data[:,:1]
y=data[:,1]
a=np.hstack([np.ones((len(x), 1)), x])
q=np.ones(a.shape[1])
alpha=0.003

for i in range(10000):
    h=1/(1+np.exp(-np.dot(a,q)))
    for j in range(len(q)):
        q[j]=q[j]-alpha*np.sum((h-y)*a[:,j])

plt.plot(x,y,'x',x,h)
plt.show()

Also note that in Python you don't need ; at the end of each line.

$\endgroup$
0
$\begingroup$

I don't write a lot of Python code but it appears you are hard coding alpha, which would explain your results. I suggest reading up on the math and intuition behind logistic regression, along with how model parameters are estimated in general.(https://en.wikipedia.org/wiki/Maximum_likelihood_estimation).

For guidance on implementation this article does a good job of going through the steps and has the benefit of being in Python: http://aimotion.blogspot.com/2011/11/machine-learning-with-python-logistic.html.

While I highly suggest you learn how to write it from scratch first, you may want to check out R which has a very easy to use base implementation of logistic regression.

df = read.csv("~/data.csv", header = T, stringsAsFactors = F)
glm_train = glm(classifier ~ value, data = df, family = "binomial")

library(ggplot2)
binomial_smooth = function(...) {
  geom_smooth(method = "glm", method.args = list(family = "binomial"), ...)
}
ggplot(df, aes(x = value, y = classifier)) + geom_point() + binomial_smooth()

summary(glm_train)
# Deviance Residuals: 
#   Min        1Q    Median        3Q       Max  
# -1.70557  -0.57357  -0.04654   0.45470   1.82008  
# 
# Coefficients:
#   Estimate Std. Error z value Pr(>|z|)  
# (Intercept)  -4.0777     1.7610  -2.316   0.0206 *
#   x             1.5046     0.6287   2.393   0.0167 *
#   ---
#   Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

enter image description here

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.