1
$\begingroup$

I am a complete beginner in machine learning and coding in python. I have been tasked with coding logistic regression from scratch in comparison with using sklearn. My question is, with my code below I believe I have set the number of thetas I want with:

"X = data[['texture_mean','perimeter_mean','smoothness_mean','compactness_mean','symmetry_mean', 'diagnosis']]"  

but I am unsure how to prove this is true with my code below and its definition of theta, if I added more parameters (e.g. all 31 variables of this dataset [https://www.kaggle.com/uciml/breast-cancer-wisconsin-data] which is for classifying tumours) would I just need to add them into this list above? Any help pointing me towards the right direction just to understand this better would be appreciated.

X_train,X_test,Y_train,Y_test = train_test_split(X,Y,test_size=0.3)

X = data["diagnosis"].map(lambda x: float(x))

X = data[['texture_mean','perimeter_mean','smoothness_mean','compactness_mean','symmetry_mean', 'diagnosis']]
X = np.array(X)
X = min_max_scaler.fit_transform(X)
Y = data["diagnosis"].map(lambda x: float(x))
Y = np.array(Y)


def Sigmoid(z):
    if z < 0:
        return 1 - 1/(1 + math.exp(z))
    else:
        return 1/(1 + math.exp(-z))
def Hypothesis(theta, x):
    z = 0
    for i in range(len(theta)):
    z += x[i]*theta[i]
return Sigmoid(z)enter preformatted text here

def Cost_Function(X,Y,theta,m):
    sumOfErrors = 0
    for i in range(m):
        xi = X[i]
        hi = Hypothesis(theta,xi)
        error = Y[i] * math.log(hi if  hi >0 else 1)
        if Y[i] == 1:
            error = Y[i] * math.log(hi if  hi >0 else 1)
        elif Y[i] == 0:
            error = (1-Y[i]) * math.log(1-hi  if  1-hi >0 else 1)
        sumOfErrors += error

    const = -1/m
    J = const * sumOfErrors
    print ('cost is: ', J ) 
    return J

def Cost_Function_Derivative(X,Y,theta,j,m,alpha):
    sumErrors = 0
    for i in range(m):
        xi = X[i]
        xij = xi[j]
        hi = Hypothesis(theta,X[i])
        error = (hi - Y[i])*xij
        sumErrors += error
    m = len(Y)
    constant = float(alpha)/float(m)
    J = constant * sumErrors
    return J

def Gradient_Descent(X,Y,theta,m,alpha):
    new_theta = []
    constant = alpha/m
    for j in range(len(theta)):
        CFDerivative = Cost_Function_Derivative(X,Y,theta,j,m,alpha)
        new_theta_value = theta[j] - CFDerivative
        new_theta.append(new_theta_value)
    return new_theta

def Logistic_Regression(X,Y,alpha,theta,num_iters):
    m = len(Y)
    for x in range(num_iters):
        new_theta = Gradient_Descent(X,Y,theta,m,alpha)
        theta = new_theta
        if x % 100 == 0:
            Cost_Function(X,Y,theta,m)
            print ('theta: ', theta)    
            print ('cost is: ', Cost_Function(X,Y,theta,m))

initial_theta = [0,1]  
alpha = 0.01
iterations = 1000
Logistic_Regression(X,Y,alpha,initial_theta,iterations)
$\endgroup$

1 Answer 1

0
$\begingroup$

Your pass your initial_theta into Logistic_Regression where it defines how the cost function and its derivative are evaluated. Just make initial_theta the same width as X. If you want your code to be fool-proof, check variable sizes within your function like

def Logistic_Regression(X,Y,alpha,theta,num_iters):
    assert len(theta) == X.shape[1], 'theta should have one coefficient per each column of X'
    ....

A more convenient solution would be to define initial_theta inside the function, based on X:

def Logistic_Regression(X,Y,alpha,num_iters):
    theta = np.zeros(X.shape[1])
    ....

In this case, you can be sure that the shape of initial $\theta$ is correct.

$\endgroup$
2
  • $\begingroup$ Thank you for this - just to be sure in order to make it the same width is that saying this (as my x currently has 6 variables): initial_theta = [0,6] $\endgroup$
    – DN1
    Nov 15, 2017 at 14:10
  • $\begingroup$ array [0,6] still has only two entries :) You need something like [0,1,1,1,1,1] $\endgroup$
    – David Dale
    Nov 15, 2017 at 14:17

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.