# How to add more theta parameters into my logistic regression?

I am a complete beginner in machine learning and coding in python. I have been tasked with coding logistic regression from scratch in comparison with using sklearn. My question is, with my code below I believe I have set the number of thetas I want with:

"X = data[['texture_mean','perimeter_mean','smoothness_mean','compactness_mean','symmetry_mean', 'diagnosis']]"


but I am unsure how to prove this is true with my code below and its definition of theta, if I added more parameters (e.g. all 31 variables of this dataset [https://www.kaggle.com/uciml/breast-cancer-wisconsin-data] which is for classifying tumours) would I just need to add them into this list above? Any help pointing me towards the right direction just to understand this better would be appreciated.

X_train,X_test,Y_train,Y_test = train_test_split(X,Y,test_size=0.3)

X = data["diagnosis"].map(lambda x: float(x))

X = data[['texture_mean','perimeter_mean','smoothness_mean','compactness_mean','symmetry_mean', 'diagnosis']]
X = np.array(X)
X = min_max_scaler.fit_transform(X)
Y = data["diagnosis"].map(lambda x: float(x))
Y = np.array(Y)

def Sigmoid(z):
if z < 0:
return 1 - 1/(1 + math.exp(z))
else:
return 1/(1 + math.exp(-z))
def Hypothesis(theta, x):
z = 0
for i in range(len(theta)):
z += x[i]*theta[i]
return Sigmoid(z)enter preformatted text here

def Cost_Function(X,Y,theta,m):
sumOfErrors = 0
for i in range(m):
xi = X[i]
hi = Hypothesis(theta,xi)
error = Y[i] * math.log(hi if  hi >0 else 1)
if Y[i] == 1:
error = Y[i] * math.log(hi if  hi >0 else 1)
elif Y[i] == 0:
error = (1-Y[i]) * math.log(1-hi  if  1-hi >0 else 1)
sumOfErrors += error

const = -1/m
J = const * sumOfErrors
print ('cost is: ', J )
return J

def Cost_Function_Derivative(X,Y,theta,j,m,alpha):
sumErrors = 0
for i in range(m):
xi = X[i]
xij = xi[j]
hi = Hypothesis(theta,X[i])
error = (hi - Y[i])*xij
sumErrors += error
m = len(Y)
constant = float(alpha)/float(m)
J = constant * sumErrors
return J

new_theta = []
constant = alpha/m
for j in range(len(theta)):
CFDerivative = Cost_Function_Derivative(X,Y,theta,j,m,alpha)
new_theta_value = theta[j] - CFDerivative
new_theta.append(new_theta_value)
return new_theta

def Logistic_Regression(X,Y,alpha,theta,num_iters):
m = len(Y)
for x in range(num_iters):
theta = new_theta
if x % 100 == 0:
Cost_Function(X,Y,theta,m)
print ('theta: ', theta)
print ('cost is: ', Cost_Function(X,Y,theta,m))

initial_theta = [0,1]
alpha = 0.01
iterations = 1000
Logistic_Regression(X,Y,alpha,initial_theta,iterations)


Your pass your initial_theta into Logistic_Regression where it defines how the cost function and its derivative are evaluated. Just make initial_theta the same width as X. If you want your code to be fool-proof, check variable sizes within your function like

def Logistic_Regression(X,Y,alpha,theta,num_iters):
assert len(theta) == X.shape[1], 'theta should have one coefficient per each column of X'
....


A more convenient solution would be to define initial_theta inside the function, based on X:

def Logistic_Regression(X,Y,alpha,num_iters):
theta = np.zeros(X.shape[1])
....


In this case, you can be sure that the shape of initial $\theta$ is correct.

• Thank you for this - just to be sure in order to make it the same width is that saying this (as my x currently has 6 variables): initial_theta = [0,6]
– DN1
Nov 15, 2017 at 14:10
• array [0,6] still has only two entries :) You need something like [0,1,1,1,1,1] Nov 15, 2017 at 14:17