How to add bias consideration into logistic regression code?

Question

I am a complete beginner in coding and machine learning, and I've been tasked with learning what's under the hood of logistic regression (so I have pieced together the python code below) but I've been asked to figure out how to add bias into this code. I'm completely stuck on at what point it would need to be added into, I think at the point I am defining the hypothesis function - but if anyone would be able to point me in the right direction to figure this out I would really appreciate it.

If it helps, this logistic regression is being used to classify if a tumour is benign of malignant with the wisconsin breast cancer dataset (https://www.kaggle.com/uciml/breast-cancer-wisconsin-data)

X_train,X_test,Y_train,Y_test = train_test_split(X,Y,test_size=0.3)

X = data["diagnosis"].map(lambda x: float(x))

X = data[['texture_mean','perimeter_mean','smoothness_mean','compactness_mean','symmetry_mean', 'diagnosis']]
X = np.array(X)
X = min_max_scaler.fit_transform(X)
Y = data["diagnosis"].map(lambda x: float(x))
Y = np.array(Y)


def Sigmoid(z):
    if z < 0:
        return 1 - 1/(1 + math.exp(z))
    else:
        return 1/(1 + math.exp(-z))
def Hypothesis(theta, x):
    z = 0
    for i in range(len(theta)):
    z += x[i]*theta[i]
return Sigmoid(z)enter preformatted text here

def Cost_Function(X,Y,theta,m):
sumOfErrors = 0
for i in range(m):
    xi = X[i]
    hi = Hypothesis(theta,xi)
    error = Y[i] * math.log(hi if  hi >0 else 1)
    if Y[i] == 1:
        error = Y[i] * math.log(hi if  hi >0 else 1)
    elif Y[i] == 0:
        error = (1-Y[i]) * math.log(1-hi  if  1-hi >0 else 1)
    sumOfErrors += error

const = -1/m
J = const * sumOfErrors
print ('cost is: ', J ) 
return J

def Cost_Function_Derivative(X,Y,theta,j,m,alpha):
sumErrors = 0
for i in range(m):
    xi = X[i]
    xij = xi[j]
    hi = Hypothesis(theta,X[i])
    error = (hi - Y[i])*xij
    sumErrors += error
m = len(Y)
constant = float(alpha)/float(m)
J = constant * sumErrors
return J

def Gradient_Descent(X,Y,theta,m,alpha):
new_theta = []
constant = alpha/m
for j in range(len(theta)):
    CFDerivative = Cost_Function_Derivative(X,Y,theta,j,m,alpha)
    new_theta_value = theta[j] - CFDerivative
    new_theta.append(new_theta_value)
return new_theta

initial_theta = [0,1]  
alpha = 0.01
iterations = 1000
Logistic_Regression(X,Y,alpha,initial_theta,iterations)

Answer 1

If I understood correctly, by bias you mean the intercept term in your model, that is, $\alpha$ in the equation $$ p(y=1|x) = \frac{1}{1+e^{-(\alpha + \beta_1 x_1 + \beta_2 x_2 + ... + \beta_k x_x)}} $$

If it is the case, you can easily incorporate intercept by adding a colum of ones into your X:

X = np.hstack([np.ones([X.shape[0],1]), X])

So the first colum of $X$ is always one. Then, your model will look like

$$ p(y=1|x) = \frac{1}{1+e^{-(\beta_0 x_0 + \beta_1 x_1 + \beta_2 x_2 + ... + \beta_k x_x)}} $$ where $\beta_0$ is your bias term and $x_0$ always equals $1$. Thus, your model would be equivalent to the one with explicit intercept.