# calculating the cost function
theta = np.array([[0], [0]])
def h(X, theta):
return np.matmul(X, theta)
def cost(X, y, theta, m):
return 1/(2* m) * np.sum((h(X, theta)-y)**2)
print(cost(X, y, theta, m))
# 1-variable gradient decent
cost_vals = []
def grad_decent(X, y, theta, m, alpha):
theta1 = theta[0, 0]
theta2 = theta[1, 0]
for i in range(0, iter):
cost_vals.append(cost(X, y, np.array([[theta1], [theta2]]), m))
theta1 = theta1 - alpha/m * np.sum((h(X, theta)-y) * X[:, 0])
theta2 = theta2 - alpha/m * np.sum((h(X, theta)-y) * X[:, 1])
# print(f"Iteration {i}: ", theta1, ", ", theta2)
return np.array([[theta1], [theta2]])
As one can see I'm doing gradient descent with only one feature, i.e. I want to fit a line of the form theta1 + theta2 * x to my data. I know that for such small feature sizes one would definitely use the normal equation, but I wanted to implement it with gradient descent. My learning rate alpha is 0.01 and I'm following the machine learning course of andrew ng on coursera. However, instead of converging, my cost values increase so there is some mistake in the algorithm which I can't find. X is of the shape 97x2 where the first column is just ones since there is only one feature but I want to fit a polynomial with two coefficients.