# Plotting Gradient Descent in 3d - Contour Plots

I have generated 3 parameters along with the cost function. I have the $$\theta$$ lists and the cost list of 100 values from the 100 iterations. I would like to plot the last 2 parameters against cost in 3d to visualize the level sets on the contour plots and the cereal bowl function.

House Dataset with 3 parameters (1's, bedrooms, Sq.ft) to predict prices having shape $$(100000, 3)$$ and $$y(100000, )$$. The goal is to look at the cereal bowl function in 3d and look at how the gradients are converging.

References: Gradient descent implementation in python - contour lines:

def compute_cost(X, y, theta):
return np.sum(np.square(np.matmul(X, theta) - y)) / (2 * len(y))

def gradient_descent_multi(X, y, theta, alpha, iterations):
theta = np.zeros(X.shape[1])
m = len(X)
j_history = np.zeros(iterations)
theta_1_hist = []
theta_2_hist = []
for i in range(iterations):

gradient = (1/m) * np.matmul(X.T, np.matmul(X, theta) - y)

theta = theta - alpha * gradient

j_history[i] = compute_cost(X,y,theta)
theta_1_hist.append(theta[1])
theta_2_hist.append(theta[2])

#         J_history.append(compute_cost(X,y,theta))
#         print(J_history)

return theta ,j_history, theta_1_hist, theta_2_hist

theta = np.zeros(2)
alpha = 0.1
iterations = 100

#Computing the gradient descent
theta_result,J_history, theta_0, theta_1 = gradient_descent_multi(X,y,theta,alpha,iterations)

Theta 1:
[15.651431183495157,
28.502297542920118,
39.0665487784193,
...
105.78644212297141,
105.882701389551,
105.97741737336399]
Theta 2:
[14.713094556818124,
26.640668175454184,
36.29642936488919,
....
59.1710519900493,
59.07633606136845]
Cost array:
array([185814.55027215, 149566.02825652, 120605.70700938,  97414.66187874,
78807.39414333,  63853.50250138,  51819.24085843,  42123.5122655 ,
34304.44290442,  27993.78459818,  22897.16477958,  18778.74417703,
....
1257.38095357,   1257.13475353,   1256.89643143,   1256.66572779,
1256.44239308,   1256.22618706,   1256.01687827,   1255.81424349,
1255.61806734,   1255.42814185,   1255.24426618,   1255.06624625])


To plot the last two parameters against cost in 3D, you can use the matplotlib library in Python. Here is an example of how to do it:

import matplotlib.pyplot as plt
from mpl_toolkits.mplot3d import Axes3D

# Create a figure and a 3D Axes
fig = plt.figure()
ax = fig.add_subplot(111, projection='3d')

# Set the x, y, and z data
x = theta_0
y = theta_1
z = J_history

# Plot the data
ax.scatter(x, y, z)

# Set the x, y, and z labels
ax.set_xlabel('theta_0')
ax.set_ylabel('theta_1')
ax.set_zlabel('cost')

plt.show()


This code will create a 3D scatter plot with the last two parameters on the x and y axes, and the cost on the z axis. You can then use this plot to visualize the level sets on the contour plots and the cereal bowl function.