I have a basic 2D Linear Regression model coded out (using gradient descent), yet it doesn't seem to work as well as it should.
What I expect is that
c should approach 4 and 3 respectively, and
m's slope or
c's slope should tend to 0; yet what is actually happening is that
c's slope approaches a non-zero value, and
c itself approaches a value depending on the epoch (around 0.5 with an epoch of 100.)
If I look at the graph of
c, it very slowly tends up over time, though.
import random, math import matplotlib.pyplot as plt def linreg(x, y): """ Performs linear regression: input x, output y. """ n = float(len(x)) m = random.random() c = random.random() dm, dc = ,  rate = 0.00001 epoch = 100 for run in range(epoch): d_m = 0 d_c = 0 for i in range(len(x)): d_m += (y[i] - m*x[i] - c)*x[i] d_c += (y[i] - m*x[i] - c) d_m *= -2/n d_c *= -2/n m -= d_m * rate c -= d_c * rate dm.append(d_m) dc.append(d_c) return m, c, dm, dc x = [i for i in range(400)] y = [4*i + 3 for i in x] m, c, dm, dc = linreg(x, y) print(m, c) plt.grid() plt.scatter(x, y) plt.plot(x, [m*i + c for i in x], color='red') plt.show() plt.grid() plt.plot([i for i in range(len(dm))], dm) plt.plot([i for i in range(len(dc))], dc, color='red') plt.show()