Implementing single variable Linear Regression in python

Question

I'm trying to implement LMS algorithm in python. I have the following code:

import os
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt


def compute_cost(theta, X , y):
    inner = np.power(X.dot(theta) - y, 2)
    return np.sum(inner/2*len(X))


if __name__ == '__main__':
    path = os.getcwd() + '/data/ex1data1.txt'
    data = pd.read_csv(path, header=None, names=['Population', 'Profit'])
    data.plot(x='Population', y='Profit', kind='scatter', figsize=(10, 10))
    plt.show()
    # append a ones column to the front of the data set
    data.insert(0, 'Ones', 1)

    # set X (training data) and y (target variable)
    cols = data.shape[1]
    X = data.iloc[:, 0:cols - 1]
    y = data.iloc[:, cols - 1:cols]
    theta = np.array([0,0])
    print(X.shape, theta.shape, y.shape)
    print(compute_cost(theta, X, y))

The first print statement (printing X.shape, theta.shape, y.shape) prints the following:

(97, 2) (2,) (97, 1)

When I try to compute the cost function I get the following:

Profit    0.0
0         0.0
1         0.0
2         0.0
3         0.0
         ... 
92        0.0
93        0.0
94        0.0
95        0.0
96        0.0
Length: 98, dtype: float64

However, I'm supposed to (according to the exercise) get 32.07 I think that the bug is related to the theta shape but I tried initializing theta like this:

theta = np.zeros(shape=(2, 1))

And still this doesn't work.. To be clear, I'm not looking for help writing the algorithm simply find the syntax bug, understand why it happened and fix it.

Answer 1

The Equation for Least Square method shall be as below-

theta(0)+theta(1).X , since you have 1 variable.

if theta(0) =0 and theta(1)=0 since you are adding it theta = np.zeros(shape=(2, 1)).

the value of Y shall be 0 hence error is 0.

To breakdown nicely you can add it like-

n = X.shape[1]
theta = np.zeros((1, n))

Now in the next run the values of theta shall change as per learning rates and in then the errors shall change too, that part you haven't reached yet, by looking at your code if I might add.

Here have a look-

https://towardsdatascience.com/linear-regression-using-python-b136c91bf0a2