As I'm learning about basic machine learning concepts, I've learnt about linear regression. Part of my assignment was to implement a linear regression algorithm on a rather simple dataset, consisting of only numerous data points with only one feature

data = np.array([[0.4, 3.4], [0.95, 5.8], [0.16, 2.9], [0.7, 3.6], [0.59, 3.27], [0.11, 1.89], [0.05, 4.5]])

after implementing the algorithm as I was taught:

def Linreg_sol(X, y):
   res=np.linalg.inv(X.T @ X) @ X.T @ y
   return res

I've separated the data to 2 np arrays of size n x 1 A and b, which represent the features and the predictions, and got a weight scalar (single w) and plotted the line

(i've first set the mean to 0 by deducting the appropriate mean from the features/results)

this exercise has no test set, only to train and plot the linear line.

this is the plot I got:

enter image description here

Next, I've tried standardizing the data and predicitions, by deducting appropriate means and dividing by appropriate standard diviations, recalculated the w variable, got a different one (horray!)

However, plotting the linear line with updated w gave me:

enter image description here

which oddly resembles the first line. There is a minute difference between the y points in Graph 1 and Graph 2, only in a single one as well, which I can't understand the reason for this to happen.

More information:

These are the differences I got:

x = np.arange(-0.01, 1, 0.01)
y = w_afterStandarization * (x - mean[0]) * std[1] / std[0] + 
y2=w_zeroMean * (x - mean[0]) + mean[1]
diff=[i for i in y-y2 if i != 0]

with results:

 [-4.440892098500626e-16, -4.440892098500626e-16, 
    -4.440892098500626e-16, -4.440892098500626e-16, 4.440892098500626e- 
    16, 4.440892098500626e-16, 8.881784197001252e-16]

however, I'm unsure how to interpert these differences and what they mean.

Unfortunately, I could not find any explanation for this online, hence this post.

If more pieces of my code are required, I'm happy to share.



Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.