0
$\begingroup$

As I'm learning about basic machine learning concepts, I've learnt about linear regression. Part of my assignment was to implement a linear regression algorithm on a rather simple dataset, consisting of only numerous data points with only one feature

data = np.array([[0.4, 3.4], [0.95, 5.8], [0.16, 2.9], [0.7, 3.6], [0.59, 3.27], [0.11, 1.89], [0.05, 4.5]])

after implementing the algorithm as I was taught:

def Linreg_sol(X, y):
   res=np.linalg.inv(X.T @ X) @ X.T @ y
   return res

I've separated the data to 2 np arrays of size n x 1 A and b, which represent the features and the predictions, and got a weight scalar (single w) and plotted the line

(i've first set the mean to 0 by deducting the appropriate mean from the features/results)

this exercise has no test set, only to train and plot the linear line.

this is the plot I got:

enter image description here

Next, I've tried standardizing the data and predicitions, by deducting appropriate means and dividing by appropriate standard diviations, recalculated the w variable, got a different one (horray!)

However, plotting the linear line with updated w gave me:

enter image description here

which oddly resembles the first line. There is a minute difference between the y points in Graph 1 and Graph 2, only in a single one as well, which I can't understand the reason for this to happen.

More information:

These are the differences I got:

x = np.arange(-0.01, 1, 0.01)
y = w_afterStandarization * (x - mean[0]) * std[1] / std[0] + 
mean[1]
y2=w_zeroMean * (x - mean[0]) + mean[1]
diff=[i for i in y-y2 if i != 0]
print(diff)

with results:

 [-4.440892098500626e-16, -4.440892098500626e-16, 
    -4.440892098500626e-16, -4.440892098500626e-16, 4.440892098500626e- 
    16, 4.440892098500626e-16, 8.881784197001252e-16]

however, I'm unsure how to interpert these differences and what they mean.

Unfortunately, I could not find any explanation for this online, hence this post.

If more pieces of my code are required, I'm happy to share.

$\endgroup$

0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.