I was trying to code the Linear Regression in Python using Matrix Multiplication method using Gradient Descent and followed a code where there was no mention what is the loss but just a code as Per Iteration:
y_hat = X.dot(W) + b dW = - (2 * (X^T ).dot(Y - Y_hat)) / m # how does the minus and matrix multiplications are used instead of Summation? db = - (2 * np.sum(Y - Y_hat)) / m # np is numpy W = W - lr * dW # update weights b = b - lr * db
I know from the code is that
dW is derivative of the weight matrix per iteration,
X^T is the Transpose of the X features,
Y is original values of
Y_hat are predicted values using the formula
What I want to know is that
dW = - (2 * (X^T ).dot(Y - Y_hat)) / m. Even with the
MSE loss, as given in this link in the equation 1.4, it should be something else.
Can someone please elaborate how are the values of
dW,db are calculated here?