Mathematical bias and weight vs machine learning bias and weight

Question

I am a little confused about the term Bias and Weight with respect to machine learning.

Say we want to predict the heights of people whose weights are given. So plot weights to x-axis and height to yaxis. To find out the linear relationship between height and weight we draw a straight line that shows the relationship between height and weight.

Using the equation for a line, you could write down this relationship

$y= mx+b ...i)$ more specifically in the machine learning terminology it could be $y= b+ w_1x_1...ii)$

So here b is the bias as per machine learning. However, b is the y-intercept as per mathematics. By defination b can be defined in machine learning A value indicating how far apart the average of predictions is from the average of labels in the dataset. Is that mean bias(b) is the distance between some particular point of the red line(as per picture) to the true point(say a blue or green point).

Now another confusion if this is the case then what is loss? As per defination loss is A measure of how far a model's predictions are from its label. Then what is the difference between loss and bias?

Now, for weight, here, weight(m)means slope as per equation i). Mathematically slope can be define as

$m = \frac{rise}{run} =\frac{y_2 - y_1}{x_2 - x_1}$

However, weight$(w_1)$ can be defined in machine learning as A coefficient for a feature in a linear model. So my confusion is that is the procedure of finding weight is same as finding slope in mathematics?

Answer 1

Okay so let's start with the first question:

Is that mean bias(b) is the distance between some particular point of the red line(as per picture) to the true point(say a blue or green point).

You'd be correct had you used the word difference rather than distance. Bias is the difference between the estimated value and the true value. Think of it in this way, if your weighing machine always showed 5kg less for every measurement then you'd be adding 5 to every weight it measures, as a way to offset all measures made by the machine. So if your data is 'biased' towards one side then this bias term offsets this behaviour by adding/subtracting the 'biases of the weights. Here's some more rigorous explanation to it.

what is the difference between loss and bias?

Loss, on the other hand, is a measure of how close your estimator is to true prediction, there are different loss function that one can employ with different type of training, for linear regression we can use MSE. Using loss functions we can objectively optimize our weights and biases. So, having an appropriate value of bias(and weight) causes our loss to be lower, thereby increasing the accuracy of our model.

So my confusion is that is the procedure of finding weight is same as finding slope in mathematics?

The procedure is different, but the end result is the same. To see why you'd need to realize is that you are calculating $\hat y$ as the following: Given $m,b$ and a point $x$, calculate the value of $\hat y$ which is then used to calculate the loss. This value along with $m$ and $b$ is then used to calculate a better value for $m$ and $b$ using Gradient Descent. Notice at no point are we talking about directly calculating $m$ using $\hat y$, We are iteratively calculating better values for weights and biases. The end result? For single-dimension data, the weight that the model has learnt is indeed the slope (when plotted) and bias is the y-intercept. This stands true as the number of dimension increases but then rather than being single values, it becomes a multi-dimensional vector.