Residual plots: why do we want to know the error?

Question

The residual plot represents the error between the actual value.

Y-axis: residuals X-axis: the predictor variable or fitted values.

Why do we want to know the error, what benefits us in getting the residuals? I've uploaded an image from the video. I don't understand what is being sampled in the illustration.

Where is the independent and dependent variable(s)? How do you identify the error?

They take one calculation and apply to the right-side of the graph and so on.

I am taking this class and don't know anything about what the instructor is talking about.

Can someone break this all the way down? (This is a beginner's course!)

Here is the transcript from the video:

Examining the predicted value and actual value we see a difference. We obtain that value by subtracting the predicted value, and the actual target value. We then plot that value on the vertical axis with the dependent variable as the horizontal axis. Similarly, for the second sample, we repeat the process. Subtracting the target value from the predicted value. Then plotting the value accordingly. Looking at the plot gives us some insight into our data. We expect to see the results to have zero mean, distributed evenly around the x axis with similar variance. There is no curvature. This type of residual plot suggests a linear plot is appropriate

Answer 1

Let me try to give an example: Let's say we would like to predict a housing price by looking at the number of bathrooms. The x-axis represents your feature/independent variable (#bathrooms) and your y-axis represents the response/dependent variable. The points represent training samples you collected and know both the independent and dependent variables of.

Now in Linear Regression our goal is to predict the housing price based on a new example of #bathrooms we didn't see yet. For that we have to create a line through the data points that represents the training data the best. So, what qualifies a line as being a good or bad line? The answer is a cost/loss function. In linear regression we often use the squared mean of the error/residuals (MSE):

$$ \frac{1}{n} \sum (y_i - \hat{y_i})^2 $$

where $y_i$ is the true housing price and $\hat{y_i}$ is the predicted value. Our goal is to minimize this loss.

Plotting the residuals can give us many insights: For example on which training samples we make the highest errors. Also we can see how the residuals are distributed. For more complicated applications we need to check this distribution for the so called Markov Theorem which would go too much in detail.

Answer 2

Why to use Residual Plots?

A residual is a difference between the observed y-value (from scatter plot) and the predicted y-value (from regression equation line).

A residual plot is a graph that shows the residuals on the vertical axis and the independent variable on the horizontal axis. If the points in a residual plot are randomly dispersed around the horizontal axis, a linear regression model is appropriate for the data; otherwise, a non-linear model is more appropriate.

Good Fit

---

Bad Fit

---

Non-random residual patterns indicate a bad fit which suggests non-linear model may be required.

Source

1. Roberts, D. (2019). Residuals - MathBitsNotebook(A1 - CCSS Math). [online] Mathbitsnotebook.com. Available at: https://mathbitsnotebook.com/Algebra1/StatisticsReg/ST2Residuals.html.
2. Coursera. (2018). Model Evaluation using Visualization - Model Development | Coursera. [online] Available at: https://www.coursera.org/learn/data-analysis-with-python/lecture/istf4/model-evaluation-using-visualization [Accessed 9 Jan. 2020].

‌ ‌ ‌