# Learning cost function for linear regression

(Me: Never learned calculus or advanced math and I started Stanford openclasses for machine learning. I know basic matrix calculations.)

One chapter of my course is about cost function. I have been trying to find any example calculation of it with numbers. Googling only finds the same formula everytime, and also on Octave. But I want to do the same thing first with pen+paper and without it, I cannot understand. Please give me a very simple example of using the formula with numbers. Thanks a lot.

I require a cost function calculation example for following sample dataset:

#Rooms = Rent
1 = 4000
2 = 10000
3 = 22000
4 = 30000

• I think this is hard to answer if you have the formula in front of you but are saying you just don't understand it. Can you break down specifically what you are having trouble with? Jan 8 '15 at 23:42
• just added a sample dataset. Does this help? I thought cost function for linear regression is same. Jan 9 '15 at 1:01

There is a worked out example in the Wikipedia page for "simple linear regression"

Just for the sake of it, let me plug in your example into the formulas:

The fitted model should be a straight line with parameters $\alpha$ (value at $x = 0$) and $\beta$ (the slope):

$$f(x) = \alpha + \beta x$$

The values for these parameters that minimize the distance between line and data points are called $\hat{\alpha}$ and $\hat{\beta}$. They can be computed out of the data point values by using these formulae, derived here:

\begin{align} \hat{\beta} & = \frac{ \overline{xy} - \bar{x}\bar{y} }{ \overline{x^2} - \bar{x}^2 } , \\ \\ \hat{\alpha} & = \bar{y} - \hat{\beta}\bar{x} \end{align}

where an expression with an overline $\overline{xy}$ means the sample average of that expression: $\overline{xy} = \tfrac{1}{n} \sum_{i=1}^n{x_iy_i}$.

Here are the values I find for the datapoints you have listed in your question:

\begin{align} \overline{xy} &= \frac{1}{4} \sum{<(1 \times 4000), (2 \times 10000), (3 \times 22000), (4 \times 30000)>} \\ &= \frac{1}{4}(4000 + 20000 + 66000 + 120000) = 52500, \\ \overline{x} &= \frac{1}{4} \sum{<1, 2, 3, 4>} = 2.5 ,\\ \overline{y} &= \frac{1}{4} \sum{<4000, 10000, 22000, 30000>} = 16500 , \\ \overline{x^2} &= \frac{1}{4} \sum{<1^2, 2^2, 3^2, 4^2>} = 7.5 , \\ \overline{x}^2 &= 2.5^2 = 6.25 \end{align}

and the fitted line should be:

\begin{align}\ \hat{\beta} &= \frac{52500 - 2.5 \times 16500}{7.5 - 6.25} = \frac{41250}{1.25} = 33000 , \\ \hat{\alpha} &= 16500 - 33000 \times 2.5 = - 66000 , \\ \Rightarrow f(x) &= - 66000 + 33000 x \end{align}

Therefore, the model would predict, for a house with 10 rooms, a rent of:

$$f(x) = -66000 + 33000 \times 10 = 264000$$