3
$\begingroup$

Currently I am going through Normal Equation in Machine Learning.

$$ \hat\theta = (X^T \cdot X)^{-1} \cdot X^T \cdot y $$

But when I see how they use this equation, I found they always add an additional column of 1s in the starting of matrix X before transposing.

I don't understand why. What's the logic behind this?

The places where I found such things

1) Coursera - Theory

2) Implementation

Now let’s compute using the Normal Equation. We will use the inv() function from NumPy’s Linear Algebra module (np.linalg) to compute the inverse of a matrix, and the dot() method for matrix multiplication: X_b = np.c_[ np.ones(( 100, 1)), X] # add x0 = 1 to each instance

Géron, Aurélien. Hands-On Machine Learning with Scikit-Learn and TensorFlow: Concepts, Tools, and Techniques to Build Intelligent Systems (p. 111). O'Reilly Media. Kindle Edition.

$\endgroup$
  • 1
    $\begingroup$ They represent intercept term, also called bias. $\endgroup$ – Vaalizaadeh Feb 18 '18 at 7:30
6
$\begingroup$

The normal equations are designed such that each coefficient in the model has an input of some kind it's being multiplied against. The column of ones is the "input" to the intercept term.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.