I have read in Stanford's CS229 course notes that to justify the least-squares update rule with probability, the following is assumed:

$$y^{(i)} = \theta^Tx^{(i)}+\epsilon^{(i)}$$

, where $\epsilon^{(i)}$ represents random noise that is distributed i.i.d. w.r.t the Normal distribution.

I understand why $\epsilon^{(i)}$ would make sense when $h(\theta)=\theta^{T}x^{(i)}$ is a trained model, but since this assumption's eventual goal is to derive the update rule, it should make sense also when $h(\theta)$ is not trained yet. However, this assumption does not make too much sense to me when the model is arbitrary and not trained at all. Is my interpretation correct? Have I missed something? If not, how do we justify $\epsilon^{(i)}$ when the model is inaccurate (not trained)?

Thanks in advance.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.