1
$\begingroup$

I am taking Andrew Ng's Machine Learning Intro class. Looks like he changed the cost function without any explanation in the second week. Specifically:

  • He no longer squares each deviation between the estimated and the actual value
  • He no longer divides the sum of deviations by 2 (I guess because he doesn't square it, but that leaves the question of why for both
  • He multiplies the whole thing by the first number in the sample for which he is finding the delta between estimated and actual. Why?

This image might explain it better for those familiar with the class:

enter image description here

Also, which of the two is "correct"? Or if there is no "correct" answer, what is the use case for each?

Improve this question
$\endgroup$

1 Answer 1

Reset to default
2
$\begingroup$

The difference is the "original" is the derivative of a cost function and "new" is the result of substituting in an actual cost function.

When the derivative of a function raised to a power, the exponent becomes a constant. In this case, the derivative of the squared loss function becomes a constant 2, and the 2 in the numerator then cancels with the 2 in the denominator. The result is just dividing by $m$.

Improve this answer
$\endgroup$
2
  • $\begingroup$ You mean the "new" is just the derivative of the cost function, and the old is the cost function? Cool, that makes sense, thanks. Looks like I need to get a better feel for derivation. Didn't make sense to keep the sum in there for me, but I guess I know nothing. $\endgroup$
    – VSO
    Nov 8, 2019 at 20:11
  • $\begingroup$ I just realized I still don't understand where the last number in the derivative of the cost function comes from. $\endgroup$
    – VSO
    Nov 9, 2019 at 14:05

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.