A couple of weeks ago I joined the Standford University machine learning course on Coursera. In that course, they directly gave the cost function formula without telling how this formula was evolved. Can anyone help me by telling how that cost function has been evolved?
$\begingroup$
$\endgroup$
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
2
What I remember is that they give you more insight in future lectures, but the main reason for that is the maximum likelihood with which you try to increase the chance of making the current data set by setting the parameters. It is a good choice for setting the parameters, but its weakness is that it may overfit to your training data. You may want to take a look at Maximum Likelihood Estimation.
The cost function which is discussed there is called minimum square errors. It is found by maximum likelihood. This means that you want to increase the chance of making your training set. In other words, you want to increase the chance of $P(D|\theta)$ where $D$ can be considered as your training set. Due to the fact that your data should be iid, you can write the previous probability as $\pi p(x_i|\theta)$. You then apply some simplifications and you finally find that cost function. You can take a look at MSE as Maximum Likelihood for exact justification.
answered May 6, 2019 at 7:25
-
$\begingroup$ I don't want to know how parameters are chosen. Please explain how that complex cost function is formulated i.e. the mathematics behind cost function. $\endgroup$ May 6, 2019 at 7:41
-
$\begingroup$ @AdityaSaran I don't know whether or not you took a look at the link I've provided. The answer has been updated. $\endgroup$ May 6, 2019 at 7:58