# What does it mean when people say a cost function is something you want to minimize?

I am having a lot of trouble understanding this. Does it mean you should not use the cost function very often?

No, it means you are trying to find the inputs that make the output of the cost function the smallest. It doesn't mean that you should "minimize" use of it.

• Lol.... thx for clearing out this confusion. :) – jame_smith Jun 26 '16 at 9:44

A cost function is something you use to penalize high deviations from the expected results when compared to your actual predictions.

You can think of a cost function as a sign of how bad your prediction was. A high cost function value means the prediction was really off, hence, the focus on minimizing the cost function, thereby producing an accurate prediction model.

Cost functions in the context of Machine Learning often calculate some kind of metric that signifies how well your model is performing. A common one is for example the mean squared error, where you look at all your test examples where you know the true value and the predicted, take the difference between that and square it. By minimizing this error (cost function) you assume your predictions will be better.

Consider you have some data and you want to model a function that fits the data. This function should fit well and should not have error (ideally). How do I define this error? and voila here comes the cost function.

Minimize a (cost) function means that you want to find good values for its parameters. Good parameters means that the function can produce the best possible outcomes, namely the smallest ones, because small values mean less errors. This is an optimization problem: the problem of finding the best solution from all possible solutions. (source)