Questions tagged [cost-function]
The cost-function tag has no usage guidance.
74
questions
0
votes
1
answer
15
views
How to tune the classification threshold in a cost-sensitive manner?
I have trained a classifier outputting probabilities for each class. I want to tune the decision threshold in such a way that it accounts for different costs/gains assigned to false positives ($FP$), $...
- 706
4
votes
1
answer
108
views
What cost optimisation problem is solved by F score?
I know the general expression of the F1-score:
$$F1 = \frac{precision * recall}{precision + recall}$$
And its $F_{beta}$ variants (see: https://en.wikipedia.org/wiki/F-score):
$$F_{beta} = (1+\beta^2) ...
- 2,358
0
votes
0
answers
65
views
The cost function gets stuck at 120 epochs
I did a neural network in c++ to recognize handwritten digits using the MNIST dataset without any neural network pre-existing libraries. My network has 784 inputs neuron (the pixel of the image), 100 ...
- 1
1
vote
1
answer
850
views
Why COST FUNCTION AND MSE IS CALLED THE SAME?
Why are the cost function and mean squared errors called the same thing? WHEN THE COST FUNCTION IS 1/2M AND THE MSE IS 1/N. AND M=N
- 11
0
votes
1
answer
28
views
Are there any error functions with imbalanced negative/positive impact
I have a regression task, where positive error should be much worse than negative one. It means the importance of positive error bigger. For example, If real value is less than predicted one weights ...
- 23
3
votes
2
answers
420
views
How to assign costs to the confusion matrix
I am trying to assign costs to the confusion matrix. That is, in my problem, a FP does not have the same cost as a FN, so I want to assign to these cases a cost "x" so that the algorithm ...
- 314
1
vote
0
answers
548
views
Are cost functions typically normalized?
I'm very new to writing cost functions for optimization and I have what may be a basic question or just a misinterpretation.
I have multiple cost functions that I'd like to add up into one total cost ...
3
votes
2
answers
315
views
Difference between loss and cost function in the specific context of MAE in multiple-regression?
I've often met with the Mean Absolute Error loss function when dealing with regression problems in Artificial Neural Networks, but I'm still slightly confused about the difference between the word '...
- 133
0
votes
2
answers
320
views
In practice, what is the cost function of a neural network?
I want to ask a fairly simple question I think. I have a deep background in pure mathematics, so I don't have too much trouble understanding the mathematics of the cost function, but I would just like ...
- 101
1
vote
1
answer
67
views
Can anyone help me about cost function in linear regression. As from the below plot we have input values and predicted value there is no Y value, help
Can anyone help out please? I don't understand this
0
votes
1
answer
76
views
Finding global optimum of unknown and expensive function
I would like to find optimal combination of parameters for the algorithm affecting the disk space used by some storage. Therefore, several algorithm parameters (...
- 103
1
vote
1
answer
122
views
Cost function - Log Loss query
What is the purpose of using "log" in the logistic regression cost function "log loss"?
- 317
0
votes
1
answer
12
views
The formula of loss function uses '(i)' as power of expected and real variables. What does that mean?
In the formula below, could one understand $y^{(i)}$ as $y_i$ ? If not, what is the fundamental difference ?
$$ j(\theta_0, \theta_1) = \frac{1}{2m}\sum_{i=1}^m(h_{\theta}(x^{(i)})-y^{(i)})^2 $$
1
vote
1
answer
414
views
What's the correct cost function for Linear Regression
As we all know the cost function for linear regression is:
Where as when we use Ridge Regression we simply add lambda*slope**2 but there I always seee the below as cost function of linear Regression ...
- 193
1
vote
0
answers
39
views
Is the Cross entropy cost function the same as the Cross entropy loss?
Is the Cross entropy cost function defined as
$J(\Theta) = -\frac{1}{m}\sum_{i=1}^{m}\sum_{k=1}^{K}y_{k}^{(i)}log(\hat{p}_{k}^{(i)})$ the same as the one implemented in ...
- 121
3
votes
1
answer
407
views
Why does the MAE still remain, at all?
This may seem to be a silly question. But I just wonder why the MAE doesn't reduce to values close to 0.
It's the result of an MLP with 2 hidden layers and 6 neurons per hidden layer, trying to ...
- 48
0
votes
1
answer
130
views
Math of Logistic regression cost function
In the current scikit-learn documentation for binary Logistic regression there is the minimization of the following cost function:
$$\min_{w, c} \frac{1}{2}w^T w + C \sum_{i=1}^n \log(\exp(- y_i (X_i^...
- 113
3
votes
1
answer
209
views
Comparison between cost functions to determine the "best" model?
I'm building an LSTM neural net for time series prediction (regression) and I am incorporating custom loss functions into training. I'm trying to determine which cost function (of 3 cost functions) ...
- 1,614
2
votes
3
answers
649
views
Neural Network Loss Function - Mean Square Error: questions about what 'n' signifies
I'm very new to neural networks and have recently learnt about the loss functions used with neural networks.
This question is in regards to the mean square error metric, defined as (from the textbook ...
- 29
4
votes
2
answers
606
views
Cost sensitive learning and class balancing
I am facing a classification problem with classes that are really imbalanced (more or less 1% of positive cases). In addition, the "cost" of a False Negative (FN) is much higher than the ...
- 193
-3
votes
2
answers
502
views
Chain function in backpropagation
I'm reading a Neural Networks Tutorial. In order to answer my question you might have to take a brief look at it.
I understand everything until something they declare as "chain function":
...
- 23
0
votes
1
answer
71
views
Norm type in cost function of ANN
I'm reading a tutorial about ANN. They use the following cost function:
As you can see this equation includes a norm. I'm new to the concept of norm.
My question is what kind of norm they use here (...
- 23
4
votes
2
answers
797
views
Regularization for intercept parameter
Why is the regularization parameter not applied to the intercept parameter?
From what I have read about the cost functions for Linear and Logistic regression, the regularization parameter (λ) is ...
- 191
1
vote
1
answer
1k
views
Reason for capping Learning Rate (alpha) up to 1 for Gradient Descent
I am learning to implement Gradient Descent algorithm in Python and came across the problem of selecting the right learning rate.
I have learned that learning rates are usually selected up to 1 (...
- 113
1
vote
1
answer
103
views
Andrew Ngs Class - Why Did He Change up the Cost Function?
I am taking Andrew Ng's Machine Learning Intro class. Looks like he changed the cost function without any explanation in the second week. Specifically:
He no longer squares each deviation between the ...
- 111
1
vote
1
answer
219
views
How do the policy gradient's cost function and gradients work?
I am not a math expert but have a basic understanding of linear algebra, calculus and probability and I understand the math behind back propagation. Currently I am trying to learn about policy ...
- 301
1
vote
1
answer
609
views
Policy gradient vs cost function
I was working with continuous system RL and obviously stumbled across this Policy Gradient.
I want to know is this something like cost function for RL? It kinda gives that impression considering we ...
- 113
3
votes
0
answers
84
views
Neural network cost is constant never changing during training
I am trying to build a binary classifier to predict a pulsar star with Single Hidden layer Neural Network.
But the cost on training dataset after almost 100 iterations has no change, following is the ...
- 141
4
votes
2
answers
105
views
Question of using gradient descent instead of calculus. I checked previous questions there are still points to clarify
First of all I checked http://stats.stackexchange.com/questions/23128/solving-for-regression-parameters-in-closed-form-vs-gradient-descent, http://stackoverflow.com/questions/26804656/why-do-we-use-...
- 468
1
vote
1
answer
82
views
Reason behind the sum of rate factors for calculating cost function derivative
Suppose we have a network of neurons like below:
We make a little change in weight w[l][j][k] on our network, and it can make change on our cost function from ...
- 171
2
votes
2
answers
244
views
Two different cost functions for neural networks, how they can give the same result?
One is: $$J=-\frac{1}{m}\sum_{i=1}^{m}\sum_{k=1}^{K}\Big[y_{k}^{i}\log\big((h_{\theta}(x^{i}))_k\big)+(1-y_{k}^{i})\log\big(1-(h_{\theta}(x^{i}))_k\big)\Big]$$
The other one is: $$J=-\frac{1}{m}\sum_{...
- 468
1
vote
2
answers
83
views
Cost function - ideas
I build xgboost model for regression problem. By the default xgboost optimize $(y - y_{pred})^2$, so the RMSE will be the best eval metric to measure performance. But my task is to build the best ...
- 13
1
vote
3
answers
1k
views
How does cost function change by choice of activation function (ReLU, Sigmoid, Softmax)?
I am new to ML and as I take courses for the area DL, I am wondering, by our choice of activation function for the last layer, whether we take sigmoid, relu or softmax, would the formula for ...
user77241
26
votes
2
answers
26k
views
Why do we have to divide by 2 in the ML squared error cost function? [duplicate]
I'm not sure why you need to multiply by $\frac1{2m}$ in the beginning. I understand that you would have to divide the whole sum by $\frac1{m}$, but why do we have to multiply $m$ by two?
Is it ...
- 363
1
vote
1
answer
30
views
Decision Tree Optimize Deviation From Objective
I have the following problem: I have three classes/modes, let's call them car, bike, and walking. For any given test data instance with some environmental variables such as distance, road quality etc, ...
- 13
1
vote
1
answer
49
views
How is the linear regression cost function evolved?
A couple of weeks ago I joined the Standford University machine learning course on Coursera. In that course, they directly gave the cost function formula without telling how this formula was evolved. ...
- 113
0
votes
1
answer
203
views
How to get a rebalance strategy with a cost matrix?
In the case of a classification problem where a cost matrix is used to maximize the model performance, it is common to do a rebalance technique.
Let's say for example that I have the following costs ...
- 3,930
3
votes
1
answer
2k
views
Cost sensitive classification with individual cost
I'm currently sitting on a problem, where i'm uncertain if there is not a much simpler solution.
I'm trying to train a DNN with a dataset for a classification task that should be cost sensitive. ...
- 41
1
vote
1
answer
568
views
Logitic Regression cost function - what if ln(0)?
I am building logistic regression from scrap.
The simplified cost function I am using is (from machine learning course on coursera):
in specific case during learning,
one observation in training ...
2
votes
1
answer
93
views
Understanding minimizing cost correctly
I cannot wrap my head around this simple concept.
Suppose we have a linear regression, and there is a single parameter theta to be optimized (for simplicity purposes):
$h(x) = \theta \cdot x$
The ...
- 33
21
votes
1
answer
25k
views
What do "compile", "fit", and "predict" do in Keras sequential models?
I am a little confused between these two parts of Keras sequential models functions. May someone explains what is exactly the job of each one? I mean ...
- 1,300
3
votes
4
answers
567
views
Weights not converging while cost function has converged in neural networks
My cost/loss function drops drastically and approaches 0, which looks a sign of convergence. But the weights are still changing in a visible way, a lot faster than the cost function. Should I ensure ...
- 237
24
votes
1
answer
25k
views
How does Gradient Descent and Backpropagation work together?
Please forgive me as I am new to this. I have attached a diagram trying to model my understanding of neural network and Back-propagation? From videos on Coursera and resources online I formed the ...
- 355
5
votes
2
answers
567
views
What is an intuitive explanation for the log loss cost function?
I would really appreciate if someone could explain the log loss cost function And the use of it in measuring a classification model performance.
I have read a few articles but most of them ...
- 611
4
votes
2
answers
1k
views
0
votes
1
answer
7k
views
Should the minimum value of a cost (loss) function be equal to zero?
We know optimization techniques search in the space of all
the possible parameters for a parameter set that minimizes the cost function of the model. The most well-known loss functions, like MSE or ...
- 1,255
1
vote
1
answer
168
views
How to Define a Cost Fucntion?
I want to define a cost function in python to identify optimum value in days when i should end a marketing campaign to save spend on campaigns not generating traffic good traffic.
Problem is I dont ...
- 2,363
2
votes
1
answer
1k
views
boosting an xgboost classifier with another xgboost classifier using different sets of features
What I would like to do, is train a first model $f_{1}(\underline{x})$, where $\underline{x}$ is a set of features, fix what model 1 has learned, and then train a second model $f_{2}(\underline{y})$ ...
- 266
2
votes
0
answers
228
views
How to resolve the instability of average reward per episode in training of DQN (Deep Q-Network)?
what is shown when average reward per episode in training is unstable?
If there is big difference between average reward per episode and final reward by test section, what we can say?
For instance in ...
- 1,854
0
votes
1
answer
435
views
Policy Gradient Methods - ScoreFunction & Log(policy)
In Policy Gradient Methods, Lecture 7 (34:15), David describes a Score Function as being the Gradient of the Log of the policy
Question:
If we have a Neural ...
- 2,756