Questions tagged [cost-function]
The cost-function tag has no usage guidance.
71
questions
0
votes
1
answer
22
views
Are there any error functions with imbalanced negative/positive impact
I have a regression task, where positive error should be much worse than negative one. It means the importance of positive error bigger. For example, If real value is less than predicted one weights ...
- 23
0
votes
0
answers
7
views
L2 and L1 regularization: Why avoiding mutual cancelations of errors cause the skewness (bias) can't be determined?
I'm trying to understand about the L1 and L2 regularization and in this paper (https://arxiv.org/pdf/1809.03006.pdf) the authors mentioned the problem with using the mean absolute error or mean ...
- 101
3
votes
2
answers
183
views
How to assign costs to the confusion matrix
I am trying to assign costs to the confusion matrix. That is, in my problem, a FP does not have the same cost as a FN, so I want to assign to these cases a cost "x" so that the algorithm ...
- 302
1
vote
0
answers
220
views
Are cost functions typically normalized?
I'm very new to writing cost functions for optimization and I have what may be a basic question or just a misinterpretation.
I have multiple cost functions that I'd like to add up into one total cost ...
3
votes
2
answers
168
views
Difference between loss and cost function in the specific context of MAE in multiple-regression?
I've often met with the Mean Absolute Error loss function when dealing with regression problems in Artificial Neural Networks, but I'm still slightly confused about the difference between the word '...
- 133
0
votes
2
answers
144
views
In practice, what is the cost function of a neural network?
I want to ask a fairly simple question I think. I have a deep background in pure mathematics, so I don't have too much trouble understanding the mathematics of the cost function, but I would just like ...
- 101
1
vote
1
answer
40
views
Can anyone help me about cost function in linear regression. As from the below plot we have input values and predicted value there is no Y value, help
Can anyone help out please? I don't understand this
0
votes
1
answer
49
views
Finding global optimum of unknown and expensive function
I would like to find optimal combination of parameters for the algorithm affecting the disk space used by some storage. Therefore, several algorithm parameters (...
- 103
1
vote
1
answer
91
views
Cost function - Log Loss query
What is the purpose of using "log" in the logistic regression cost function "log loss"?
- 275
0
votes
1
answer
12
views
The formula of loss function uses '(i)' as power of expected and real variables. What does that mean?
In the formula below, could one understand $y^{(i)}$ as $y_i$ ? If not, what is the fundamental difference ?
$$ j(\theta_0, \theta_1) = \frac{1}{2m}\sum_{i=1}^m(h_{\theta}(x^{(i)})-y^{(i)})^2 $$
1
vote
1
answer
192
views
What's the correct cost function for Linear Regression
As we all know the cost function for linear regression is:
Where as when we use Ridge Regression we simply add lambda*slope**2 but there I always seee the below as cost function of linear Regression ...
- 193
1
vote
0
answers
29
views
Is the Cross entropy cost function the same as the Cross entropy loss?
Is the Cross entropy cost function defined as
$J(\Theta) = -\frac{1}{m}\sum_{i=1}^{m}\sum_{k=1}^{K}y_{k}^{(i)}log(\hat{p}_{k}^{(i)})$ the same as the one implemented in ...
- 121
3
votes
1
answer
386
views
Why does the MAE still remain, at all?
This may seem to be a silly question. But I just wonder why the MAE doesn't reduce to values close to 0.
It's the result of an MLP with 2 hidden layers and 6 neurons per hidden layer, trying to ...
- 48
0
votes
1
answer
105
views
Math of Logistic regression cost function
In the current scikit-learn documentation for binary Logistic regression there is the minimization of the following cost function:
$$\min_{w, c} \frac{1}{2}w^T w + C \sum_{i=1}^n \log(\exp(- y_i (X_i^...
- 113
3
votes
1
answer
132
views
Comparison between cost functions to determine the "best" model?
I'm building an LSTM neural net for time series prediction (regression) and I am incorporating custom loss functions into training. I'm trying to determine which cost function (of 3 cost functions) ...
- 1,564
2
votes
3
answers
552
views
Neural Network Loss Function - Mean Square Error: questions about what 'n' signifies
I'm very new to neural networks and have recently learnt about the loss functions used with neural networks.
This question is in regards to the mean square error metric, defined as (from the textbook ...
- 29
4
votes
2
answers
465
views
Cost sensitive learning and class balancing
I am facing a classification problem with classes that are really imbalanced (more or less 1% of positive cases). In addition, the "cost" of a False Negative (FN) is much higher than the ...
- 193
-3
votes
2
answers
409
views
Chain function in backpropagation
I'm reading a Neural Networks Tutorial. In order to answer my question you might have to take a brief look at it.
I understand everything until something they declare as "chain function":
...
- 23
0
votes
1
answer
45
views
Norm type in cost function of ANN
I'm reading a tutorial about ANN. They use the following cost function:
As you can see this equation includes a norm. I'm new to the concept of norm.
My question is what kind of norm they use here (...
- 23
4
votes
2
answers
477
views
Regularization for intercept parameter
Why is the regularization parameter not applied to the intercept parameter?
From what I have read about the cost functions for Linear and Logistic regression, the regularization parameter (λ) is ...
- 181
1
vote
1
answer
905
views
Reason for capping Learning Rate (alpha) up to 1 for Gradient Descent
I am learning to implement Gradient Descent algorithm in Python and came across the problem of selecting the right learning rate.
I have learned that learning rates are usually selected up to 1 (...
- 113
1
vote
1
answer
79
views
Andrew Ngs Class - Why Did He Change up the Cost Function?
I am taking Andrew Ng's Machine Learning Intro class. Looks like he changed the cost function without any explanation in the second week. Specifically:
He no longer squares each deviation between the ...
- 111
1
vote
1
answer
193
views
How do the policy gradient's cost function and gradients work?
I am not a math expert but have a basic understanding of linear algebra, calculus and probability and I understand the math behind back propagation. Currently I am trying to learn about policy ...
- 291
1
vote
1
answer
582
views
Policy gradient vs cost function
I was working with continuous system RL and obviously stumbled across this Policy Gradient.
I want to know is this something like cost function for RL? It kinda gives that impression considering we ...
- 113
3
votes
0
answers
76
views
Neural network cost is constant never changing during training
I am trying to build a binary classifier to predict a pulsar star with Single Hidden layer Neural Network.
But the cost on training dataset after almost 100 iterations has no change, following is the ...
- 141
4
votes
2
answers
85
views
Question of using gradient descent instead of calculus. I checked previous questions there are still points to clarify
First of all I checked http://stats.stackexchange.com/questions/23128/solving-for-regression-parameters-in-closed-form-vs-gradient-descent, http://stackoverflow.com/questions/26804656/why-do-we-use-...
- 458
1
vote
1
answer
76
views
Reason behind the sum of rate factors for calculating cost function derivative
Suppose we have a network of neurons like below:
We make a little change in weight w[l][j][k] on our network, and it can make change on our cost function from ...
- 161
2
votes
2
answers
186
views
Two different cost functions for neural networks, how they can give the same result?
One is: $$J=-\frac{1}{m}\sum_{i=1}^{m}\sum_{k=1}^{K}\Big[y_{k}^{i}\log\big((h_{\theta}(x^{i}))_k\big)+(1-y_{k}^{i})\log\big(1-(h_{\theta}(x^{i}))_k\big)\Big]$$
The other one is: $$J=-\frac{1}{m}\sum_{...
- 458
1
vote
2
answers
76
views
Cost function - ideas
I build xgboost model for regression problem. By the default xgboost optimize $(y - y_{pred})^2$, so the RMSE will be the best eval metric to measure performance. But my task is to build the best ...
- 13
1
vote
3
answers
825
views
How does cost function change by choice of activation function (ReLU, Sigmoid, Softmax)?
I am new to ML and as I take courses for the area DL, I am wondering, by our choice of activation function for the last layer, whether we take sigmoid, relu or softmax, would the formula for ...
user77241
25
votes
2
answers
20k
views
Why do we have to divide by 2 in the ML squared error cost function? [duplicate]
I'm not sure why you need to multiply by $\frac1{2m}$ in the beginning. I understand that you would have to divide the whole sum by $\frac1{m}$, but why do we have to multiply $m$ by two?
Is it ...
- 353
1
vote
1
answer
29
views
Decision Tree Optimize Deviation From Objective
I have the following problem: I have three classes/modes, let's call them car, bike, and walking. For any given test data instance with some environmental variables such as distance, road quality etc, ...
- 13
1
vote
1
answer
48
views
How is the linear regression cost function evolved?
A couple of weeks ago I joined the Standford University machine learning course on Coursera. In that course, they directly gave the cost function formula without telling how this formula was evolved. ...
- 113
0
votes
1
answer
176
views
How to get a rebalance strategy with a cost matrix?
In the case of a classification problem where a cost matrix is used to maximize the model performance, it is common to do a rebalance technique.
Let's say for example that I have the following costs ...
- 3,850
3
votes
1
answer
2k
views
Cost sensitive classification with individual cost
I'm currently sitting on a problem, where i'm uncertain if there is not a much simpler solution.
I'm trying to train a DNN with a dataset for a classification task that should be cost sensitive. ...
- 41
1
vote
1
answer
512
views
Logitic Regression cost function - what if ln(0)?
I am building logistic regression from scrap.
The simplified cost function I am using is (from machine learning course on coursera):
in specific case during learning,
one observation in training ...
2
votes
1
answer
85
views
Understanding minimizing cost correctly
I cannot wrap my head around this simple concept.
Suppose we have a linear regression, and there is a single parameter theta to be optimized (for simplicity purposes):
$h(x) = \theta \cdot x$
The ...
- 33
21
votes
1
answer
23k
views
What do "compile", "fit", and "predict" do in Keras sequential models?
I am a little confused between these two parts of Keras sequential models functions. May someone explains what is exactly the job of each one? I mean ...
- 1,260
3
votes
4
answers
455
views
Weights not converging while cost function has converged in neural networks
My cost/loss function drops drastically and approaches 0, which looks a sign of convergence. But the weights are still changing in a visible way, a lot faster than the cost function. Should I ensure ...
- 227
23
votes
1
answer
21k
views
How does Gradient Descent and Backpropagation work together?
Please forgive me as I am new to this. I have attached a diagram trying to model my understanding of neural network and Back-propagation? From videos on Coursera and resources online I formed the ...
- 345
5
votes
2
answers
276
views
What is an intuitive explanation for the log loss cost function?
I would really appreciate if someone could explain the log loss cost function And the use of it in measuring a classification model performance.
I have read a few articles but most of them ...
- 601
3
votes
2
answers
810
views
0
votes
1
answer
6k
views
Should the minimum value of a cost (loss) function be equal to zero?
We know optimization techniques search in the space of all
the possible parameters for a parameter set that minimizes the cost function of the model. The most well-known loss functions, like MSE or ...
- 1,225
1
vote
1
answer
158
views
How to Define a Cost Fucntion?
I want to define a cost function in python to identify optimum value in days when i should end a marketing campaign to save spend on campaigns not generating traffic good traffic.
Problem is I dont ...
- 2,333
2
votes
1
answer
722
views
boosting an xgboost classifier with another xgboost classifier using different sets of features
What I would like to do, is train a first model $f_{1}(\underline{x})$, where $\underline{x}$ is a set of features, fix what model 1 has learned, and then train a second model $f_{2}(\underline{y})$ ...
- 266
2
votes
0
answers
198
views
How to resolve the instability of average reward per episode in training of DQN (Deep Q-Network)?
what is shown when average reward per episode in training is unstable?
If there is big difference between average reward per episode and final reward by test section, what we can say?
For instance in ...
- 1,754
0
votes
1
answer
367
views
Policy Gradient Methods - ScoreFunction & Log(policy)
In Policy Gradient Methods, Lecture 7 (34:15), David describes a Score Function as being the Gradient of the Log of the policy
Question:
If we have a Neural ...
- 2,686
0
votes
2
answers
642
views
Should the cost function be zero using TensorFlow's sigmoid_cross_entropy_with_logits?
I'm building a CNN to make a binary classification (1 or zero). For this, I'm using the cost function sigmoid_cross_entropy_with_logits.
But for some reason, the ...
- 121
10
votes
2
answers
16k
views
What is the Time Complexity of Linear Regression?
I am working with linear regression and I would like to know the Time complexity in big-O notation. The cost function of linear regression without an optimisation algorithm (such as Gradient descent) ...
- 149
1
vote
2
answers
257
views
Why do we double the number in a quadratic cost function or MSE?
$$ C(w,b) = \frac{1}{2n}\sum_{x}||y(x)-a||^2 $$
Where y is a 10-dimensional vector, a is the output, w is the weight and b is the bias and n is the number of inputs. If this is the MSE, shouldn't it ...
- 113