Questions tagged [cost-function]

The tag has no usage guidance.

Filter by
Sorted by
Tagged with
0 votes
0 answers
45 views

The cost function gets stuck at 120 epochs

I did a neural network in c++ to recognize handwritten digits using the MNIST dataset without any neural network pre-existing libraries. My network has 784 inputs neuron (the pixel of the image), 100 ...
kripi's user avatar
  • 1
0 votes
0 answers
104 views

TensorFlow Gradient Tape - LookupError (Tandem Neural Network project)

I am trying to train a network by using a custom lost function, then computing the gradient of the loss, and updating the trainable variables in the reverse network (named: reverse_loaded). For ...
user151125's user avatar
1 vote
1 answer
191 views

Why COST FUNCTION AND MSE IS CALLED THE SAME?

Why are the cost function and mean squared errors called the same thing? WHEN THE COST FUNCTION IS 1/2M AND THE MSE IS 1/N. AND M=N
Rubayet Alam's user avatar
0 votes
1 answer
24 views

Are there any error functions with imbalanced negative/positive impact

I have a regression task, where positive error should be much worse than negative one. It means the importance of positive error bigger. For example, If real value is less than predicted one weights ...
Timofey's user avatar
  • 23
3 votes
2 answers
265 views

How to assign costs to the confusion matrix

I am trying to assign costs to the confusion matrix. That is, in my problem, a FP does not have the same cost as a FN, so I want to assign to these cases a cost "x" so that the algorithm ...
PicaR's user avatar
  • 304
1 vote
0 answers
361 views

Are cost functions typically normalized?

I'm very new to writing cost functions for optimization and I have what may be a basic question or just a misinterpretation. I have multiple cost functions that I'd like to add up into one total cost ...
TrapAlcubierreDrive's user avatar
3 votes
2 answers
220 views

Difference between loss and cost function in the specific context of MAE in multiple-regression?

I've often met with the Mean Absolute Error loss function when dealing with regression problems in Artificial Neural Networks, but I'm still slightly confused about the difference between the word '...
Jack Avante's user avatar
0 votes
2 answers
236 views

In practice, what is the cost function of a neural network?

I want to ask a fairly simple question I think. I have a deep background in pure mathematics, so I don't have too much trouble understanding the mathematics of the cost function, but I would just like ...
Terrence J's user avatar
1 vote
1 answer
48 views

Can anyone help me about cost function in linear regression. As from the below plot we have input values and predicted value there is no Y value, help

Can anyone help out please? I don't understand this
khushbul alam's user avatar
0 votes
1 answer
57 views

Finding global optimum of unknown and expensive function

I would like to find optimal combination of parameters for the algorithm affecting the disk space used by some storage. Therefore, several algorithm parameters (...
Vitaly Isaev's user avatar
1 vote
1 answer
111 views

Cost function - Log Loss query

What is the purpose of using "log" in the logistic regression cost function "log loss"?
Apoorva's user avatar
  • 275
0 votes
1 answer
12 views

The formula of loss function uses '(i)' as power of expected and real variables. What does that mean?

In the formula below, could one understand $y^{(i)}$ as $y_i$ ? If not, what is the fundamental difference ? $$ j(\theta_0, \theta_1) = \frac{1}{2m}\sum_{i=1}^m(h_{\theta}(x^{(i)})-y^{(i)})^2 $$
Alex Javarotti's user avatar
1 vote
1 answer
272 views

What's the correct cost function for Linear Regression

As we all know the cost function for linear regression is: Where as when we use Ridge Regression we simply add lambda*slope**2 but there I always seee the below as cost function of linear Regression ...
Chris_007's user avatar
  • 193
1 vote
0 answers
32 views

Is the Cross entropy cost function the same as the Cross entropy loss?

Is the Cross entropy cost function defined as $J(\Theta) = -\frac{1}{m}\sum_{i=1}^{m}\sum_{k=1}^{K}y_{k}^{(i)}log(\hat{p}_{k}^{(i)})$ the same as the one implemented in ...
rober_dinero's user avatar
3 votes
1 answer
396 views

Why does the MAE still remain, at all?

This may seem to be a silly question. But I just wonder why the MAE doesn't reduce to values close to 0. It's the result of an MLP with 2 hidden layers and 6 neurons per hidden layer, trying to ...
Turnvater's user avatar
0 votes
1 answer
121 views

Math of Logistic regression cost function

In the current scikit-learn documentation for binary Logistic regression there is the minimization of the following cost function: $$\min_{w, c} \frac{1}{2}w^T w + C \sum_{i=1}^n \log(\exp(- y_i (X_i^...
Riccardo D's user avatar
3 votes
1 answer
157 views

Comparison between cost functions to determine the "best" model?

I'm building an LSTM neural net for time series prediction (regression) and I am incorporating custom loss functions into training. I'm trying to determine which cost function (of 3 cost functions) ...
PyRsquared's user avatar
  • 1,594
2 votes
3 answers
603 views

Neural Network Loss Function - Mean Square Error: questions about what 'n' signifies

I'm very new to neural networks and have recently learnt about the loss functions used with neural networks. This question is in regards to the mean square error metric, defined as (from the textbook ...
Josh Lowe's user avatar
4 votes
2 answers
511 views

Cost sensitive learning and class balancing

I am facing a classification problem with classes that are really imbalanced (more or less 1% of positive cases). In addition, the "cost" of a False Negative (FN) is much higher than the ...
A1010's user avatar
  • 193
-3 votes
2 answers
434 views

Chain function in backpropagation

I'm reading a Neural Networks Tutorial. In order to answer my question you might have to take a brief look at it. I understand everything until something they declare as "chain function": ...
Alon's user avatar
  • 23
0 votes
1 answer
53 views

Norm type in cost function of ANN

I'm reading a tutorial about ANN. They use the following cost function: As you can see this equation includes a norm. I'm new to the concept of norm. My question is what kind of norm they use here (...
Alon's user avatar
  • 23
4 votes
2 answers
627 views

Regularization for intercept parameter

Why is the regularization parameter not applied to the intercept parameter? From what I have read about the cost functions for Linear and Logistic regression, the regularization parameter (λ) is ...
N.M's user avatar
  • 181
1 vote
1 answer
1k views

Reason for capping Learning Rate (alpha) up to 1 for Gradient Descent

I am learning to implement Gradient Descent algorithm in Python and came across the problem of selecting the right learning rate. I have learned that learning rates are usually selected up to 1 (...
SajidSalim's user avatar
1 vote
1 answer
88 views

Andrew Ngs Class - Why Did He Change up the Cost Function?

I am taking Andrew Ng's Machine Learning Intro class. Looks like he changed the cost function without any explanation in the second week. Specifically: He no longer squares each deviation between the ...
VSO's user avatar
  • 111
1 vote
1 answer
207 views

How do the policy gradient's cost function and gradients work?

I am not a math expert but have a basic understanding of linear algebra, calculus and probability and I understand the math behind back propagation. Currently I am trying to learn about policy ...
Eka's user avatar
  • 291
1 vote
1 answer
591 views

Policy gradient vs cost function

I was working with continuous system RL and obviously stumbled across this Policy Gradient. I want to know is this something like cost function for RL? It kinda gives that impression considering we ...
Sarvagya Gupta's user avatar
3 votes
0 answers
80 views

Neural network cost is constant never changing during training

I am trying to build a binary classifier to predict a pulsar star with Single Hidden layer Neural Network. But the cost on training dataset after almost 100 iterations has no change, following is the ...
Chinmaya B's user avatar
4 votes
2 answers
92 views

Question of using gradient descent instead of calculus. I checked previous questions there are still points to clarify

First of all I checked http://stats.stackexchange.com/questions/23128/solving-for-regression-parameters-in-closed-form-vs-gradient-descent, http://stackoverflow.com/questions/26804656/why-do-we-use-...
J.Smith's user avatar
  • 458
1 vote
1 answer
78 views

Reason behind the sum of rate factors for calculating cost function derivative

Suppose we have a network of neurons like below: We make a little change in weight w[l][j][k] on our network, and it can make change on our cost function from ...
amin msh's user avatar
  • 161
2 votes
2 answers
207 views

Two different cost functions for neural networks, how they can give the same result?

One is: $$J=-\frac{1}{m}\sum_{i=1}^{m}\sum_{k=1}^{K}\Big[y_{k}^{i}\log\big((h_{\theta}(x^{i}))_k\big)+(1-y_{k}^{i})\log\big(1-(h_{\theta}(x^{i}))_k\big)\Big]$$ The other one is: $$J=-\frac{1}{m}\sum_{...
J.Smith's user avatar
  • 458
1 vote
2 answers
80 views

Cost function - ideas

I build xgboost model for regression problem. By the default xgboost optimize $(y - y_{pred})^2$, so the RMSE will be the best eval metric to measure performance. But my task is to build the best ...
riss59's user avatar
  • 13
1 vote
3 answers
939 views

How does cost function change by choice of activation function (ReLU, Sigmoid, Softmax)?

I am new to ML and as I take courses for the area DL, I am wondering, by our choice of activation function for the last layer, whether we take sigmoid, relu or softmax, would the formula for ...
user avatar
25 votes
2 answers
23k views

Why do we have to divide by 2 in the ML squared error cost function? [duplicate]

I'm not sure why you need to multiply by $\frac1{2m}$ in the beginning. I understand that you would have to divide the whole sum by $\frac1{m}$, but why do we have to multiply $m$ by two? Is it ...
Marton Langa's user avatar
1 vote
1 answer
29 views

Decision Tree Optimize Deviation From Objective

I have the following problem: I have three classes/modes, let's call them car, bike, and walking. For any given test data instance with some environmental variables such as distance, road quality etc, ...
Layla's user avatar
  • 13
1 vote
1 answer
48 views

How is the linear regression cost function evolved?

A couple of weeks ago I joined the Standford University machine learning course on Coursera. In that course, they directly gave the cost function formula without telling how this formula was evolved. ...
Aditya Saran's user avatar
0 votes
1 answer
184 views

How to get a rebalance strategy with a cost matrix?

In the case of a classification problem where a cost matrix is used to maximize the model performance, it is common to do a rebalance technique. Let's say for example that I have the following costs ...
Tasos's user avatar
  • 3,890
3 votes
1 answer
2k views

Cost sensitive classification with individual cost

I'm currently sitting on a problem, where i'm uncertain if there is not a much simpler solution. I'm trying to train a DNN with a dataset for a classification task that should be cost sensitive. ...
T.Tos's user avatar
  • 41
1 vote
1 answer
540 views

Logitic Regression cost function - what if ln(0)?

I am building logistic regression from scrap. The simplified cost function I am using is (from machine learning course on coursera): in specific case during learning, one observation in training ...
Mateusz Konopelski's user avatar
2 votes
1 answer
89 views

Understanding minimizing cost correctly

I cannot wrap my head around this simple concept. Suppose we have a linear regression, and there is a single parameter theta to be optimized (for simplicity purposes): $h(x) = \theta \cdot x$ The ...
zafirzarya's user avatar
21 votes
1 answer
24k views

What do "compile", "fit", and "predict" do in Keras sequential models?

I am a little confused between these two parts of Keras sequential models functions. May someone explains what is exactly the job of each one? I mean ...
user3486308's user avatar
  • 1,260
3 votes
4 answers
493 views

Weights not converging while cost function has converged in neural networks

My cost/loss function drops drastically and approaches 0, which looks a sign of convergence. But the weights are still changing in a visible way, a lot faster than the cost function. Should I ensure ...
feynman's user avatar
  • 227
23 votes
1 answer
23k views

How does Gradient Descent and Backpropagation work together?

Please forgive me as I am new to this. I have attached a diagram trying to model my understanding of neural network and Back-propagation? From videos on Coursera and resources online I formed the ...
Mohamed Mahyoub's user avatar
5 votes
2 answers
360 views

What is an intuitive explanation for the log loss cost function?

I would really appreciate if someone could explain the log loss cost function And the use of it in measuring a classification model performance. I have read a few articles but most of them ...
Sai Kumar's user avatar
  • 601
3 votes
2 answers
1k views

Logistic regression cost function

In Aurelien Geron's book I found this line ...
Akash Dubey's user avatar
0 votes
1 answer
6k views

Should the minimum value of a cost (loss) function be equal to zero?

We know optimization techniques search in the space of all the possible parameters for a parameter set that minimizes the cost function of the model. The most well-known loss functions, like MSE or ...
Mo-'s user avatar
  • 1,225
1 vote
1 answer
164 views

How to Define a Cost Fucntion?

I want to define a cost function in python to identify optimum value in days when i should end a marketing campaign to save spend on campaigns not generating traffic good traffic. Problem is I dont ...
Vaibhav Thakur's user avatar
2 votes
1 answer
892 views

boosting an xgboost classifier with another xgboost classifier using different sets of features

What I would like to do, is train a first model $f_{1}(\underline{x})$, where $\underline{x}$ is a set of features, fix what model 1 has learned, and then train a second model $f_{2}(\underline{y})$ ...
gazza89's user avatar
  • 266
2 votes
0 answers
208 views

How to resolve the instability of average reward per episode in training of DQN (Deep Q-Network)?

what is shown when average reward per episode in training is unstable? If there is big difference between average reward per episode and final reward by test section, what we can say? For instance in ...
user10296606's user avatar
  • 1,824
0 votes
1 answer
398 views

Policy Gradient Methods - ScoreFunction & Log(policy)

In Policy Gradient Methods, Lecture 7 (34:15), David describes a Score Function as being the Gradient of the Log of the policy Question: If we have a Neural ...
Kari's user avatar
  • 2,706
0 votes
2 answers
654 views

Should the cost function be zero using TensorFlow's sigmoid_cross_entropy_with_logits?

I'm building a CNN to make a binary classification (1 or zero). For this, I'm using the cost function sigmoid_cross_entropy_with_logits. But for some reason, the ...
WilsonPena's user avatar