Skip to main content
Share Your Experience: Take the 2024 Developer Survey

Questions tagged [cost-function]

The tag has no usage guidance.

Filter by
Sorted by
Tagged with
4 votes
1 answer
107 views

What cost optimisation problem is solved by F score?

I know the general expression of the F1-score: $$F1 = \frac{precision * recall}{precision + recall}$$ And its $F_{beta}$ variants (see: https://en.wikipedia.org/wiki/F-score): $$F_{beta} = (1+\beta^2) ...
Lucas Morin's user avatar
  • 2,234
0 votes
0 answers
47 views

The cost function gets stuck at 120 epochs

I did a neural network in c++ to recognize handwritten digits using the MNIST dataset without any neural network pre-existing libraries. My network has 784 inputs neuron (the pixel of the image), 100 ...
kripi's user avatar
  • 1
0 votes
0 answers
161 views

TensorFlow Gradient Tape - LookupError (Tandem Neural Network project)

I am trying to train a network by using a custom lost function, then computing the gradient of the loss, and updating the trainable variables in the reverse network (named: reverse_loaded). For ...
user151125's user avatar
1 vote
1 answer
628 views

Why COST FUNCTION AND MSE IS CALLED THE SAME?

Why are the cost function and mean squared errors called the same thing? WHEN THE COST FUNCTION IS 1/2M AND THE MSE IS 1/N. AND M=N
Rubayet Alam's user avatar
0 votes
1 answer
27 views

Are there any error functions with imbalanced negative/positive impact

I have a regression task, where positive error should be much worse than negative one. It means the importance of positive error bigger. For example, If real value is less than predicted one weights ...
Timofey's user avatar
  • 23
3 votes
2 answers
360 views

How to assign costs to the confusion matrix

I am trying to assign costs to the confusion matrix. That is, in my problem, a FP does not have the same cost as a FN, so I want to assign to these cases a cost "x" so that the algorithm ...
PicaR's user avatar
  • 314
1 vote
0 answers
480 views

Are cost functions typically normalized?

I'm very new to writing cost functions for optimization and I have what may be a basic question or just a misinterpretation. I have multiple cost functions that I'd like to add up into one total cost ...
TrapAlcubierreDrive's user avatar
3 votes
2 answers
285 views

Difference between loss and cost function in the specific context of MAE in multiple-regression?

I've often met with the Mean Absolute Error loss function when dealing with regression problems in Artificial Neural Networks, but I'm still slightly confused about the difference between the word '...
Jack Avante's user avatar
0 votes
2 answers
295 views

In practice, what is the cost function of a neural network?

I want to ask a fairly simple question I think. I have a deep background in pure mathematics, so I don't have too much trouble understanding the mathematics of the cost function, but I would just like ...
Terrence J's user avatar
1 vote
1 answer
59 views

Can anyone help me about cost function in linear regression. As from the below plot we have input values and predicted value there is no Y value, help

Can anyone help out please? I don't understand this
khushbul alam's user avatar
0 votes
1 answer
67 views

Finding global optimum of unknown and expensive function

I would like to find optimal combination of parameters for the algorithm affecting the disk space used by some storage. Therefore, several algorithm parameters (...
Vitaly Isaev's user avatar
1 vote
1 answer
117 views

Cost function - Log Loss query

What is the purpose of using "log" in the logistic regression cost function "log loss"?
Apoorva's user avatar
  • 307
0 votes
1 answer
12 views

The formula of loss function uses '(i)' as power of expected and real variables. What does that mean?

In the formula below, could one understand $y^{(i)}$ as $y_i$ ? If not, what is the fundamental difference ? $$ j(\theta_0, \theta_1) = \frac{1}{2m}\sum_{i=1}^m(h_{\theta}(x^{(i)})-y^{(i)})^2 $$
Alex Javarotti's user avatar
1 vote
1 answer
377 views

What's the correct cost function for Linear Regression

As we all know the cost function for linear regression is: Where as when we use Ridge Regression we simply add lambda*slope**2 but there I always seee the below as cost function of linear Regression ...
Chris_007's user avatar
  • 193
1 vote
0 answers
37 views

Is the Cross entropy cost function the same as the Cross entropy loss?

Is the Cross entropy cost function defined as $J(\Theta) = -\frac{1}{m}\sum_{i=1}^{m}\sum_{k=1}^{K}y_{k}^{(i)}log(\hat{p}_{k}^{(i)})$ the same as the one implemented in ...
rober_dinero's user avatar
3 votes
1 answer
404 views

Why does the MAE still remain, at all?

This may seem to be a silly question. But I just wonder why the MAE doesn't reduce to values close to 0. It's the result of an MLP with 2 hidden layers and 6 neurons per hidden layer, trying to ...
Turnvater's user avatar
0 votes
1 answer
129 views

Math of Logistic regression cost function

In the current scikit-learn documentation for binary Logistic regression there is the minimization of the following cost function: $$\min_{w, c} \frac{1}{2}w^T w + C \sum_{i=1}^n \log(\exp(- y_i (X_i^...
Riccardo D's user avatar
3 votes
1 answer
189 views

Comparison between cost functions to determine the "best" model?

I'm building an LSTM neural net for time series prediction (regression) and I am incorporating custom loss functions into training. I'm trying to determine which cost function (of 3 cost functions) ...
PyRsquared's user avatar
  • 1,604
2 votes
3 answers
643 views

Neural Network Loss Function - Mean Square Error: questions about what 'n' signifies

I'm very new to neural networks and have recently learnt about the loss functions used with neural networks. This question is in regards to the mean square error metric, defined as (from the textbook ...
Josh Lowe's user avatar
4 votes
2 answers
583 views

Cost sensitive learning and class balancing

I am facing a classification problem with classes that are really imbalanced (more or less 1% of positive cases). In addition, the "cost" of a False Negative (FN) is much higher than the ...
A1010's user avatar
  • 193
-3 votes
2 answers
484 views

Chain function in backpropagation

I'm reading a Neural Networks Tutorial. In order to answer my question you might have to take a brief look at it. I understand everything until something they declare as "chain function": ...
Alon's user avatar
  • 23
0 votes
1 answer
61 views

Norm type in cost function of ANN

I'm reading a tutorial about ANN. They use the following cost function: As you can see this equation includes a norm. I'm new to the concept of norm. My question is what kind of norm they use here (...
Alon's user avatar
  • 23
4 votes
2 answers
758 views

Regularization for intercept parameter

Why is the regularization parameter not applied to the intercept parameter? From what I have read about the cost functions for Linear and Logistic regression, the regularization parameter (λ) is ...
N.M's user avatar
  • 191
1 vote
1 answer
1k views

Reason for capping Learning Rate (alpha) up to 1 for Gradient Descent

I am learning to implement Gradient Descent algorithm in Python and came across the problem of selecting the right learning rate. I have learned that learning rates are usually selected up to 1 (...
SajidSalim's user avatar
1 vote
1 answer
97 views

Andrew Ngs Class - Why Did He Change up the Cost Function?

I am taking Andrew Ng's Machine Learning Intro class. Looks like he changed the cost function without any explanation in the second week. Specifically: He no longer squares each deviation between the ...
VSO's user avatar
  • 111
1 vote
1 answer
216 views

How do the policy gradient's cost function and gradients work?

I am not a math expert but have a basic understanding of linear algebra, calculus and probability and I understand the math behind back propagation. Currently I am trying to learn about policy ...
Eka's user avatar
  • 301
1 vote
1 answer
604 views

Policy gradient vs cost function

I was working with continuous system RL and obviously stumbled across this Policy Gradient. I want to know is this something like cost function for RL? It kinda gives that impression considering we ...
Sarvagya Gupta's user avatar
3 votes
0 answers
84 views

Neural network cost is constant never changing during training

I am trying to build a binary classifier to predict a pulsar star with Single Hidden layer Neural Network. But the cost on training dataset after almost 100 iterations has no change, following is the ...
Chinmaya B's user avatar
4 votes
2 answers
102 views

Question of using gradient descent instead of calculus. I checked previous questions there are still points to clarify

First of all I checked http://stats.stackexchange.com/questions/23128/solving-for-regression-parameters-in-closed-form-vs-gradient-descent, http://stackoverflow.com/questions/26804656/why-do-we-use-...
J.Smith's user avatar
  • 458
1 vote
1 answer
81 views

Reason behind the sum of rate factors for calculating cost function derivative

Suppose we have a network of neurons like below: We make a little change in weight w[l][j][k] on our network, and it can make change on our cost function from ...
amin msh's user avatar
  • 171
2 votes
2 answers
235 views

Two different cost functions for neural networks, how they can give the same result?

One is: $$J=-\frac{1}{m}\sum_{i=1}^{m}\sum_{k=1}^{K}\Big[y_{k}^{i}\log\big((h_{\theta}(x^{i}))_k\big)+(1-y_{k}^{i})\log\big(1-(h_{\theta}(x^{i}))_k\big)\Big]$$ The other one is: $$J=-\frac{1}{m}\sum_{...
J.Smith's user avatar
  • 458
1 vote
2 answers
82 views

Cost function - ideas

I build xgboost model for regression problem. By the default xgboost optimize $(y - y_{pred})^2$, so the RMSE will be the best eval metric to measure performance. But my task is to build the best ...
riss59's user avatar
  • 13
1 vote
3 answers
1k views

How does cost function change by choice of activation function (ReLU, Sigmoid, Softmax)?

I am new to ML and as I take courses for the area DL, I am wondering, by our choice of activation function for the last layer, whether we take sigmoid, relu or softmax, would the formula for ...
user avatar
26 votes
2 answers
25k views

Why do we have to divide by 2 in the ML squared error cost function? [duplicate]

I'm not sure why you need to multiply by $\frac1{2m}$ in the beginning. I understand that you would have to divide the whole sum by $\frac1{m}$, but why do we have to multiply $m$ by two? Is it ...
Marton Langa's user avatar
1 vote
1 answer
29 views

Decision Tree Optimize Deviation From Objective

I have the following problem: I have three classes/modes, let's call them car, bike, and walking. For any given test data instance with some environmental variables such as distance, road quality etc, ...
Layla's user avatar
  • 13
1 vote
1 answer
48 views

How is the linear regression cost function evolved?

A couple of weeks ago I joined the Standford University machine learning course on Coursera. In that course, they directly gave the cost function formula without telling how this formula was evolved. ...
Aditya Saran's user avatar
0 votes
1 answer
195 views

How to get a rebalance strategy with a cost matrix?

In the case of a classification problem where a cost matrix is used to maximize the model performance, it is common to do a rebalance technique. Let's say for example that I have the following costs ...
Tasos's user avatar
  • 3,930
3 votes
1 answer
2k views

Cost sensitive classification with individual cost

I'm currently sitting on a problem, where i'm uncertain if there is not a much simpler solution. I'm trying to train a DNN with a dataset for a classification task that should be cost sensitive. ...
T.Tos's user avatar
  • 41
1 vote
1 answer
562 views

Logitic Regression cost function - what if ln(0)?

I am building logistic regression from scrap. The simplified cost function I am using is (from machine learning course on coursera): in specific case during learning, one observation in training ...
Mateusz Konopelski's user avatar
2 votes
1 answer
91 views

Understanding minimizing cost correctly

I cannot wrap my head around this simple concept. Suppose we have a linear regression, and there is a single parameter theta to be optimized (for simplicity purposes): $h(x) = \theta \cdot x$ The ...
zafirzarya's user avatar
21 votes
1 answer
25k views

What do "compile", "fit", and "predict" do in Keras sequential models?

I am a little confused between these two parts of Keras sequential models functions. May someone explains what is exactly the job of each one? I mean ...
user3486308's user avatar
  • 1,280
3 votes
4 answers
555 views

Weights not converging while cost function has converged in neural networks

My cost/loss function drops drastically and approaches 0, which looks a sign of convergence. But the weights are still changing in a visible way, a lot faster than the cost function. Should I ensure ...
feynman's user avatar
  • 237
23 votes
1 answer
24k views

How does Gradient Descent and Backpropagation work together?

Please forgive me as I am new to this. I have attached a diagram trying to model my understanding of neural network and Back-propagation? From videos on Coursera and resources online I formed the ...
Mohamed Mahyoub's user avatar
5 votes
2 answers
466 views

What is an intuitive explanation for the log loss cost function?

I would really appreciate if someone could explain the log loss cost function And the use of it in measuring a classification model performance. I have read a few articles but most of them ...
Sai Kumar's user avatar
  • 611
4 votes
2 answers
1k views

Logistic regression cost function

In Aurelien Geron's book I found this line ...
Akash Dubey's user avatar
0 votes
1 answer
6k views

Should the minimum value of a cost (loss) function be equal to zero?

We know optimization techniques search in the space of all the possible parameters for a parameter set that minimizes the cost function of the model. The most well-known loss functions, like MSE or ...
Mo-'s user avatar
  • 1,255
1 vote
1 answer
167 views

How to Define a Cost Fucntion?

I want to define a cost function in python to identify optimum value in days when i should end a marketing campaign to save spend on campaigns not generating traffic good traffic. Problem is I dont ...
Vaibhav Thakur's user avatar
2 votes
1 answer
1k views

boosting an xgboost classifier with another xgboost classifier using different sets of features

What I would like to do, is train a first model $f_{1}(\underline{x})$, where $\underline{x}$ is a set of features, fix what model 1 has learned, and then train a second model $f_{2}(\underline{y})$ ...
gazza89's user avatar
  • 266
2 votes
0 answers
221 views

How to resolve the instability of average reward per episode in training of DQN (Deep Q-Network)?

what is shown when average reward per episode in training is unstable? If there is big difference between average reward per episode and final reward by test section, what we can say? For instance in ...
user10296606's user avatar
  • 1,844
0 votes
1 answer
425 views

Policy Gradient Methods - ScoreFunction & Log(policy)

In Policy Gradient Methods, Lecture 7 (34:15), David describes a Score Function as being the Gradient of the Log of the policy Question: If we have a Neural ...
Kari's user avatar
  • 2,736