Questions tagged [cost-function]

The tag has no usage guidance.

Filter by
Sorted by
Tagged with
1 vote
0 answers
22 views

Are cost functions typically normalized?

I'm very new to writing cost functions for optimization and I have what may be a basic question or just a misinterpretation. I have multiple cost functions that I'd like to add up into one total cost ...
user avatar
0 votes
0 answers
8 views

Find suitable function to score example data based on given rule

I'm not an expert in the AI topic but for my underlying problem I need to find a function which rates data samples based on a specific value x. This means that based on the output of the function it ...
user avatar
  • 1
2 votes
2 answers
36 views

Difference between loss and cost function in the specific context of MAE in multiple-regression?

I've often met with the Mean Absolute Error loss function when dealing with regression problems in Artificial Neural Networks, but I'm still slightly confused about the difference between the word '...
user avatar
0 votes
2 answers
28 views

In practice, what is the cost function of a neural network?

I want to ask a fairly simple question I think. I have a deep background in pure mathematics, so I don't have too much trouble understanding the mathematics of the cost function, but I would just like ...
user avatar
0 votes
0 answers
18 views

How do you create a custom loss in tensorflow which uses a external tensor?

I have a problem where I want to minimize the monetary cost associated with the prediction error (Mean Error, ME) from the feature I want to predict. The monetary cost is calculated by multiplying ME ...
user avatar
  • 1
1 vote
1 answer
21 views

Can anyone help me about cost function in linear regression. As from the below plot we have input values and predicted value there is no Y value, help

Can anyone help out please? I don't understand this
user avatar
0 votes
1 answer
35 views

Finding global optimum of unknown and expensive function

I would like to find optimal combination of parameters for the algorithm affecting the disk space used by some storage. Therefore, several algorithm parameters (...
user avatar
1 vote
1 answer
35 views

Cost function - Log Loss query

What is the purpose of using "log" in the logistic regression cost function "log loss"?
user avatar
  • 313
0 votes
1 answer
10 views

The formula of loss function uses '(i)' as power of expected and real variables. What does that mean?

In the formula below, could one understand $y^{(i)}$ as $y_i$ ? If not, what is the fundamental difference ? $$ j(\theta_0, \theta_1) = \frac{1}{2m}\sum_{i=1}^m(h_{\theta}(x^{(i)})-y^{(i)})^2 $$
user avatar
1 vote
1 answer
59 views

What's the correct cost function for Linear Regression

As we all know the cost function for linear regression is: Where as when we use Ridge Regression we simply add lambda*slope**2 but there I always seee the below as cost function of linear Regression ...
user avatar
  • 163
0 votes
0 answers
27 views

Training seems to be plateauing at every learning rate

So firstly I have a network that I'm using to approximate the value of a function. Recently, at about 50000 trains, it began to show no further advancement in training, at any learning rate. The ...
user avatar
1 vote
0 answers
19 views

Is the Cross entropy cost function the same as the Cross entropy loss?

Is the Cross entropy cost function defined as $J(\Theta) = -\frac{1}{m}\sum_{i=1}^{m}\sum_{k=1}^{K}y_{k}^{(i)}log(\hat{p}_{k}^{(i)})$ the same as the one implemented in ...
user avatar
0 votes
0 answers
178 views

Implementation cost function in logistic regression in python using numpy

I am implementing the cost function for logistic regression and have a question. The formulation for cost function is $J = -\frac{1}{m}\sum_{i=1}^{m}(y^{(i)}\log(a^{(i)})+(1-y^{(i)})\log(1-a^{(i)}))$ ...
user avatar
3 votes
1 answer
361 views

Why does the MAE still remain, at all?

This may seem to be a silly question. But I just wonder why the MAE doesn't reduce to values close to 0. It's the result of an MLP with 2 hidden layers and 6 neurons per hidden layer, trying to ...
user avatar
0 votes
1 answer
68 views

Math of Logistic regression cost function

In the current scikit-learn documentation for binary Logistic regression there is the minimization of the following cost function: $$\min_{w, c} \frac{1}{2}w^T w + C \sum_{i=1}^n \log(\exp(- y_i (X_i^...
user avatar
3 votes
1 answer
103 views

Comparison between cost functions to determine the "best" model?

I'm building an LSTM neural net for time series prediction (regression) and I am incorporating custom loss functions into training. I'm trying to determine which cost function (of 3 cost functions) ...
user avatar
  • 1,484
2 votes
3 answers
325 views

Neural Network Loss Function - Mean Square Error: questions about what 'n' signifies

I'm very new to neural networks and have recently learnt about the loss functions used with neural networks. This question is in regards to the mean square error metric, defined as (from the textbook ...
user avatar
3 votes
2 answers
352 views

Cost sensitive learning and class balancing

I am facing a classification problem with classes that are really imbalanced (more or less 1% of positive cases). In addition, the "cost" of a False Negative (FN) is much higher than the ...
user avatar
  • 183
-3 votes
2 answers
388 views

Chain function in backpropagation

I'm reading a Neural Networks Tutorial. In order to answer my question you might have to take a brief look at it. I understand everything until something they declare as "chain function": ...
user avatar
  • 23
0 votes
1 answer
22 views

Norm type in cost function of ANN

I'm reading a tutorial about ANN. They use the following cost function: As you can see this equation includes a norm. I'm new to the concept of norm. My question is what kind of norm they use here (...
user avatar
  • 23
2 votes
2 answers
180 views

Regularization for intercept parameter

Why is the regularization parameter not applied to the intercept parameter? From what I have read about the cost functions for Linear and Logistic regression, the regularization parameter (λ) is ...
user avatar
  • 161
1 vote
1 answer
610 views

Reason for capping Learning Rate (alpha) up to 1 for Gradient Descent

I am learning to implement Gradient Descent algorithm in Python and came across the problem of selecting the right learning rate. I have learned that learning rates are usually selected up to 1 (...
user avatar
1 vote
1 answer
66 views

Andrew Ngs Class - Why Did He Change up the Cost Function?

I am taking Andrew Ng's Machine Learning Intro class. Looks like he changed the cost function without any explanation in the second week. Specifically: He no longer squares each deviation between the ...
user avatar
  • 111
1 vote
1 answer
146 views

How do the policy gradient's cost function and gradients work?

I am not a math expert but have a basic understanding of linear algebra, calculus and probability and I understand the math behind back propagation. Currently I am trying to learn about policy ...
user avatar
  • 261
1 vote
1 answer
545 views

Policy gradient vs cost function

I was working with continuous system RL and obviously stumbled across this Policy Gradient. I want to know is this something like cost function for RL? It kinda gives that impression considering we ...
user avatar
3 votes
0 answers
72 views

Neural network cost is constant never changing during training

I am trying to build a binary classifier to predict a pulsar star with Single Hidden layer Neural Network. But the cost on training dataset after almost 100 iterations has no change, following is the ...
user avatar
4 votes
2 answers
77 views

Question of using gradient descent instead of calculus. I checked previous questions there are still points to clarify

First of all I checked http://stats.stackexchange.com/questions/23128/solving-for-regression-parameters-in-closed-form-vs-gradient-descent, http://stackoverflow.com/questions/26804656/why-do-we-use-...
user avatar
  • 438
1 vote
1 answer
67 views

Reason behind the sum of rate factors for calculating cost function derivative

Suppose we have a network of neurons like below: We make a little change in weight w[l][j][k] on our network, and it can make change on our cost function from ...
user avatar
  • 141
2 votes
2 answers
145 views

Two different cost functions for neural networks, how they can give the same result?

One is: $$J=-\frac{1}{m}\sum_{i=1}^{m}\sum_{k=1}^{K}\Big[y_{k}^{i}\log\big((h_{\theta}(x^{i}))_k\big)+(1-y_{k}^{i})\log\big(1-(h_{\theta}(x^{i}))_k\big)\Big]$$ The other one is: $$J=-\frac{1}{m}\sum_{...
user avatar
  • 438
1 vote
2 answers
56 views

Cost function - ideas

I build xgboost model for regression problem. By the default xgboost optimize $(y - y_{pred})^2$, so the RMSE will be the best eval metric to measure performance. But my task is to build the best ...
user avatar
  • 13
0 votes
3 answers
659 views

How does cost function change by choice of activation function (ReLU, Sigmoid, Softmax)?

I am new to ML and as I take courses for the area DL, I am wondering, by our choice of activation function for the last layer, whether we take sigmoid, relu or softmax, would the formula for ...
user avatar
21 votes
2 answers
14k views

Why do we have to divide by 2 in the ML squared error cost function? [duplicate]

I'm not sure why you need to multiply by $\frac1{2m}$ in the beginning. I understand that you would have to divide the whole sum by $\frac1{m}$, but why do we have to multiply $m$ by two? Is it ...
user avatar
1 vote
1 answer
26 views

Decision Tree Optimize Deviation From Objective

I have the following problem: I have three classes/modes, let's call them car, bike, and walking. For any given test data instance with some environmental variables such as distance, road quality etc, ...
user avatar
  • 13
1 vote
1 answer
46 views

How is the linear regression cost function evolved?

A couple of weeks ago I joined the Standford University machine learning course on Coursera. In that course, they directly gave the cost function formula without telling how this formula was evolved. ...
user avatar
0 votes
1 answer
119 views

How to get a rebalance strategy with a cost matrix?

In the case of a classification problem where a cost matrix is used to maximize the model performance, it is common to do a rebalance technique. Let's say for example that I have the following costs ...
user avatar
  • 3,680
3 votes
1 answer
2k views

Cost sensitive classification with individual cost

I'm currently sitting on a problem, where i'm uncertain if there is not a much simpler solution. I'm trying to train a DNN with a dataset for a classification task that should be cost sensitive. ...
user avatar
  • 41
1 vote
1 answer
429 views

Logitic Regression cost function - what if ln(0)?

I am building logistic regression from scrap. The simplified cost function I am using is (from machine learning course on coursera): in specific case during learning, one observation in training ...
user avatar
2 votes
1 answer
80 views

Understanding minimizing cost correctly

I cannot wrap my head around this simple concept. Suppose we have a linear regression, and there is a single parameter theta to be optimized (for simplicity purposes): $h(x) = \theta \cdot x$ The ...
user avatar
20 votes
1 answer
20k views

What do "compile", "fit", and "predict" do in Keras sequential models?

I am a little confused between these two parts of Keras sequential models functions. May someone explains what is exactly the job of each one? I mean ...
user avatar
  • 1,132
3 votes
4 answers
355 views

Weights not converging while cost function has converged in neural networks

My cost/loss function drops drastically and approaches 0, which looks a sign of convergence. But the weights are still changing in a visible way, a lot faster than the cost function. Should I ensure ...
user avatar
  • 227
15 votes
1 answer
17k views

How does Gradient Descent and Backpropagation work together?

Please forgive me as I am new to this. I have attached a diagram trying to model my understanding of neural network and Back-propagation? From videos on Coursera and resources online I formed the ...
user avatar
5 votes
2 answers
225 views

What is an intuitive explanation for the log loss cost function?

I would really appreciate if someone could explain the log loss cost function And the use of it in measuring a classification model performance. I have read a few articles but most of them ...
user avatar
  • 571
3 votes
2 answers
645 views

Logistic regression cost function

In Aurelien Geron's book I found this line ...
user avatar
0 votes
1 answer
5k views

Should the minimum value of a cost (loss) function be equal to zero?

We know optimization techniques search in the space of all the possible parameters for a parameter set that minimizes the cost function of the model. The most well-known loss functions, like MSE or ...
user avatar
  • 1,165
1 vote
1 answer
128 views

How to Define a Cost Fucntion?

I want to define a cost function in python to identify optimum value in days when i should end a marketing campaign to save spend on campaigns not generating traffic good traffic. Problem is I dont ...
user avatar
2 votes
1 answer
605 views

boosting an xgboost classifier with another xgboost classifier using different sets of features

What I would like to do, is train a first model $f_{1}(\underline{x})$, where $\underline{x}$ is a set of features, fix what model 1 has learned, and then train a second model $f_{2}(\underline{y})$ ...
user avatar
  • 266
2 votes
0 answers
179 views

How to resolve the instability of average reward per episode in training of DQN (Deep Q-Network)?

what is shown when average reward per episode in training is unstable? If there is big difference between average reward per episode and final reward by test section, what we can say? For instance in ...
user avatar
  • 1,516
0 votes
1 answer
305 views

Policy Gradient Methods - ScoreFunction & Log(policy)

In Policy Gradient Methods, Lecture 7 (34:15), David describes a Score Function as being the Gradient of the Log of the policy Question: If we have a Neural ...
user avatar
  • 2,546
0 votes
2 answers
596 views

Should the cost function be zero using TensorFlow's sigmoid_cross_entropy_with_logits?

I'm building a CNN to make a binary classification (1 or zero). For this, I'm using the cost function sigmoid_cross_entropy_with_logits. But for some reason, the ...
user avatar
7 votes
2 answers
13k views

What is the Time Complexity of Linear Regression?

I am working with linear regression and I would like to know the Time complexity in big-O notation. The cost function of linear regression without an optimisation algorithm (such as Gradient descent) ...
user avatar