Questions tagged [cost-function]

The tag has no usage guidance.

Filter by
Sorted by
Tagged with
0
votes
1answer
25 views

Andrew Ngs Class - Why Did He Change up the Cost Function?

I am taking Andrew Ng's Machine Learning Intro class. Looks like he changed the cost function without any explanation in the second week. Specifically: He no longer squares each deviation between the ...
0
votes
0answers
14 views

Theano gradient descent and cost function issue

Sorry but can anyone point out whats wrong with this code? ...
1
vote
1answer
31 views

How do the policy gradient's cost function and gradients work?

I am not a math expert but have a basic understanding of linear algebra, calculus and probability and I understand the math behind back propagation. Currently I am trying to learn about policy ...
1
vote
1answer
71 views

Policy gradient vs cost function

I was working with continuous system RL and obviously stumbled across this Policy Gradient. I want to know is this something like cost function for RL? It kinda gives that impression considering we ...
2
votes
0answers
55 views

Neural network cost is constant never changing during training

I am trying to build a binary classifier to predict a pulsar star with Single Hidden layer Neural Network. But the cost on training dataset after almost 100 iterations has no change, following is the ...
4
votes
2answers
43 views

Question of using gradient descent instead of calculus. I checked previous questions there are still points to clarify

First of all I checked http://stats.stackexchange.com/questions/23128/solving-for-regression-parameters-in-closed-form-vs-gradient-descent, http://stackoverflow.com/questions/26804656/why-do-we-use-...
1
vote
1answer
25 views

Reason behind the sum of rate factors for calculating cost function derivative

Suppose we have a network of neurons like below: We make a little change in weight w[l][j][k] on our network, and it can make change on our cost function from ...
0
votes
0answers
41 views

Minimum cost is not zero when calculating cross-entropy on soft labels

I am training a neural network using batches of soft labels, e.g. y = [[0.00, 0.25, 0.25, 0.50], ... [0.75, 0.00, 0.20, 0.05]] However, as ...
0
votes
0answers
24 views

Understanding about RNN loss

Problem I am now taking Andrew Ng's deep learning course on Coursera. Everything is great but when it comes to RNN, I sometimes feel confused. Here is a question about RNN (or more specifically, the ...
2
votes
2answers
57 views

Two different cost functions for neural networks, how they can give the same result?

One is: $$J=-\frac{1}{m}\sum_{i=1}^{m}\sum_{k=1}^{K}\Big[y_{k}^{i}\log\big((h_{\theta}(x^{i}))_k\big)+(1-y_{k}^{i})\log\big(1-(h_{\theta}(x^{i}))_k\big)\Big]$$ The other one is: $$J=-\frac{1}{m}\sum_{...
1
vote
2answers
38 views

Cost function - ideas

I build xgboost model for regression problem. By the default xgboost optimize $(y - y_{pred})^2$, so the RMSE will be the best eval metric to measure performance. But my task is to build the best ...
0
votes
3answers
50 views

How does cost function change by choice of activation function (ReLU, Sigmoid, Softmax)?

I am new to ML and as I take courses for the area DL, I am wondering, by our choice of activation function for the last layer, whether we take sigmoid, relu or softmax, would the formula for ...
0
votes
0answers
17 views

What's the correct way of implementation of cost function and gradient function in logistical regression after regularisation?

This is the cost function of logistic regression: which i could implement correctly, with the code : ...
5
votes
2answers
1k views

Why do we have to divide by 2 in the ML squared error cost function? [duplicate]

I'm not sure why you need to multiply by $\frac1{2m}$ in the beginning. I understand that you would have to divide the whole sum by $\frac1{m}$, but why do we have to multiply $m$ by two? Is it ...
1
vote
1answer
19 views

Decision Tree Optimize Deviation From Objective

I have the following problem: I have three classes/modes, let's call them car, bike, and walking. For any given test data instance with some environmental variables such as distance, road quality etc, ...
1
vote
1answer
28 views

How is the linear regression cost function evolved?

A couple of weeks ago I joined the Standford University machine learning course on Coursera. In that course, they directly gave the cost function formula without telling how this formula was evolved. ...
0
votes
1answer
33 views

How to get a rebalance strategy with a cost matrix?

In the case of a classification problem where a cost matrix is used to maximize the model performance, it is common to do a rebalance technique. Let's say for example that I have the following costs ...
3
votes
1answer
738 views

Cost sensitive classification with individual cost

I'm currently sitting on a problem, where i'm uncertain if there is not a much simpler solution. I'm trying to train a DNN with a dataset for a classification task that should be cost sensitive. ...
1
vote
1answer
77 views

Logitic Regression cost function - what if ln(0)?

I am building logistic regression from scrap. The simplified cost function I am using is (from machine learning course on coursera): in specific case during learning, one observation in training ...
2
votes
1answer
59 views

Understanding minimizing cost correctly

I cannot wrap my head around this simple concept. Suppose we have a linear regression, and there is a single parameter theta to be optimized (for simplicity purposes): $h(x) = \theta \cdot x$ The ...
5
votes
1answer
3k views

What do “compile” , “fit” and “predict” do in Keras sequential models?

I am a little confused between these two parts of Keras sequential models functions. May someone explains what is exactly the job of each one? I mean ...
0
votes
0answers
14 views

cost/loss function being a multi-well function in neural networks

In a 0 or 1 binary classification problem using neural networks, given the activation function is taken as sigmoid, if one takes the cost/loss function as sum of square of differences, the loss ...
3
votes
4answers
127 views

Weights not converging while cost function has converged in neural networks

My cost/loss function drops drastically and approaches 0, which looks a sign of convergence. But the weights are still changing in a visible way, a lot faster than the cost function. Should I ensure ...
3
votes
1answer
2k views

How does Gradient Descent and Backpropagation work together?

Please forgive me as I am new to this. I have attached a diagram trying to model my understanding of neural network and Back-propagation? From videos on coursera and resources online I formed the ...
5
votes
2answers
158 views

What is an intuitive explanation for the log loss cost function?

I would really appreciate if someone could explain the log loss cost function And the use of it in measuring a classification model performance. I have read a few articles but most of them ...
3
votes
2answers
243 views

Logistic regression cost function

In Aurelien Geron's book I found this line ...
0
votes
1answer
1k views

Should the minimum value of a cost (loss) function be equal to zero?

We know optimization techniques search in the space of all the possible parameters for a parameter set that minimizes the cost function of the model. The most well-known loss functions, like MSE or ...
1
vote
1answer
73 views

How to Define a Cost Fucntion?

I want to define a cost function in python to identify optimum value in days when i should end a marketing campaign to save spend on campaigns not generating traffic good traffic. Problem is I dont ...
2
votes
1answer
300 views

boosting an xgboost classifier with another xgboost classifier using different sets of features

What I would like to do, is train a first model $f_{1}(\underline{x})$, where $\underline{x}$ is a set of features, fix what model 1 has learned, and then train a second model $f_{2}(\underline{y})$ ...
2
votes
0answers
125 views

How to resolve the instability of average reward per episode in training of DQN (Deep Q-Network)?

what is shown when average reward per episode in training is unstable? If there is big difference between average reward per episode and final reward by test section, what we can say? For ...
0
votes
1answer
127 views

Policy Gradient Methods - ScoreFunction & Log(policy)

In Policy Gradient Methods, Lecture 7 (34:15), David describes a Score Function as being the Gradient of the Log of the policy Question: If we have a Neural ...
0
votes
2answers
404 views

Should the cost function be zero using TensorFlow's sigmoid_cross_entropy_with_logits?

I'm building a CNN to make a binary classification (1 or zero). For this, I'm using the cost function sigmoid_cross_entropy_with_logits. But for some reason, the ...
4
votes
1answer
4k views

What is the Time Complexity of Linear Regression?

I am working with linear regression and I would like to know the Time complexity in big-O notation. The cost function of linear regression without an optimisation algorithm (such as Gradient descent) ...
1
vote
2answers
89 views

Why do we double the number in a quadratic cost function or MSE?

$$ C(w,b) = \frac{1}{2n}\sum_{x}||y(x)-a||^2 $$ Where y is a 10-dimensional vector, a is the output, w is the weight and b is the bias and n is the number of inputs. If this is the MSE, shouldn't it ...
4
votes
2answers
2k views

What are the cases where it is fine to initialize all weights to zero

I've taken a few online courses in machine learning, and in general, the advice has been to choose random weights for a neural network to ensure that your neurons don't all learn the same thing, ...
2
votes
0answers
322 views

Cost/loss functions for multi-tasking regression neural networks

The mean square loss function is the standard for regression neural networks. However, if I have a neural network learning two tasks (two outputs) at once, is it more advisable to train on the sum of ...
1
vote
1answer
422 views

asymmetric cost function for deep neural network binary classifier

I am building a deep neural network based binary classifier, with single output. The loss function I actually want to minimize is $$ \mathcal L(\hat y,y) = \begin{cases} 0, & \text{if $\hat y$ = ...
2
votes
2answers
738 views

logistic regression algorithm fails to work

I'm trying to code my own logistic regression algorithm using Andrew NG's machine learning using Octave. lectures. So what I did was make a csv file, the first row being some parameter and the second ...
0
votes
1answer
187 views

Derivation of the cross-entropy equation in Michael Nielsen's book

I am reading the book http://neuralnetworksanddeeplearning.com/chap3.html by Michael Nielsen. So this is a question mostly for the people familiar with the book and understanding the material. In the ...
1
vote
1answer
1k views

Gradient Descent in logistic regression

Logistic and Linear Regression have different cost functions. But I don't get how the gradient descent in logistic regression is the same as Linear Regression. We get the Gradient Descent formula by ...
2
votes
2answers
1k views

Softmax classifier never allows for 100% probability in LSTM?

When working with LSTM I am using a softmax classifier and a one-hot encoded vector approach. The softmax looks like this: $$S(h_i) = \frac{e^{h_i}}{\sum e^{h_{total}}}$$ notice, LSTM's result is a $...
0
votes
1answer
140 views

Custom c++ LSTM slows down at 0.36 cost is usual?

Am I missing a part of the puzzle? I have implemented LSTM in c++ which steadily decreases in error, but slows down at the certain error value. It also seems to predict most of the characters, but ...
1
vote
0answers
100 views

How to decrease the learning rate only when cost function is stagnant?

I'm training a very complex function in tensorflow. Is there a way to decrease the learning rate only when the cost isin't decreasing very much?
1
vote
1answer
639 views

Logistic Regression : Solving the cross-entropy cost function analytically

Logistic regression cost function is cross-entropy. It is defined as below: This is a convex function. To reach the minimum, scikit-learn provides multiple types of solvers such as : ‘liblinear’ ...
8
votes
1answer
2k views

Cost function for Ordinal Regression using neural networks

What is the best cost function to train a neural network to perform ordinal regression, i.e. to predict a result whose value exists on an arbitrary scale where only the relative ordering between ...
1
vote
1answer
1k views

Understanding Logistic Regression Cost function

Linear Regression cost function: $$J(\theta) = \frac{1}{2 m} \sum_{i=1}^m (h_{\theta}(x^{(i)}) - y^{(i)})^2$$ where: $$h_{\theta}(x) = \theta_0 + \theta_1 x_1$$ Logistic Regression cost function $...
18
votes
3answers
20k views

Python implementation of cost function in logistic regression: why dot multiplication in one expression but element-wise multiplication in another

I have a very basic question which relates to Python, numpy and multiplication of matrices in the setting of logistic regression. First, let me apologise for not using math notation. I am confused ...
1
vote
1answer
822 views

Is cross-entropy a good cost function if I'm interested in the probabilities of a sample belonging to a certain class?

I'm training a neural network that, for each of six classes, tries to predict the probability that a sample belongs to it. After that, I want to use these probabilities as fractions of the sample ...
0
votes
1answer
92 views

Any Neural Network implementations that allow for a cost function of more than just the network output?

I have an application of a straightforward MLP, for which the cost function is a function of both the network output, in addition to another value calculated from the network weights (actually the ...
3
votes
2answers
6k views

XGBoost change loss function

I'm using XGBoost (through the sklearn API) and I'm trying to do a binary classification. False Positives are much worse for me than False Negatives, how can I take this into account? The API ...