Stack Exchange Network

Stack Exchange network consists of 175 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [cost-function]

The tag has no usage guidance.

0
votes
0answers
7 views

What's the correct way of implementation of cost function and gradient function in logistical regression after regularisation?

This is the cost function of logistic regression: which i could implement correctly, with the code : ...
4
votes
2answers
174 views

Why do we have to divide by 2 in the ML squared error cost function? [duplicate]

I'm not sure why you need to multiply by $\frac1{2m}$ in the beginning. I understand that you would have to divide the whole sum by $\frac1{m}$, but why do we have to multiply $m$ by two? Is it ...
1
vote
1answer
13 views

Decision Tree Optimize Deviation From Objective

I have the following problem: I have three classes/modes, let's call them car, bike, and walking. For any given test data instance with some environmental variables such as distance, road quality etc, ...
1
vote
1answer
18 views

How is the linear regression cost function evolved?

A couple of weeks ago I joined the Standford University machine learning course on Coursera. In that course, they directly gave the cost function formula without telling how this formula was evolved. ...
2
votes
1answer
86 views

Cost sensitive classification with individual cost

I'm currently sitting on a problem, where i'm uncertain if there is not a much simpler solution. I'm trying to train a DNN with a dataset for a classification task that should be cost sensitive. ...
1
vote
1answer
22 views

Logitic Regression cost function - what if ln(0)?

I am building logistic regression from scrap. The simplified cost function I am using is (from machine learning course on coursera): in specific case during learning, one observation in training ...
2
votes
1answer
55 views

Understanding minimizing cost correctly

I cannot wrap my head around this simple concept. Suppose we have a linear regression, and there is a single parameter theta to be optimized (for simplicity purposes): $h(x) = \theta \cdot x$ The ...
4
votes
1answer
306 views

What do “compile” , “fit” and “predict” do in Keras sequential models?

I am a little confused between these two parts of Keras sequential models functions. May someone explains what is exactly the job of each one? I mean ...
0
votes
0answers
13 views

cost/loss function being a multi-well function in neural networks

In a 0 or 1 binary classification problem using neural networks, given the activation function is taken as sigmoid, if one takes the cost/loss function as sum of square of differences, the loss ...
3
votes
4answers
120 views

Weights not converging while cost function has converged in neural networks

My cost/loss function drops drastically and approaches 0, which looks a sign of convergence. But the weights are still changing in a visible way, a lot faster than the cost function. Should I ensure ...
2
votes
1answer
86 views

How does Gradient Descent and Backpropagation work together?

Please forgive me as I am new to this. I have attached a diagram trying to model my understanding of neural network and Back-propagation? From videos on coursera and resources online I formed the ...
5
votes
2answers
113 views

What is an intuitive explanation for the log loss cost function?

I would really appreciate if someone could explain the log loss cost function And the use of it in measuring a classification model performance. I have read a few articles but most of them ...
3
votes
2answers
127 views

Logistic regression cost function

In Aurelien Geron's book I found this line ...
0
votes
1answer
373 views

Should the minimum value of a cost (loss) function be equal to zero?

We know optimization techniques search in the space of all the possible parameters for a parameter set that minimizes the cost function of the model. The most well-known loss functions, like MSE or ...
0
votes
1answer
57 views

How to Define a Cost Fucntion?

I want to define a cost function in python to identify optimum value in days when i should end a marketing campaign to save spend on campaigns not generating traffic good traffic. Problem is I dont ...
1
vote
1answer
170 views

boosting an xgboost classifier with another xgboost classifier using different sets of features

What I would like to do, is train a first model $f_{1}(\underline{x})$, where $\underline{x}$ is a set of features, fix what model 1 has learned, and then train a second model $f_{2}(\underline{y})$ ...
2
votes
0answers
97 views

How to resolve the instability of average reward per episode in training of DQN (Deep Q-Network)?

what is shown when average reward per episode in training is unstable? If there is big difference between average reward per episode and final reward by test section, what we can say? For ...
0
votes
1answer
88 views

Policy Gradient Methods - ScoreFunction & Log(policy)

In Policy Gradient Methods, Lecture 7 (34:15), David describes a Score Function as being the Gradient of the Log of the policy Question: If we have a Neural ...
0
votes
1answer
306 views

Should the cost function be zero using TensorFlow's sigmoid_cross_entropy_with_logits?

I'm building a CNN to make a binary classification (1 or zero). For this I'm using the cost function sigmoid_cross_entropy_with_logits. But for some reason the ...
3
votes
1answer
2k views

What is the Time Complexity of Linear Regression?

I am working with linear regression and I would like to know the Time complexity in big-O notation. The cost function of linear regression without an optimisation algorithm (such as Gradient descent) ...
1
vote
2answers
53 views

Why do we double the number in a quadratic cost function or MSE?

$$ C(w,b) = \frac{1}{2n}\sum_{x}||y(x)-a||^2 $$ Where y is a 10-dimensional vector, a is the output, w is the weight and b is the bias and n is the number of inputs. If this is the MSE, shouldn't it ...
4
votes
2answers
852 views

What are the cases where it is fine to initialize all weights to zero

I've taken a few online courses in machine learning, and in general, the advice has been to choose random weights for a neural network to ensure that your neurons don't all learn the same thing, ...
2
votes
0answers
294 views

Cost/loss functions for multi-tasking regression neural networks

The mean square loss function is the standard for regression neural networks. However, if I have a neural network learning two tasks (two outputs) at once, is it more advisable to train on the sum of ...
1
vote
1answer
336 views

asymmetric cost function for deep neural network binary classifier

I am building a deep neural network based binary classifier, with single output. The loss function I actually want to minimize is $$ \mathcal L(\hat y,y) = \begin{cases} 0, & \text{if $\hat y$ = ...
2
votes
2answers
531 views

logistic regression algorithm fails to work

I'm trying to code my own logistic regression algorithm using Andrew NG's machine learning using Octave. lectures. So what I did was make a csv file, the first row being some parameter and the second ...
0
votes
1answer
168 views

Derivation of the cross-entropy equation in Michael Nielsen's book

I am reading the book http://neuralnetworksanddeeplearning.com/chap3.html by Michael Nielsen. So this is a question mostly for the people familiar with the book and understanding the material. In the ...
1
vote
1answer
668 views

Gradient Descent in logistic regression

Logistic and Linear Regression have different cost functions. But I don't get how the gradient descent in logistic regression is the same as Linear Regression. We get the Gradient Descent formula by ...
2
votes
2answers
961 views

Softmax classifier never allows for 100% probability in LSTM?

When working with LSTM I am using a softmax classifier and a one-hot encoded vector approach. The softmax looks like this: $$S(h_i) = \frac{e^{h_i}}{\sum e^{h_{total}}}$$ notice, LSTM's result is a $...
0
votes
1answer
118 views

Custom c++ LSTM slows down at 0.36 cost is usual?

Am I missing a part of the puzzle? I have implemented LSTM in c++ which steadily decreases in error, but slows down at the certain error value. It also seems to predict most of the characters, but ...
1
vote
0answers
92 views

How to decrease the learning rate only when cost function is stagnant?

I'm training a very complex function in tensorflow. Is there a way to decrease the learning rate only when the cost isin't decreasing very much?
1
vote
1answer
574 views

Logistic Regression : Solving the cross-entropy cost function analytically

Logistic regression cost function is cross-entropy. It is defined as below: This is a convex function. To reach the minimum, scikit-learn provides multiple types of solvers such as : ‘liblinear’ ...
7
votes
1answer
1k views

Cost function for Ordinal Regression using neural networks

What is the best cost function to train a neural network to perform ordinal regression, i.e. to predict a result whose value exists on an arbitrary scale where only the relative ordering between ...
1
vote
1answer
931 views

Understanding Logistic Regression Cost function

Linear Regression cost function: $$J(\theta) = \frac{1}{2 m} \sum_{i=1}^m (h_{\theta}(x^{(i)}) - y^{(i)})^2$$ where: $$h_{\theta}(x) = \theta_0 + \theta_1 x_1$$ Logistic Regression cost function $...
15
votes
2answers
16k views

Python implementation of cost function in logistic regression: why dot multiplication in one expression but element-wise multiplication in another

I have a very basic question which relates to Python, numpy and multiplication of matrices in the setting of logistic regression. First, let me apologise for not using math notation. I am confused ...
1
vote
1answer
713 views

Is cross-entropy a good cost function if I'm interested in the probabilities of a sample belonging to a certain class?

I'm training a neural network that, for each of six classes, tries to predict the probability that a sample belongs to it. After that, I want to use these probabilities as fractions of the sample ...
0
votes
1answer
89 views

Any Neural Network implementations that allow for a cost function of more than just the network output?

I have an application of a straightforward MLP, for which the cost function is a function of both the network output, in addition to another value calculated from the network weights (actually the ...
1
vote
2answers
5k views

XGBoost change loss function

I'm using XGBoost (through the sklearn API) and I'm trying to do a binary classification. False Positives are much worse for me than False Negatives, how can I take this into account? The API ...
2
votes
2answers
2k views

Does MLP always find local minimum

In linear regression we use the following cost function which is a convex function: We Use the following cost function in ...
0
votes
1answer
759 views

Cost functions penalizing certain types of misclassification more heavily in tensorflow

I have 2 classes of data A and C. I want to create a NN with 3 output nodes, classifying data as either A, B, or C. Classifying an A or C correctly should have zero cost (very good). Classifying ...
0
votes
1answer
115 views

Computing weights in batch gradient descent

I have a reasonably large set of images that I want to classify using a neural network. I can't fil them all into memory at once, so I decided to process them in batches of 200. I'm using an cross-...
1
vote
0answers
185 views

Altered priors for classification trees

I am using the rpart function (in the "rpart" package) in R to fit a cost-sensitive classification tree. I have read the corresponding vignette, which says that misclassication costs are incorporated ...
1
vote
1answer
1k views

Cost of greater than 1, is there an error?

I'm computing cost in the following way: cross_entropy = tf.nn.softmax_cross_entropy_with_logits(y, y_) cost = tf.reduce_mean(cross_entropy); For the first ...