# Questions tagged [cost-function]

The tag has no usage guidance.

59 questions
Filter by
Sorted by
Tagged with
8 views

### Simplifying Log Likelihood equation

I am reading through a paper (https://www.mitpressjournals.org/doi/pdf/10.1162/0891201053630273) where they describe logloss as a ranking function and can be simplified to the margin of the training ...
23 views

### Chain function in backpropagation

I'm reading a Neural Networks Tutorial. In order to answer my question you might have to take a brief look at it. I understand everything until something they declare as "chain function": It's ...
11 views

### Norm type in cost function of ANN

I'm reading a tutorial about ANN. They use the following cost function: As you can see this equation includes a norm. I'm new to the concept of norm. My question is what kind of norm they use here (...
464 views

### Should the cost function be zero using TensorFlow's sigmoid_cross_entropy_with_logits?

I'm building a CNN to make a binary classification (1 or zero). For this, I'm using the cost function sigmoid_cross_entropy_with_logits. But for some reason, the ...
28 views

### Regularization for intercept parameter

Why is the regularization parameter not applied to the intercept parameter? From what I have read about the cost functions for Linear and Logistic regression, the regularization parameter (λ) is ...
7 views

### How to design a two-factor optimization score function?

What if I have two parameters as optimization target and want to maximize one and minimize the other one? Example: food products dataset, look for strawberries and artichokes in USDA FDC. Task: find ...
30 views

### Reason behind the sum of rate factors for calculating cost function derivative

Suppose we have a network of neurons like below: We make a little change in weight w[l][j][k] on our network, and it can make change on our cost function from ...
44 views

### Reason for capping Learning Rate (alpha) up to 1 for Gradient Descent

I am learning to implement Gradient Descent algorithm in Python and came across the problem of selecting the right learning rate. I have learned that learning rates are usually selected up to 1 (...
92 views

### How to Define a Cost Fucntion?

I want to define a cost function in python to identify optimum value in days when i should end a marketing campaign to save spend on campaigns not generating traffic good traffic. Problem is I dont ...
5k views

### How does Gradient Descent and Backpropagation work together?

Please forgive me as I am new to this. I have attached a diagram trying to model my understanding of neural network and Back-propagation? From videos on Coursera and resources online I formed the ...
45 views

### How to get a rebalance strategy with a cost matrix?

In the case of a classification problem where a cost matrix is used to maximize the model performance, it is common to do a rebalance technique. Let's say for example that I have the following costs ...
162 views

### My LSTM can't reduce error down to zero, when overfitting

I have implemented LSTM in c++ which steadily decreases in error, but slows down at the certain error value. It also seems to predict most of the characters, but gets stuck and not able to correct ...
27 views

### Andrew Ngs Class - Why Did He Change up the Cost Function?

I am taking Andrew Ng's Machine Learning Intro class. Looks like he changed the cost function without any explanation in the second week. Specifically: He no longer squares each deviation between the ...
24k views

### Python implementation of cost function in logistic regression: why dot multiplication in one expression but element-wise multiplication in another

I have a very basic question which relates to Python, numpy and multiplication of matrices in the setting of logistic regression. First, let me apologise for not using math notation. I am confused ...
17 views

### Theano gradient descent and cost function issue

Sorry but can anyone point out whats wrong with this code? ...
63 views

I am not a math expert but have a basic understanding of linear algebra, calculus and probability and I understand the math behind back propagation. Currently I am trying to learn about policy ...
205 views

### Policy gradient vs cost function

I was working with continuous system RL and obviously stumbled across this Policy Gradient. I want to know is this something like cost function for RL? It kinda gives that impression considering we ...
38 views

### Cost function - ideas

I build xgboost model for regression problem. By the default xgboost optimize $(y - y_{pred})^2$, so the RMSE will be the best eval metric to measure performance. But my task is to build the best ...
58 views

### Neural network cost is constant never changing during training

I am trying to build a binary classifier to predict a pulsar star with Single Hidden layer Neural Network. But the cost on training dataset after almost 100 iterations has no change, following is the ...
49 views

### Question of using gradient descent instead of calculus. I checked previous questions there are still points to clarify

First of all I checked http://stats.stackexchange.com/questions/23128/solving-for-regression-parameters-in-closed-form-vs-gradient-descent, http://stackoverflow.com/questions/26804656/why-do-we-use-...
96 views

### Minimum cost is not zero when calculating cross-entropy on soft labels

I am training a neural network using batches of soft labels, e.g. y = [[0.00, 0.25, 0.25, 0.50], ... [0.75, 0.00, 0.20, 0.05]] However, as ...
34 views

Problem I am now taking Andrew Ng's deep learning course on Coursera. Everything is great but when it comes to RNN, I sometimes feel confused. Here is a question about RNN (or more specifically, the ...
78 views

897 views

### logistic regression algorithm fails to work

I'm trying to code my own logistic regression algorithm using Andrew NG's machine learning using Octave. lectures. So what I did was make a csv file, the first row being some parameter and the second ...
202 views

### Derivation of the cross-entropy equation in Michael Nielsen's book

I am reading the book http://neuralnetworksanddeeplearning.com/chap3.html by Michael Nielsen. So this is a question mostly for the people familiar with the book and understanding the material. In the ...
1k views

### Gradient Descent in logistic regression

Logistic and Linear Regression have different cost functions. But I don't get how the gradient descent in logistic regression is the same as Linear Regression. We get the Gradient Descent formula by ...
704 views

### Logistic Regression : Solving the cross-entropy cost function analytically

Logistic regression cost function is cross-entropy. It is defined as below: This is a convex function. To reach the minimum, scikit-learn provides multiple types of solvers such as : ‘liblinear’ ...
2k views

### Softmax classifier never allows for 100% probability in LSTM?

When working with LSTM I am using a softmax classifier and a one-hot encoded vector approach. The softmax looks like this: $$S(h_i) = \frac{e^{h_i}}{\sum e^{h_{total}}}$$ notice, LSTM's result is a \$...