Questions tagged [loss-function]

A function used to quantify the difference between observed data and predicted values according to a model. Minimization of loss functions is a way to estimate the parameters of the model.

Filter by
Sorted by
Tagged with
8
votes
3answers
27k views

Keras Sequential model returns loss 'nan'

I'm implementing a neural network with Keras, but the Sequential model returns nan as loss value. I have sigmoid activation ...
3
votes
1answer
409 views

Why does putting a 1/2 in front of the squared error make the math easier?

Per wiki, the mean squared error (MSE) looks like: $$ \operatorname {MSE} ={\frac {1}{m}}\sum _{i=1}^{m}(y_{i}-{\hat y_{i}})^{2} $$ The professor added a $1\over2$ in front of the formula and ...
104
votes
5answers
63k views

Why do cost functions use the square error?

I'm just getting started with some machine learning, and until now I have been dealing with linear regression over one variable. I have learnt that there is a hypothesis, which is: $h_\theta(x)=\...
2
votes
1answer
2k views

Intuitive explanation of Lovasz Softmax loss for Image Segmentation problems

Lovasz Softmax is used a lot these days for segmentation problem and the original paper is really bad at explaining why it works.
9
votes
3answers
3k views

Why is there a $2$ at the denominator of the mean squared error function?

In the famous Deep Learning Book, in chapter 1, equation 6, the Quadratic Cost (or Mean Squared Error) in a neural network is defined as $ C(w, b) = \frac{1}{2n}\sum_{x}||y(x)-a||^2 $ where $w$ is ...
2
votes
2answers
3k views

Understanding Training and Test Loss Plots

I have attached a figure that contains 6 subplots below. Each shows training and test loss over multiple epochs. Just by looking at each graph, how can I see which one is the best? Which ones are ...
6
votes
2answers
8k views

Loss being outputed as nan in keras RNN

Since the first Epoch of the RNN, the loss value is being outputted as nan. Epoch 1/100 9787/9787 [==============================] - 22s 2ms/step - loss: nan I have normalized the data. ...
2
votes
2answers
4k views

loss/val_loss are decreasing but accuracies are the same in LSTM!

I am trying to train a LSTM model, but the problem is that the loss and val_loss are decreasing from 12 and 5 to less than 0.01, but the training set ...
38
votes
5answers
33k views

Intuitive explanation of Noise Contrastive Estimation (NCE) loss?

I read about NCE (a form of candidate sampling) from these two sources: Tensorflow writeup Original Paper Can someone help me with the following: A simple explanation of how NCE works (I found the ...
17
votes
2answers
35k views

Custom loss function with additional parameter in Keras

I'm looking for a way to create a loss function that looks like this: The function should then maximize for the reward. Is this possible to achieve in Keras? Any suggestions how this can be achieved ...
6
votes
1answer
5k views

What's the difference between Error, Risk and Loss?

When we talk about 'Minimizing Loss', we often talk about loss functions such as Mean Squared Error (MSE); the term 'Empirical Risk Minimization' is often used interchangeably. So what's the ...
3
votes
2answers
9k views

XGBoost change loss function

I'm using XGBoost (through the sklearn API) and I'm trying to do a binary classification. False Positives are much worse for me than False Negatives, how can I take this into account? The API ...
2
votes
2answers
7k views

what is the difference between euclidean distance and RMSE?

I'm searching for a loss function that fits my Project. Actually I have two question but they are in the same direction. I take a look at the definition of the root mean squared error and the ...
1
vote
1answer
31 views

What quantile is used for the initial DummyRegressor for Gradient Boosting Regressor in scikit-learn?

According to the documentation of Scikit-Learn Gradient Boosting Regressor: init: estimator or ‘zero’, default=None: An estimator object that is used to compute the initial predictions. init has to ...
1
vote
1answer
54 views

Tree complexity and gamma parameter in xgboost

According to xgboost paper, regularization is given by: $$\Omega(f) = \gamma T + \lambda || w||^2$$ where $\gamma$ is the complexity of a tree (i.e., number of leaves in the tree). The parameter ...
5
votes
0answers
664 views

XGBoost custom objective for regression in R

I implemented a custom objective and metric for a xgboost regression. In order to see if I'm doing this correctly, I started with a quadratic loss. The ...
3
votes
1answer
2k views

Activation method and Loss function for multilabel multiclass classification

I am using CNN for Sentence Classification code by Yoonkim. This is used for text classification. I noticed that he uses softmax layer and negative log likelihood error. This is optimal for single ...
2
votes
2answers
49 views

Boosted tree regression loss function when data has occasionally very large values to predict?

I have a regression problem where most of my target variables are down in the range 5-30, but occasionally the target variable will spike up to 100, 500, or even 5000. These values are not spurious ...
2
votes
3answers
88 views

Reducing Regression to Classification

If a regression problem is reduced to classification, does minimizing the classification loss translate to minimizing regression error and hence better regression performance?
2
votes
3answers
223 views

Neural Network Loss Function - Mean Square Error: questions about what 'n' signifies

I'm very new to neural networks and have recently learnt about the loss functions used with neural networks. This question is in regards to the mean square error metric, defined as (from the textbook ...
1
vote
0answers
32 views

Which activation function of the output layer and which loss function are advised to be used for bounded regression?

I want my (deep) neural network to produce an output from a certain range, in my case between 0 and 255. I have scaled the labels from [0..255] to [0..1]. For the neural network, I have tried a ...
1
vote
1answer
279 views

Implement the following loss function without interrupting the gradient chain registered by the gradient tape

I have spent five days trying to implement the following algorithm as a loss function to use it in my neural network, but it has been impossible for me. Impossible because, when I have finally ...