Questions tagged [loss-function]

A function used to quantify the difference between observed data and predicted values according to a model. Minimization of loss functions is a way to estimate the parameters of the model.

Filter by
Sorted by
Tagged with
1
vote
1answer
21 views

How does keras (TF) know how to differentiate my custom loss?

Suppose I have this custom loss: ...
0
votes
0answers
11 views

How to optimize neural network with mutile losses

There is a multi-task problem that I try to solve with a single neutral network. In general it works fine, but it seems like there is a room for improvement. The final loss to optimize looks like ...
0
votes
1answer
20 views

The loss and accuracy of this LSTM both drop to nearly 0 at the same epoch

I'm trying to train an LSTM to predict the the Nth token using the N-1 tokens preceding it For each One-Hot encoded token, I ...
0
votes
0answers
10 views

Why is Dice loss neglecting random classes?

I implemented Dice loss for a semantic segmentation problem (with a severe class imbalance in my dataset) as follows: ...
2
votes
1answer
32 views

What is auxiliary loss in Character-level Transformer model?

I am reading Character-Level Language Modeling with Deeper Self-Attention from Rami Al-Rfou. In the second page, they had mentioned about Auxiliary Losses which can speed-up the model convergence and ...
0
votes
0answers
15 views

Combining multiple loss functions in the right way

I am trying to build an (variational) Autoencoder to generate fake but representative data from a generic data set with a couple of numeric and categorical columns. So far I have built functions that ...
0
votes
0answers
26 views

Plotting an loss/cost function wrt parameters

What would be the most sensible way to plot, for example the cross-entropy loss function (or the quadratic error function) for a binary classification algorithm (e.g. in this case logistic regression) ...
0
votes
0answers
4 views

Derive main prediction for zero-one loss function

Intuitively I can see why mode of predictions is the main prediction of a zero-one loss function, but mathematically I am not sure how it is derived? Main prediction $= argmin_{y'}E_D[L(\widehat y, y'...
0
votes
0answers
30 views

Cross entropy loss for sigmoid function

Suppose instead of squared error loss I take cross entropy loss: $$H(y) = - \sum y' \log(y) $$ ( where y' is the actual distribution) . I read somewhere that this loss function converges faster ...
0
votes
0answers
24 views

Interpreting Gensim Word2Vec Training Loss

I am using Gensim to build a Word2Vec model and identify the convergence of training loss, so that I can figure out the optimal number of iterations. For understanding this since gensim's ...
0
votes
1answer
32 views

Gradient Descent - how many values are calculated in loss function?

I'm a little bit confused how loss function is calculated in neural network training. There's is said that in theory when using Grid Search or Monte Carlo methods we can calculate all the possible ...
0
votes
1answer
16 views

Explanation behind the calculation of training loss in deep learning model

I am trying to model an image classification problem using convolution neural network. I came across a code on Github in which I am not able to understand the meaning of following line for loss ...
1
vote
1answer
20 views

Metric/loss for bin classification

I have a model that has to classify inputs into one of 45 categories but those categories actually represent bins (e.g. bins 1, 2 and 3 are between 1 and 10, 11 and 20, 21 and 30 respectively). What I ...
0
votes
0answers
16 views

Difference between binary cross_entropy and mse

model = Sequential() model.add(Dense(16, activation="selu")) model.add(Dense(1, activation="sigmoid")) I am using Keras to create a binary classifier. My data ...
0
votes
1answer
16 views

sklearn: sklearn.linear_model.HuberRegressor vs sklearn.linear_model.ElasticNet

I am experimenting different loss functions for my regression model. I noticed that in the sklearn, there are: sklearn.linear_model.HuberRegressor and sklearn.linear_model.ElasticNet To me, both use ...
0
votes
0answers
7 views

Nan in target variables Neural Network

Is it possible to train on a dataset with some nan in the target variables? I imagine a sort of loss calculation only for the given target data. Is this Doable in Tensorflow/Keras =?
1
vote
1answer
107 views

Why $L2$ loss is strictly convex if number of samples $N$ is larger than input dimension $d$?

I am using $L2$ loss in my linear regression problem and I have to prove that my $L2$ loss is strictly convex if number of samples $N$ is larger than input dimension $d$. I think, if I can prove ...
0
votes
1answer
43 views

Can we use Binary Cross Entropy for Multiclass Classification?

In this link, the author has implemented a CNN which classifies 15 classes and has used Binary Cross Entropy as the loss function. But since it's multiclass classification, is it valid to use Binary ...
0
votes
2answers
74 views

What can be the cause of a sudden explosion in the loss when training a CNN (Deeplab)

I am training the following deeplab CNN: https://github.com/tensorflow/models/tree/master/research/deeplab During training I see the following loss: The first 50k steps of the training the loss is ...
0
votes
0answers
14 views

Hierarchical prediction

Suppose the problem is the following: there is, say, binary target variable $x$, and real-valued target variable $y$, which is only relevant if $x = 1$. What is the best way to train a model to ...
0
votes
0answers
21 views

Issue creating custom Keras loss function

Hey I need some help constructing/using a custom cost function in Keras. What I have is N rows of sales data each with an expected cost, and an actual cost, and they can be different because of ...
1
vote
0answers
27 views

Disadvantages of Mean Squared Error? [closed]

I'm using mean squared error as reconstruction error for my autoencoder. The dataset is ECG (time series) and model is conv1d. I assumed MSE as the best option for reconstruction error, but it's ...
0
votes
0answers
10 views

Loss Function for Sparse Sequence Tagging

I'm currently working on trying to classify words in a sentence using bert. Unfortunately the labels I'm using are relatively sparse compared to the full text. Many sentences are only labelled 'O' ...
0
votes
0answers
112 views

TensorFlow: What is wrong with my (generalized) dice loss implementation?

I use TensorFlow 1.12 for semantic (image) segmentation based on materials. With a multinomial cross-entropy loss function, this yields okay-ish results, especially considering the sparse amount of ...
6
votes
2answers
101 views

How to know which loss function is suitable for image classification?

I am implementing a CNN model for image classification where I am learning about loss functions. There are several types of loss functions to determine error. However, how do we find out which ...
2
votes
0answers
14 views

Simmultainiously calculting loss from target interdependend metric

Is there a way to incorporate multiple targets into one loss? Currently, I work with the Sequential() API, I guess this won't be sufficient.... I work with area predictions as targets. Each sample ...
3
votes
0answers
16 views

Heavy regression loss for false non 0 prediction

My regression should predict values >=0 But a wrongly predicted value >0(e.g. 0.001 instead of 0) is much worse then a a slight missprediction of 0.001 (e.g. 0.002 instead of 0.003) I am thinking ...
1
vote
4answers
60 views

Neural net only works for small datasets

I have a neural network that attempts to solve some regression problem. The network works fine when the dataset has a small number of training examples, lets say 20. It overfits of course, but the ...
1
vote
0answers
14 views

How to backpropogate error from convolutional layer with respect to the input when using multiple channels

I have been attempting to implement a Convolutional Neural Network in python and have run into a bit of a roadblock. When backpropogating the error in a convolutional layer let us say that we receive ...
1
vote
0answers
10 views

How does the Backpropagation through time work?

I have to write a paper on LSTMs and I want to explain why LSTMs exist in the first place. According to some papers and books because usual RNNs had problems with vanishing gradients and the LSTM has ...
0
votes
1answer
41 views

Understanding REINFORCE loss

The loss used in REINFORCE algorithm is confusing me. From Pytorch documentation : loss = -m.log_prob(action) * reward We want to minimize this loss. If a ...
1
vote
0answers
81 views

Intuitive explanation of Lovasz Softmax loss for Image Segmentation problems

Lovasz Softmax is used a lot these days for segmentation problem and the original paper is really bad at explaining why it works.
0
votes
1answer
48 views

Why doesn't the binary classification log loss formula make it explicit that natural log is being used?

I'm completing a DataCamp course where we are introduced to the log loss formula for binary classification: Two scenarios are given to show how the formula is used. One with p=0.1 and one with p=0.5....
2
votes
1answer
29 views

How to train a neural network for high recall?

I would like to train a neural network for named entity recognition to tag an unlabeled dataset of texts. The generated labels will then be checked via a crowdsourcing platform. The goal is to ...
0
votes
0answers
17 views

Similarity of perceptron criterion and SVM

In the book "Neural Networks and Deep Learning" by Aggarwal there is an exercise 2.10.1: Consider the following loss function for training pair $(\overline{X},y)$: $$L=max(0, a -y(\overline{W} \...
1
vote
1answer
104 views

Poor performance of regression model for imbalanced data

I am trying to train a neural network model to solve a regression problem. The specificity of my dataset is that it has something like an exponential distribution of target values (imbalanced). ...
0
votes
0answers
42 views

Trainable parameter in a Keras loss function

all, I am trying to implement a loss function with trainable parameters in a Keras model. I managed it off keras.backend as follows: ...
0
votes
0answers
18 views

Rank by sorting the output probabilities of a binary classifier or by “learning to rank”?

I have a binary classification problem (5M points, only around 0.01% is positive). As output I want to give a list of data points sorted by "positiveness", and then only the top ...
0
votes
2answers
19 views

Is the Cross Entropy Loss important at all, because at Backpropagation only the Softmax probability and the one hot vector are relevant?

Is the Cross Entropy Loss (CEL) important at all, because at Backpropagation (BP) only the Softmax (SM) probability and the one hot vector are relevant? When applying BP, the derivative of CEL is the ...
0
votes
0answers
17 views

Understanding about RNN loss

Problem I am now taking Andrew Ng's deep learning course on Coursera. Everything is great but when it comes to RNN, I sometimes feel confused. Here is a question about RNN (or more specifically, the ...
1
vote
1answer
360 views

Pytorch doing a cross entropy loss when the predictions already have probabilities

So, normally categorical cross-entropy could be applied using a cross-entropy loss function in PyTorch or by combing a logsoftmax with the negative log likelyhood function such as follows: ...
1
vote
1answer
28 views

better confussion matrix higher LogLoss ? Is that possible>

I have tried a 2 different versions of a gbm in a multinomial classification problem. The second model results in better confusion matrix but in worse Log Loss value (at the test sample). How is that ...
0
votes
0answers
13 views

Does validation loss increase if the dataset is small?

This is my loss vs epoch image... You can see that my model converges too early. However, the frustating point is validation loss does not decrease accordingly compared to training loss. I am doing ...
0
votes
0answers
12 views

Loss function for multi-class classifiction where output variable is a level i.e the various classes are dependent on each other

Let's say we are classifying Images of cat , fish and human. Classifying a cat as human is as wrong as classifying it as fish, so here the normal loss functions/ metrics like Confusion matrix is fine. ...
1
vote
1answer
63 views

Tweedie Loss for Keras

We are currently using XGBoost model with Tweedie loss for solving a regression problem which works very good, now I wanted to move our model to Keras and experience with neural networks, do anybody ...
3
votes
1answer
208 views

How do I create a Keras custom loss function for a one-hot-encoded binary classifier?

I have a CNN binary classifier with one-hot-encoded labels that I've written using Keras and it's just not training to the metric I want to encourage. My data is very imbalanced (91% class 0, 9% class ...
1
vote
0answers
32 views

Huge cost not converging well with TensorFlow logistic regression

I try to use Logistic Regression for a dataset which contains 15 numeric features and 4238 rows of examples. The calculated cost started at 415.91, and converged when the cost was reduced to 220.119 ...
0
votes
0answers
118 views

Can Keras automatically calculate my custom loss gradient?

I'm using a custom loss function that I wrote completely in tensorflow. I am using Tensorflow 2.0 though so I am not sure that it is all part of one graph, I don't know how that works. Also, I use the ...
0
votes
0answers
111 views

Trying to get shannon entropy of image in tensorflow 2.0

I'm trying to incorporate the shannon entropy of an image in my custom loss function in tensorflow. I tried to just find a function that does that in tensorflow and I found tfp.distributions....
0
votes
0answers
213 views

Trying to build a custom loss function in Keras/Tensorflow but get an error

I am using tensorflow 2.0 and I built a loss function that has two inputs, y_pred and y_true. I made sure to use only tensorflow operations throughout the entire function (it's long) and to return a ...