Stack Exchange Network

Stack Exchange network consists of 175 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [loss-function]

The tag has no usage guidance.

1
vote
1answer
26 views

Meaning of this notion in 0-1 loss?

I am reading a paper and encountered this notion: $$1_{\{Y=1\}}$$ To me it seems to be the expression as below, but I am not entirely sure and I don't think the author explictly explained it: <...
0
votes
0answers
14 views

How to pass X_test to custom loss function Keras [on hold]

I need to pass also X_test data to custom loss like this: def customLoss(y_true, y_pred, X_test): some code return loss How can I do that? Is it possible ...
1
vote
1answer
20 views

Why increasing the number of units or layers does not increase the accuracy and decrease the loss?

I have an LSTM neural network; when I increase the number of units, layers, epochs or add dropout, it seems it has no effect and still I have persistent errors and accuracies like the following: ...
0
votes
0answers
44 views

Model loss and validation loss not decreasing? How to speed?

Every image was resized to 256x256 pixels. Batch size = 4. (GPU GTX 1050 memory ~4GB). The mask R-CNN model was initial-ized using pretrained weights from COCO dataset TRAIN_ROIS_PER_IMAGE = 8 ...
2
votes
1answer
30 views

MSE loss different in Keras and PyToch

My problem is that in PyTorch I cannot reproduce the MSE loss that I have achieved in Keras. I have trained the following model in Keras: ...
0
votes
1answer
21 views

Loss is decreasing but val_loss not! [duplicate]

If loss is decreasing but val_loss not, what is the problem and how can I fix it? I get such vague result:
0
votes
2answers
20 views

What is purpose of partial derivatives in loss calculation (linear regression)?

I am studying ML and data science stuff from scratch. As a part of the course, I am studying how the models are derived. And for most of them, starting with the simplest - linear regression, we take ...
3
votes
1answer
61 views

How to utilize user feedback due to miss-classification when correct class label is unknown?

Suppose we are developing an app which is supposed to predict a dog's breed by it's picture. We trained a classifier (in my case an MLP) using some dataset and shipped the app to users. Now suppose ...
1
vote
1answer
23 views

Why do people use CrossEntropyLoss and not just a softmax probability as the loss?

I don't understand why one would add additional complexity to log, probabilities for the loss function of a classification Neural Network. What benefit does that have, as opposed to just using the 0-1....
1
vote
1answer
13 views

What does it mean: “Everything looks OK but loss won't decreases!”

I have written a LSTM network. It seems all the things are OK but when I train the network, I get the same loss amount about 4.9e-4 for every iterations! What is the problem? Why my network can't ...
0
votes
1answer
15 views

Difference between “reducing batch_size” and “increasing epochs” to decrease loss amount?

In my experience, both reducing batch_size and increasing epochs can decrease loss amount. But I like to know is there any ...
1
vote
1answer
39 views

Is there any standard or normal range for the amount of LSTM loss function?

I am working on a LSTM network that I get loss amounts around 4.7 e-4 . It seems adding more layers and increasing epochs don't help to decreasing it. I also using a ...
2
votes
0answers
16 views

policy gradient loss [closed]

I am confused with the process for calculating loss. My code is below: ...
1
vote
0answers
11 views

Performance diagnostics in mxnet gluon (e.g. plotting training vs validation loss over time)?

Tensorflow has tensorboard, is there any recommended way to plot classification error/loss over time in mxnet?
1
vote
1answer
34 views

Geting batch size in keras custom loss

I am implementing a custom loss in keras, for example, a sum: def custom_loss(y_true, y_pred): K.sum(y_true, y_pred) Now, I want to normalize it by the batch ...
0
votes
0answers
12 views

cost/loss function being a multi-well function in neural networks

In a 0 or 1 binary classification problem using neural networks, given the activation function is taken as sigmoid, if one takes the cost/loss function as sum of square of differences, the loss ...
1
vote
0answers
36 views

Contrastive loss problem in a character-level, siamese NN model

Summary: my NN with contrastive loss does not work, need help debugging Background: I am trying to replicate this paper At first, I used binary crossentropy for loss, and the results were very good,...
2
votes
1answer
49 views

Loss Function for Probability Regression

I'm trying to predict a probability with a neural network, but having trouble figuring out which loss function is best. Cross entropy was my first thought, but other resources always talk about it in ...
1
vote
1answer
20 views

Meaning of subscript in min max value function

This possibly is a very stupid question, but i have not been able to find the answer on the internet and have got no clue which keywords to use while searching. What's the meaning of $\mathbb{E}_{x \...
0
votes
0answers
17 views

Implementing WARP Loss in Tensorflow

I notice there are attempts to implement WARP loss in Keras such as (https://stackoverflow.com/questions/46299554/implimentation-of-warp-loss-in-keras) But I have not seen any githubs or publications ...
0
votes
0answers
24 views

In generative adversarial models (GANs), why should we solve min-max problem and not max-min?

I know that in GANs model, there is min-max game between generator and discriminator which discriminator tries to maximize the loss function and the goal of generator is to minimize it. But why we ...
0
votes
0answers
6 views

Connectionist Temporal Classification Loss for Astroturfing Detection

I'm trying to detect astroturfing in social networks through post timestamp patterns. That is, if it's the same person posting across several different accounts, then these accounts are expected to ...
0
votes
2answers
43 views

how to consider some miss classifications “half correct” using as base categorical_crossentropy - for a trading system

I have a trading system where the model receives 9 time-series and predict : A - strong down B - week down C - neutral D - week up E - strong up (these classes ...
1
vote
1answer
32 views

Why Huber loss has its form?

Huber loss formula is $\hspace{3.0cm} L_\delta(a) = \begin{cases} \frac{1}{2} a^2 && |a| \leq \delta \\ \delta (|a| - \frac{1}{2} \delta) && |a| > \delta\end{cases}$ where $a = y - ...
0
votes
0answers
25 views

Continuous Function Input

If a continuous function has trouble differentiating between small close numbers (i.e. 0.1 and 0.001) why can we train and learn such small numbers in something like word2vec?
0
votes
0answers
31 views

Is empirical risk the same thing as loss function?

I am reading the article Stochastic Gradient Descent Tricks by Léon Bottou (avaible here) and on the very first page they introduce empirical risk $E_n(f) = \frac{1}{n} \sum_{i=1}^{n} l(f(x_i),y_i),$ ...
1
vote
1answer
33 views

Which Loss function is correct for binary mapping?

I have built a 3 layer neural network to perform a binary mapping (2016 inputs, 288 outputs.) I am getting decent results with mean square error and stochastic gradient decent. My question is: Is ...
1
vote
0answers
16 views

Strangeness in validation loss between CPU vs GPU when training CNN

I've been training an implementation of Mask R-CNN and it was training very successfully on my CPU but I've just set up my GPU and it is giving some strange results when looking at my validation loss. ...
0
votes
0answers
34 views

Backpropagation through LSTM and MLP layers

For didactic reason, I am currently implementing in numpy an LSTM network for classifications. I need to add on top of the LSTM another fully connected layer, because I don't want the output to have ...
-1
votes
0answers
53 views

Triplet loss training problem

My results are very poor and I cannot make out the reason on why is it so? I am using euclidean distance measure for hard mining of triplets. It is prior to training with the initial random set of ...
1
vote
0answers
33 views

Unbalanced multi-label multi-class classification

What are common approaches in order to deal with unbalanced multi-label multi-class classification problems in deep learning? Furthermore there is correlation between the labels. I tried two ...
0
votes
1answer
68 views

Classification loss function: how to implement individual weights for each observation and class

The problem I have to solve is a classification problem. The costs of a misclassification are very different (but known) for the various observations, so I plan to include them by assigning weights to ...
-1
votes
0answers
15 views

Multi-Task Loss: Bischke, et al

I'm trying to understand the multi-task loss function from paper Bischke, et al. Specifically, I'm stumped at Equation 8 and how this is actually computed given ...
1
vote
1answer
393 views

custom loss in keras, problem with batch size

I am trying to create a custom loss function,custom_loss(y_true, y_pred). I understand that y_pred is calculated by my model but ...
0
votes
1answer
23 views

Enable to reproduce the loss of training while predicting

i use CNN model for a regression problem with a custom loss ...
1
vote
1answer
104 views

What is non-decomposable and/or non-differentiable loss function?

I have been reading some deep learning literature and came up with these concepts of non-decomposable and non-differentiable loss functions. My question is are these same thing? if not how are they ...
3
votes
1answer
969 views

Sparse_categorical_crossentropy vs categorical_crossentropy (keras, accuracy)

Which is better for accuracy or are they the same? Of course, if you use categorical_crossentropy you use one hot encoding, and if you use sparse_categorical_crossentropy you encode as normal integers....
1
vote
1answer
32 views

Loss is bad, but accuracy increases?

I have a multicategorial classification problem for images. There are 5 (imbalanced) classes for which i use different class weights. In general there are only a few training images per class: ~56-238 ...
2
votes
0answers
169 views

Custom Loss Function on a Keras Neural Network

I'm training a Neural Network on Keras to predict class as a triplet of the form S,P,T, where S, ...
0
votes
1answer
61 views

What loss function avoids overconfidence?

In the case of a neural net with a relatively small training data set, doing simple classification with categorical cross entropy (log loss), it is very easy for the results of the network to be "...
4
votes
1answer
60 views

issue with early-stopping on f1 score with imbalanced data

I have a highly imbalanced dataset with less than 0.5% of the minor class. Using Keras, I'm training DNN on the training set and evaluate performance on validation set. Loss function is ...
1
vote
1answer
124 views

Using SMAPE as a loss function for an LSTM

I am currently working on a time series forecasting problem and am looking into using an LSTM. My final accuracy metric that I use to determine whether or not the forecast is good or not is defined ...
0
votes
1answer
190 views

Should the minimum value of a cost (loss) function be equal to zero?

We know optimization techniques search in the space of all the possible parameters for a parameter set that minimizes the cost function of the model. The most well-known loss functions, like MSE or ...
0
votes
1answer
621 views

What is the advantage of using log softmax instead of softmax

i am wondering if there are any advantages of log softmax over softmax. And also, when i should use softmax or log-softmax. is there any specific reason for choosing one over another?
0
votes
3answers
291 views

What Base Should Be Used For Negative Log Likelihood?

When calculating the negative log likelihood loss, what base of log are we supposed to use?
2
votes
1answer
25 views

“help” decision tree by tying 2 features together

Assuming I have in my dataset 2 (or more) features that are for sure linked (for example: feature B indicates the amount of relevance of feature A), is there a way I could design a decision tree that ...
1
vote
0answers
23 views

Should I rescale losses before combining them for multitask learning?

I have a multitask network taking one input and trying to achieve two tasks (with several shared layers, and then separate layers). One task is multiclass classification using the CrossEntropy loss, ...
1
vote
0answers
39 views

Tuning neural network loss function for space physics [closed]

I am trying to work on neural network classification (with python, Keras) for space physics purposes, where I want to identify specific planetary regions based on multivariate temporal data and multi ...
0
votes
1answer
39 views

Validation loss differs on GPU vs CPU

I am consistently seeing higher validation loss when I train & evaluate a model on AWS GPU vs local CPU. I am using the exact same train/eval datasets and the exact same Tensorflow code and ...
0
votes
1answer
873 views

Validation loss is lower than the training loss

I am using autoencoder for anomaly detection in warranty data. Architecture 1: The plot shows the training vs validation loss based on Architecture 1. As we see in the plot, validation loss is ...