Questions tagged [loss-function]
A function used to quantify the difference between observed data and predicted values according to a model. Minimization of loss functions is a way to estimate the parameters of the model.
351
questions
0
votes
1answer
16 views
Multiclassification with large number of labels
I am attempting to build a classifier with a large input space of one hot encoded vectors. The output should be a vector of labels, with 10000 possible labels each. For example, the labels could ...
2
votes
0answers
12 views
CNN gradients with different magnitude
I have a CNN architecture with two cross entropy losses $\mathcal{L}_1$ and $\mathcal{L}_2$ summed in the total loss $\mathcal{L} = \mathcal{L}_1 + \mathcal{L}_2$. The task I want to solve is ...
1
vote
2answers
18 views
SparseCategoricalCrosstentropy vs sparse_categorical_crossentropy
What is the difference between SparseCategoricalCrosstentropy and sparse_categorical_crossentropy ?
SparseCategoricalCrossentropy - Computes the crossentropy loss between the labels and predictions.
...
0
votes
0answers
15 views
Example of Reverse KL Divergence in Supervised Learning
It is of common knowledge that Supervised Learning uses forward KL divergence. However, I would like to use Reverse KL Divergence and am looking for examples of similar usage in literature. Most ...
0
votes
0answers
15 views
DICE loss too low but no overlap between prediction and label
I am trying to achieve the segmentation of the bone on the cross sectional area of MRI images with the Unet I found here. The label is a binary png image which I intend to compare to my prediction. ...
0
votes
1answer
21 views
Log-loss - but only if too far away from target?
I'm training a neural net on a "needle in haystack" problem - there is a huge amount of data, also with a huge class imbalance (~1 positive per 10k negatives). If I train the net on ...
1
vote
2answers
25 views
Can points not being passed to the loss function influence it?
If I were to construct a model where only a subset of the training data is passed to the loss function, can the other parts of the dataset influence the fit?
More concretely, in this first example:
<...
0
votes
0answers
12 views
proper activation function at output and loss function to optimize for OCR?
I am trying to make a CNN model on IAM handwritten words data(which has images of words handwritten by multiple people and targets are text in the images). So, I can encode words to numbers(A=0, B=1 ...
0
votes
1answer
17 views
regression network that will be optimised on a subset of the data
I am trying to optimise my network that is trying to perform regression. Currently, the dataset is dominated by values in a certain range. It is very good at predicting these values. I want to try to ...
1
vote
2answers
25 views
Lower loss always better for Probabilistic loss functions?
I am working on an neural net int Tensorflow that predicts percentages for win, draw, loss for given data of a game. The labels I provide are always {1, 0, 0}, {0, 1, 0} or {0, 0, 1}. After some ...
1
vote
0answers
26 views
Why the validation loss is 0 even if accuracy not 100%?
I am training a CNN to perform a binary classification and at some point I get validation loss 0, however the validation accuracy is 84.5%. Why does this happen? I use binary_crossentropy as loss ...
0
votes
0answers
18 views
How to interpret curve of regularization loss during CNN training?
I am fine-tuning a single shot detector (SSD) in tensorflow object detection api.
I didn't freeze the backbone (mobilenet), I programmed the learning rate to go from e-3 to e-4 to e-5.
In the paper ...
0
votes
0answers
29 views
Loss function bounces back up
I'm training a very simple LSTM in PyTorch. It's a single layer, and I'm using it for multi-label classification with a BCEWithLogitsLoss. My batches are shuffled,...
0
votes
1answer
44 views
Why is my loss blowing up after adding regularization
I tried to add L2 regularization to a network class I wrote however when I train it the loss blows up even though accuracy also increases. Can someone explain where I am going wrong? (I am using the ...
0
votes
2answers
24 views
Understanding Loss function and Learning Algorithm
In Keras, when specifying a loss such as Mean absolute error, does it replace the cost function in the learning algorithm (Adam or SGD) with the mean absolute error? I'm new to ML and a bit confused ...
1
vote
0answers
41 views
how to implement squared hinge loss in pytorch
does anyone have any advice on how to implement this loss in order to use it with a convolutional neural network?
Also, how should I encode the labels of my training data? We were using one hot ...
1
vote
1answer
31 views
Why does categorical_crossentropy work in when input is not 1-hot encoded?
I'm going through lessons on the REINFORCE algorithm to solve Cartpole/Pong/etc (using AIGym) and every one uses categorical_crossentropy as the loss function.
What's confusing me is that ...
2
votes
2answers
23 views
Boosted tree regression loss function when data has occasionally very large values to predict?
I have a regression problem where most of my target variables are down in the range 5-30, but occasionally the target variable will spike up to 100, 500, or even 5000. These values are not spurious ...
1
vote
1answer
32 views
Implement the following loss function without interrupting the gradient chain registered by the gradient tape
I have spent five days trying to implement the following algorithm as a loss function to use it in my neural network, but it has been impossible for me. Impossible because, when I have finally ...
1
vote
1answer
17 views
Interpreting Loss in neural network: Neural network train loss gradually tappers and validation loss never reaches a minima
Unable to improve the network validation loss. Is it overfitting/underfitting. How can I get a better validation loss?.The code is below
...
2
votes
1answer
29 views
Weighted loss functions vs weighted sampling?
For image classification tasks, is there a practical difference between using weighted loss functions vs. using weighted sampling? (I would appreciate theoretical arguments, experience or published ...
2
votes
2answers
29 views
Which loss function is the best loss function when using XGB regression with highly skewed dataset?
Which loss function is the best loss function when using XGB regression with a highly skewed dataset?
The skewness of the data is very high. I used XGBoost with objective function of linear ...
1
vote
2answers
23 views
Appropriate loss function for multi-hot output vectors
I have some data in which model inputs and outputs (which are the same size) belong to multiple classes concurrently. A single input or output is a vector of zeros somewhere between one and four ...
0
votes
2answers
23 views
What are some good loss functions used to minimize extreme errors in regression and time series forecasting?
E.g. In detriment of a smaller mean error, I want to have fewer big mistakes
I'm working on a time series forecasting task and in some specific cases I don't need perfect accuracy, but the network ...
0
votes
1answer
91 views
How do we implement a custom loss that backpropagates with PyTorch?
In a neural network code written in PyTorch, we have defined and used this custom loss, that should replicate the behavior of the Cross Entropy loss:
...
0
votes
1answer
29 views
Using a part of a trained model in a custom loss function -Tensorflow
I want to write a custom loss function that uses the intermediate result of a trained discriminator.
the loss function compares images.
the loss function is for recovering the latent vector of an ...
0
votes
1answer
13 views
Is there a loss function that measures the cross similarity between two 2D tensors?
Given two input tensors x1 and x2 with the shape [batch_size, hidden_size], let ...
0
votes
0answers
11 views
Derivative of Loss wrt bias term
I read this and I have an important doubt.
I try to understand well how to calculate the derivative of Loss wrt to bias.
In this question, we have this definition:
...
1
vote
0answers
10 views
Why N-pair Loss (NIPS 2016) stops minimizing in Image retrieval task?
Currently, I am using a Deep Learning model to build a search engine for retrieving images.
With a dataset of pairs of (image, description), I am using a ...
1
vote
2answers
12 views
How discriminator loss generated?
The images generated by generator has no labels, then how do Discriminator loss is generated on the basis of classification of generator generated images.
0
votes
1answer
17 views
When exactly should I use weighted loss?
Do I have to use it in any case when the class distribution is imbalance
(Train: class A:10%, B:90% and Test: class A:10%, B:90%)
or when it is different
(Train: class A:10%, B:90% and Test: class A:...
3
votes
1answer
91 views
VAE generates bad images. due to unbalanced loss functions?
I'm training a variational autoencoder on CelebA dataset using TensorFlow.keras
The problem I'm facing is that the generated images are not diverse enough and look kinda bad.
******(new) Example:****...
2
votes
1answer
28 views
How to implement my own loss function for Prototype learning using Keras Model
I'm trying to migrate this code, "Omniglot Character Set Classification Using Prototypical Network", into Tensorflow 2.1.0 and Keras 2.3.1.
My problem is about how to use euclidean distance between ...
0
votes
0answers
9 views
Simplifying Log Likelihood equation
I am reading through a paper (https://www.mitpressjournals.org/doi/pdf/10.1162/0891201053630273) where they describe logloss as a ranking function and can be simplified to the margin of the training ...
0
votes
0answers
11 views
Loss function for non-uniform distribution in pixel regression?
Goal: Given RGB images (x,y,3) and a grayscale heatmap (x,y,1), predict the heatmap using the RGB image as input to a neural network implemented inside Keras.
Approach: Multiply heatmap by (1./255) ...
0
votes
0answers
8 views
Question about a basic aspect of how text-2-speech spectrogram frames are aligned?
A key aspect of how text-to-speech (TTS) machine-learning works is very unclear to me even after reading the Tacotron-2 paper and the Google AI blog.
https://ai.googleblog.com/2017/12/tacotron-2-...
0
votes
2answers
18 views
Loss Function - Decreasing a lot at beginning of the epoch
I noticed something when looking at the output of the verbose. When I train my model, in the early part of the epoch (first 20 %), the loss is decreasing a lot. And then in the rest of the epoch (last ...
0
votes
0answers
18 views
Understanding the implementation of domain adaptation algorithm
I'm trying to implement domain adaptation using stochastic neighborhood embedding based on this article. I have different input shapes in target and source domain and using 2 parallel CNNs for ...
0
votes
2answers
36 views
Loss of NAN, Accuracy of 0 - Any idea why? Full code provided
I've been on this for the past few days and couldn't figure it out. Posted on various groups, StackOverflow etc and got suggestions from many users. I implemented these suggestions into the code shown ...
0
votes
0answers
11 views
What to do when there is more than one object in a cell in Yolo v1
I'm trying to implement Yolo v1 and I'm kinda stuck with its loss function.
I understand the need of multiple bounding boxes per cell although only one object is detected per cell. Each bounding box ...
0
votes
0answers
7 views
WGAN training tradeoffs
Currently training a WGAN with weight clipping and having reread the architecture and pitfalls mentioned in code, I am running it with 2 layers in critic and generator, no batch norm in generator and ...
1
vote
2answers
41 views
How to tell my neural network that I care much more about precision than recall?
I am training a neural network for a multilabel classification problem, so my last layer consist of n_classes sigmoid neurons.
Now, I know that it is impossible to ...
0
votes
0answers
10 views
Methods of working with unbalanced dataset
I've got problem where I need to classify the images for about 400 classes and to do this I'm using model with neural network. In my dataset (about 300k images) there are classes represented by about ...
-1
votes
1answer
379 views
What does from_logits=True do in SparseCategoricalcrossEntropy loss function?
In the documentation it has been mentioned that y_pred needs to be in the range of [-inf to inf] when from_logits=True. I truly ...
0
votes
0answers
15 views
Weighted RMSE in RNN for Multivariate Time Series: Deal with different Starting Dates and Mini-Batch
I am doing Multivariate Time Series prediction with Deep Learning. My metric is a Weighted RMSE (where each serie has its own weight), and I am trying to implement the WRMSE as my loss function for my ...
2
votes
1answer
36 views
Does small batch size improve the model?
I'm training an LSTM with Keras.
I've noticed that the smaller the batch size, the more the loss decreases during periods: so ...
1
vote
0answers
24 views
Good losses curves but bad predictions
I trained an LSTM with a Dense layer that has ReLU as an activation function.
Training and ...
0
votes
0answers
18 views
How to write masked loss function
I am voll new with tf and keras and I need to write a masked loss function.
y_true is [512] a sequence of integers representing ...
0
votes
1answer
32 views
Shouldn't training and validation loss be approximate before the first epoch is finished?
I'm having this burning question in my head, and I couldn't find the answer anywhere. During training, at least in Keras, the training loss is computed on the current batch, so the weights can be ...
0
votes
0answers
18 views
Keras validation loss fluctuates
I'm training an LSTM model created with Keras.
During epochs, I notice that the training loss decreases epoch after epoch, but ...
