Questions tagged [loss-function]
A function used to quantify the difference between observed data and predicted values according to a model. Minimization of loss functions is a way to estimate the parameters of the model.
437
questions
0
votes
0answers
9 views
Optimizing the Loss Function For Another Metric
Suppose I have a machine learning model which is used to improve the profitability of a business. One of the components of the model is a loss function, say for measuring the success of a ...
0
votes
0answers
13 views
Negative loss values for adaptive loss in tensorflow
I have used adaptive loss implementation on a neural network, however after training a model long enough, I am getting negative loss values. Any help/suggestion would be highly appreciated! Please let ...
0
votes
1answer
19 views
Can someone explain the solution to the following problem?
Q) We want to learn a function f(x) of the form f(x) = ax + b which is parameterized by
(a, b).
Using squared error as the loss function, which of the following parameters would you use to model this ...
1
vote
1answer
18 views
Loss drops to NaN after a short time for a time series classification
here is my model code for a binary classification of a time series:
...
1
vote
0answers
33 views
Which Neural Network or Gradient Boosting framework is the simplest for Custom Loss Functions?
I need to implement a custom loss function.
The function is relatively simple:
$$-\sum \limits_{i=1}^m [O_{1,i} \cdot y_i-1] \ \cdot \ \operatorname{ReLu}(O_{1,i} \cdot \hat{y_i} - 1)$$
With $O$ being ...
3
votes
0answers
25 views
How to train a neural network where computing the loss requires multiple object values?
I want to train a function that given metadata about an image produces hyper-parameters for an algorithm which operates on the image.
My understanding is (please forgive me I'm a novice here) a neural ...
0
votes
0answers
5 views
Problem in convergence of hebbian learning approach for Fuzzy Cognitive Map
I was trying to learn Fuzzy Cognitive Map by Active Hebbian Learning approach from here. What I have understand is that the model learns iteratively, at each step a new concept values enters and tune ...
0
votes
1answer
62 views
Keras custom loss function with weight function
My LSTM neural network predicts nominal values between -1 and 1. I would like to set up a custom loss function in Keras that assigns a weight function depending on the predicted sign.
If the predicted ...
1
vote
0answers
41 views
Worse performance after Hyperparameter tuning
I first construct a base model (using default parameters) and obtain MAE.
...
0
votes
1answer
24 views
SparseCategoricalcrossEntropy(from_logits=True) internally apply softmax?
Regarding Tensorflow/Keras SparseCategoricalcrossEntropy.
SparseCategoricalcrossEntropy(from_logits=True) expects the logits that has not been normalized by softmax....
1
vote
1answer
44 views
Difference in performance Sigmoid vs. Softmax
For the same Binary Image Classification task, if in the final layer I use 1 node with Sigmoid activation function and ...
2
votes
2answers
51 views
How to interpreter Binary Cross Entropy loss function?
I saw some examples of Autoencoders (on images) which use sigmoid as output layer and BinaryCrossentropy as loss function.
The ...
0
votes
0answers
11 views
Why checkpoint loss is different?
I am training a Mask RCNN model in Keras. I used checkpoints to save weights so I can resume training with the last optimized values.
However, the loss is different when I save the checkpoint and ...
0
votes
0answers
12 views
How to incorporate multi-task in CTR/recommendation model (deep & wide/ xDeepFM etc)?
I am building a rank algorithm for an e-commerce website that ranks the product based on likely hood of purchase and I have formulated this problem into a binary classification problem. Given each ...
2
votes
1answer
25 views
What is the entity of cross entropy (loss)
Cross-entropy (loss), $-\sum y_i\;\log(\hat{p_i})$, estimates the amount of information needed to encode $y$ using Huffman encoding based on the estimated probabilities $\hat{p}$. Therefore one could ...
1
vote
0answers
31 views
Implementing cosine embedding loss with labels 0 and 1
My dataset has two labels, 0 and 1, 1 meaning high similarity and 0 meaning high dissimilarity.
Two output vectors from the two-tower model are compared using dot product (with normalization) and ...
0
votes
0answers
11 views
Scene Graph Generation. How to choose a loss function
I am implementing a paper Graph R-CNN for Scene Graph Generation
They say:
For P (E|V:I), we use another binary cross-entropy loss on the relation proposals.
This ...
4
votes
2answers
53 views
Is it advisable to use a model which is underfit but gives very high accuracy?
I am training a model for a single-label classification task in Vision. In this training, I am using oversampling of all the classes, and MixUp augmentation, along with rotation and dihedral ...
1
vote
0answers
11 views
Incorrect derivative [duplicate]
Currently I'm reading a book named "Grokking Deep Learning" and I'm confused with the way author takes derivative from function. Let me explain.
We have loss function, where pred is ...
1
vote
1answer
26 views
LogCoshLoss on pytorch
Hi I am currently testing multiple loss on my code using PyTorch, but when I stumbled on log cosh loss function I did not find ...
0
votes
0answers
30 views
Why training loss is decreasing down too fast?
I have a dataset of video sequences, I have trained them, and calculated the training loss using mean square error, but my training loss is decreasing down very fast. Like 0.06-0.02. Is it just fine ...
0
votes
0answers
28 views
L1 vs. L2 Robustness?
I am very new to ML so I apologize in advance if the answer to my question is very obvious. I am reading about performance measures and how the L1 norm is more robust than the L2 norm. In other words, ...
1
vote
3answers
36 views
what n represents in the MSE loss function?
Neural Network Loss Function - Mean Square Error: questions about what 'n' signifies
I can't understand how the answers in this question answered the question. please help me to understand the ...
0
votes
1answer
31 views
Does Mixup requires two loss functions?
I created a neural network with multi-label classification using MSE.
Now, I would like to use Mixup. Do I need two loss functions (for each target one) or is the result the same if I just combine the ...
1
vote
1answer
34 views
Training set Distribution and Activation function/Loss function correlation
How should the probability distribution of the training set influence the choice of the activation function / loss function?
For instance if I have a Multinoulli distribution, which activation ...
2
votes
0answers
31 views
Running out of memory when calculating loss using SigmoidFocalCrossEntropy
Code Versions:
python == 3.8
tensorflow == 2.2.0
tensorflow-addons == 0.11.2
Recently I've been using tensorflow addon's focal loss function for one of my models. ...
0
votes
0answers
14 views
Weighting the loss function based on previous seen true positive rates
Similiar to class imbalance there is always something I would call "learnability imbalance" in multi-class classification. What I mean by that: Even when the classes are evenly distributed ...
0
votes
1answer
27 views
Keras: Very high loss for Autoencoder
I am trying to implement an autoencoder for prediction of multiple labels using Keras. This is a snippet:
...
0
votes
0answers
22 views
Is my approach for a loss function that adds more importance to negative samples reasonable?
For my current project I'm using XGBoost Regression to predict values y_pred with mean = 0 and std = 1. I want my model to place more emphasis on predicting samples right, where the true value y_true ...
0
votes
0answers
11 views
Using a multi-headed neural network, how should I approach the regression head loss
I have a multi-headed NN where one head performs multi-label classification and the other a regression task on a set of images. The classification head outputs a one-hot vector where each value in the ...
0
votes
1answer
17 views
Criteria for saving best model during training neural network?
I am doing 4-class semantic segmentation with U-net using generalised dice loss as loss function.
General approach to save best model during training is to monitor validation loss at each epoch and ...
0
votes
0answers
6 views
Is it okay to train Triplet Loss where the anchor and the positive are the same?
Triplet loss roughly defined as total of$$ max (- distance(Anchor,Negative) + distance(Anchor,Positive) +margin,0)$$
I'm working with triplets data, and it turns out that some triplets(just small ...
1
vote
1answer
28 views
Tree complexity and gamma parameter in xgboost
According to xgboost paper, regularization is given by:
$$\Omega(f) = \gamma T + \lambda || w||^2$$
where $\gamma$ is the complexity of a tree (i.e., number of leaves in the tree).
The parameter ...
0
votes
1answer
38 views
class weights formula for imbalanced dataset
I am trying to make some semantic segmentation. I have 7 imbalanced classes in my case.
I found several methods for handling Class Imbalance in a dataset is to perform Undersampling for the Majority ...
1
vote
1answer
38 views
“Optimal range for loss function”: Myth or Truth?
I am currently working on a regression problem using a deep neural network that given a volume 32x32x256 in input need to generate a second volume of the same dimensions in output, this is not a ...
0
votes
0answers
54 views
What's the difference between multiclass categorical crossentropy, mlogloss and multi:softprob?
As far as I understand, an objective is something I'm trying to optimize and an evaluation statistic is something I use to look for overfitting. I stumbled upon 4 losses that seem to be the same, but ...
0
votes
0answers
6 views
Proper loss for multiclass 2D segmentation with coarse masks
Trying to figure out which is the proper loss for the task that has:
High dimensional grayscale images with relatively small coarse annotated masks
With a high imbalance in classes, as bg is x100 ...
1
vote
1answer
50 views
LSTM for Stock Return Prediction
I am writing my masters thesis and am using LSTMs for daily stock return prediction. So far I am only predicting numerical values but will soon explore a classification style problem and predict ...
1
vote
0answers
89 views
BERT MLM overfitting [closed]
We are training the BERT model on masked language modeling task for the Russian Language. Our dataset consists of 60 mln texts with (128 tokens for each text) from online social networks, ...
0
votes
0answers
28 views
Multi-class cross entropy loss with negative sampling
So I have multiple networks, each representing a transformation, followed by a multi-class cross entropy loss. For each training sample, there's one right transformation and then the right label.
My ...
3
votes
1answer
42 views
Negative learning implementation in pytorch
I have read a paper on Negative Learning: https://arxiv.org/abs/1908.07387.
The idea is that you can train a network not only by telling what label of the sample is, but by telling what it surely is ...
0
votes
0answers
57 views
Interpretation of Loss and validation Loss in Keras
I am building a model to predict one label by taking one feature as an input. The two variables seems to be strongly correlated. I wanted to build a (sequential) Neural Network model with Keras in ...
0
votes
0answers
3 views
Use tf.image.ssim_multiscale for a batch of images in sequential model
I am trying to use SSIM as loss value for my Keras Sequential model. The output value is a set of 6 images - (6, 32, 28, 3). I want to implement a custom loss function for the model.fit() function but ...
1
vote
1answer
38 views
Binary Cross Entropy | Manual scalars [closed]
I am wanting to make print statements "showing my working out" of Binary Cross Entropy loss function, that works with ...
0
votes
2answers
33 views
In classification task, is it possible that a truly classified data has a higher loss compared to a miscallisfied data?
Given a classifier using softmax, is it possible that say, for data point a which our model has correctly classified, has higher loss compared to data point ...
1
vote
1answer
13 views
Why do neural networks use cost minimization of loss function and not profit maximization of profit function?
In neural networks, gradient descent is used to find optimum minimum value of cost function. Why this preference instead of finding maximum value of profit function?
What are the pros and cons of ...
0
votes
0answers
49 views
Is the output of this loss function considered as a percentage or something else?
I am currently working on fine-tuning the PEGASUS model for abstractive summarization.
The script for fine-tuning the class Trainer from Transformers is imported.
The output of training loss is ...
1
vote
0answers
14 views
How to interpret high loss value from model.evaluate() on test data
I'm collecting some metrics for my model's performance using:
...
0
votes
0answers
38 views
YOLO for timeseries, Error when training
I'm currently trying to implement the YOLO object detection algorithm for the localization and classification of events in time series ('signals').
To do this, I have defined a custom loss function, a ...
1
vote
0answers
15 views
