# Questions tagged [loss-function]

A function used to quantify the difference between observed data and predicted values according to a model. Minimization of loss functions is a way to estimate the parameters of the model.

437 questions
Filter by
Sorted by
Tagged with
9 views

### Optimizing the Loss Function For Another Metric

Suppose I have a machine learning model which is used to improve the profitability of a business. One of the components of the model is a loss function, say for measuring the success of a ...
13 views

### Negative loss values for adaptive loss in tensorflow

I have used adaptive loss implementation on a neural network, however after training a model long enough, I am getting negative loss values. Any help/suggestion would be highly appreciated! Please let ...
19 views

### Can someone explain the solution to the following problem?

Q) We want to learn a function f(x) of the form f(x) = ax + b which is parameterized by (a, b). Using squared error as the loss function, which of the following parameters would you use to model this ...
18 views

### Loss drops to NaN after a short time for a time series classification

here is my model code for a binary classification of a time series: ...
33 views

### Which Neural Network or Gradient Boosting framework is the simplest for Custom Loss Functions?

I need to implement a custom loss function. The function is relatively simple: $$-\sum \limits_{i=1}^m [O_{1,i} \cdot y_i-1] \ \cdot \ \operatorname{ReLu}(O_{1,i} \cdot \hat{y_i} - 1)$$ With $O$ being ...
25 views

### How to train a neural network where computing the loss requires multiple object values?

I want to train a function that given metadata about an image produces hyper-parameters for an algorithm which operates on the image. My understanding is (please forgive me I'm a novice here) a neural ...
5 views

### Problem in convergence of hebbian learning approach for Fuzzy Cognitive Map

I was trying to learn Fuzzy Cognitive Map by Active Hebbian Learning approach from here. What I have understand is that the model learns iteratively, at each step a new concept values enters and tune ...
62 views

### Keras custom loss function with weight function

My LSTM neural network predicts nominal values between -1 and 1. I would like to set up a custom loss function in Keras that assigns a weight function depending on the predicted sign. If the predicted ...
41 views

### Worse performance after Hyperparameter tuning

I first construct a base model (using default parameters) and obtain MAE. ...
24 views

### SparseCategoricalcrossEntropy(from_logits=True) internally apply softmax?

Regarding Tensorflow/Keras SparseCategoricalcrossEntropy. SparseCategoricalcrossEntropy(from_logits=True) expects the logits that has not been normalized by softmax....
44 views

### Difference in performance Sigmoid vs. Softmax

For the same Binary Image Classification task, if in the final layer I use 1 node with Sigmoid activation function and ...
51 views

### How to interpreter Binary Cross Entropy loss function?

I saw some examples of Autoencoders (on images) which use sigmoid as output layer and BinaryCrossentropy as loss function. The ...
11 views

### Why checkpoint loss is different?

I am training a Mask RCNN model in Keras. I used checkpoints to save weights so I can resume training with the last optimized values. However, the loss is different when I save the checkpoint and ...
12 views

### How to incorporate multi-task in CTR/recommendation model (deep & wide/ xDeepFM etc)?

I am building a rank algorithm for an e-commerce website that ranks the product based on likely hood of purchase and I have formulated this problem into a binary classification problem. Given each ...
25 views

### What is the entity of cross entropy (loss)

Cross-entropy (loss), $-\sum y_i\;\log(\hat{p_i})$, estimates the amount of information needed to encode $y$ using Huffman encoding based on the estimated probabilities $\hat{p}$. Therefore one could ...
31 views

### Implementing cosine embedding loss with labels 0 and 1

My dataset has two labels, 0 and 1, 1 meaning high similarity and 0 meaning high dissimilarity. Two output vectors from the two-tower model are compared using dot product (with normalization) and ...
11 views

### Scene Graph Generation. How to choose a loss function

I am implementing a paper Graph R-CNN for Scene Graph Generation They say: For P (E|V:I), we use another binary cross-entropy loss on the relation proposals. This ...
53 views

### Is it advisable to use a model which is underfit but gives very high accuracy?

I am training a model for a single-label classification task in Vision. In this training, I am using oversampling of all the classes, and MixUp augmentation, along with rotation and dihedral ...
11 views

### Incorrect derivative [duplicate]

Currently I'm reading a book named "Grokking Deep Learning" and I'm confused with the way author takes derivative from function. Let me explain. We have loss function, where pred is ...
26 views

### LogCoshLoss on pytorch

Hi I am currently testing multiple loss on my code using PyTorch, but when I stumbled on log cosh loss function I did not find ...
30 views

### Why training loss is decreasing down too fast?

I have a dataset of video sequences, I have trained them, and calculated the training loss using mean square error, but my training loss is decreasing down very fast. Like 0.06-0.02. Is it just fine ...
28 views

### L1 vs. L2 Robustness?

I am very new to ML so I apologize in advance if the answer to my question is very obvious. I am reading about performance measures and how the L1 norm is more robust than the L2 norm. In other words, ...
36 views

31 views

### Does Mixup requires two loss functions?

I created a neural network with multi-label classification using MSE. Now, I would like to use Mixup. Do I need two loss functions (for each target one) or is the result the same if I just combine the ...
34 views

### Training set Distribution and Activation function/Loss function correlation

How should the probability distribution of the training set influence the choice of the activation function / loss function? For instance if I have a Multinoulli distribution, which activation ...
31 views

### Running out of memory when calculating loss using SigmoidFocalCrossEntropy

Code Versions: python == 3.8 tensorflow == 2.2.0 tensorflow-addons == 0.11.2 Recently I've been using tensorflow addon's focal loss function for one of my models. ...
14 views

### Weighting the loss function based on previous seen true positive rates

Similiar to class imbalance there is always something I would call "learnability imbalance" in multi-class classification. What I mean by that: Even when the classes are evenly distributed ...
27 views

### Keras: Very high loss for Autoencoder

I am trying to implement an autoencoder for prediction of multiple labels using Keras. This is a snippet: ...
22 views

### Is my approach for a loss function that adds more importance to negative samples reasonable?

For my current project I'm using XGBoost Regression to predict values y_pred with mean = 0 and std = 1. I want my model to place more emphasis on predicting samples right, where the true value y_true ...
11 views

### Using a multi-headed neural network, how should I approach the regression head loss

I have a multi-headed NN where one head performs multi-label classification and the other a regression task on a set of images. The classification head outputs a one-hot vector where each value in the ...
17 views

### Criteria for saving best model during training neural network?

I am doing 4-class semantic segmentation with U-net using generalised dice loss as loss function. General approach to save best model during training is to monitor validation loss at each epoch and ...
6 views

### Is it okay to train Triplet Loss where the anchor and the positive are the same?

Triplet loss roughly defined as total of$$max (- distance(Anchor,Negative) + distance(Anchor,Positive) +margin,0)$$ I'm working with triplets data, and it turns out that some triplets(just small ...
28 views

### Tree complexity and gamma parameter in xgboost

According to xgboost paper, regularization is given by: $$\Omega(f) = \gamma T + \lambda || w||^2$$ where $\gamma$ is the complexity of a tree (i.e., number of leaves in the tree). The parameter ...
38 views

### class weights formula for imbalanced dataset

I am trying to make some semantic segmentation. I have 7 imbalanced classes in my case. I found several methods for handling Class Imbalance in a dataset is to perform Undersampling for the Majority ...
38 views

### “Optimal range for loss function”: Myth or Truth?

I am currently working on a regression problem using a deep neural network that given a volume 32x32x256 in input need to generate a second volume of the same dimensions in output, this is not a ...
54 views

### What's the difference between multiclass categorical crossentropy, mlogloss and multi:softprob?

As far as I understand, an objective is something I'm trying to optimize and an evaluation statistic is something I use to look for overfitting. I stumbled upon 4 losses that seem to be the same, but ...
6 views

### Proper loss for multiclass 2D segmentation with coarse masks

Trying to figure out which is the proper loss for the task that has: High dimensional grayscale images with relatively small coarse annotated masks With a high imbalance in classes, as bg is x100 ...
50 views

### LSTM for Stock Return Prediction

I am writing my masters thesis and am using LSTMs for daily stock return prediction. So far I am only predicting numerical values but will soon explore a classification style problem and predict ...
89 views

### BERT MLM overfitting [closed]

We are training the BERT model on masked language modeling task for the Russian Language. Our dataset consists of 60 mln texts with (128 tokens for each text) from online social networks, ...
28 views

### Multi-class cross entropy loss with negative sampling

So I have multiple networks, each representing a transformation, followed by a multi-class cross entropy loss. For each training sample, there's one right transformation and then the right label. My ...
42 views

### Negative learning implementation in pytorch

I have read a paper on Negative Learning: https://arxiv.org/abs/1908.07387. The idea is that you can train a network not only by telling what label of the sample is, but by telling what it surely is ...
57 views

### Interpretation of Loss and validation Loss in Keras

I am building a model to predict one label by taking one feature as an input. The two variables seems to be strongly correlated. I wanted to build a (sequential) Neural Network model with Keras in ...
3 views

### Use tf.image.ssim_multiscale for a batch of images in sequential model

I am trying to use SSIM as loss value for my Keras Sequential model. The output value is a set of 6 images - (6, 32, 28, 3). I want to implement a custom loss function for the model.fit() function but ...
38 views

### Binary Cross Entropy | Manual scalars [closed]

I am wanting to make print statements "showing my working out" of Binary Cross Entropy loss function, that works with ...
33 views

### In classification task, is it possible that a truly classified data has a higher loss compared to a miscallisfied data?

Given a classifier using softmax, is it possible that say, for data point a which our model has correctly classified, has higher loss compared to data point ...
13 views

### Why do neural networks use cost minimization of loss function and not profit maximization of profit function?

In neural networks, gradient descent is used to find optimum minimum value of cost function. Why this preference instead of finding maximum value of profit function? What are the pros and cons of ...
49 views

### Is the output of this loss function considered as a percentage or something else?

I am currently working on fine-tuning the PEGASUS model for abstractive summarization. The script for fine-tuning the class Trainer from Transformers is imported. The output of training loss is ...
14 views

### How to interpret high loss value from model.evaluate() on test data

I'm collecting some metrics for my model's performance using: ...