# Questions tagged [loss-function]

A function used to quantify the difference between observed data and predicted values according to a model. Minimization of loss functions is a way to estimate the parameters of the model.

462 questions
Filter by
Sorted by
Tagged with
6 views

### Validation accuracy have an almost good accuracy but loss function is high too

Based on my project, I find a little problem there with the statement like this. I want to make model with neural networks for text dataset. Then I use Pad Sequence for my text and Array for the ...
27 views

### What if training loss is negative

I have designed a custom loss function where I have to maximize the KL divergence. So I took KL(P||Q) and negated it and now my loss function is -KL(P||Q) and this loss function leads to negative ...
18 views

### Siamese netwroks - how to choose loss function?

I have read several articles about siamese netwroks, and I understand that there are 3 different types of loss functions: ...
17 views

### Early stopping based on average val_loss of last ten epoches and with some n partiences

I am training a DNN with CNN in Keras. Though, I can write an EarlyStopping criteria based on val_loss but due to minor oscillations in the val_loss, I want to monitor the average validation loss over ...
22 views

### Finding logistic loss/negative log likelihood - binary logistic regression classification

I am new to ML and data science and am struggling with a simple problem. In my problem, I am given a series of datapoints $X_i$ where $X_i = (x_{i1}, x_{i2})$ with each data point having a label $y_i$ ...
28 views

### How do I know if my model's result is good enough?

I have a dataset of different people with their insurance cost. I have trained a neural-network to predict the insurance cost (charges column) based on the other features (age,bmi, etc.). Here is how ...
9 views

### How to treat losses asymmetrically in Keras?

I am developing a Keras NN that predicts three categories: A, B and C. However, the impact of errors is not symmetrical. i.e. If the network predicts A when the label is C, that is much less desirable ...
11 views

### Is this a correct implementation of a meandice loss function

HarDNet was trained to perform semantic segmentation of images. Increase of a meandice implies that the model quality improved. ...
27 views

### Cost function - Log Loss query

What is the purpose of using "log" in the logistic regression cost function "log loss"?
33 views

### How to Fine Tune a BERT model for sentiment analysis to get the best f1 score

I am building a multi-class sentiment analysis BERT model that's optimized to give the best f1 score. More specifically, I train each epoch by optimizing binary cross entropy per class, taking the ...
47 views

### Why is cross entropy based on Bernoulli or Multinoulli probability distribution?

When we use logistic regression, we use cross entropy as the loss function. However, based on my understanding and https://machinelearningmastery.com/cross-entropy-for-machine-learning/, cross entropy ...
10 views

### The formula of loss function uses '(i)' as power of expected and real variables. What does that mean?

In the formula below, could one understand $y^{(i)}$ as $y_i$ ? If not, what is the fundamental difference ? $$j(\theta_0, \theta_1) = \frac{1}{2m}\sum_{i=1}^m(h_{\theta}(x^{(i)})-y^{(i)})^2$$
12 views

### Keras Custom loss Penalize more when actual and prediction are on opposite sides of Zero

I'm training a model to predict percentage change in prices. Both MSE and RMSE are giving me up to 99% accuracy but when I check how often both actual and prediction are pointing in the same direction ...
25 views

### Understanding data and regularization loss

I am reading & coding along Neural Networks from scratch in Python. In it there is a section about L1 / L2 regularization ...
9 views

### Multi-class classification but a single feature sometimes boils it down to a binary-classification

I have a three-class classification problem for a large dataset. Classes are 0, 1 and 2. There's a categorical variable in my feature vectors such that when a sample point has this variable positive, ...
37 views

### Why margin loss is used in Capsule Network instead of Cross Entropy loss?

I'm reading the Capsule Network paper proposed by Hinton. I'm not sure why the margin loss is used instead of the cross entropy loss. Any intuitive explaination for this?
12 views

### validation loss early increase (during warm-up)

Several questions have been asked about validation loss behavior during training of a DNN. It's clear to me that validation loss and accuracy are somehow correlated, but their curves can differ from ...
6 views

### Loss function for normal distribution regression problem

My project involves training an input of random uniformly distributed data using regression (this is my approach) to output random normally distributed data. The issue with formulating the problem is ...
19 views

### Loss function for a regression model in image segmentation task

I am training a model to segment an image to predict the degree of damage (ranging from 0: no damage, to 5: severe damage) for each pixel of an image. I have approached it this way: ...
27 views

### How to use Softmax and CrossEntropyLoss in a classification problem?

I am new in the deep learning field and I would like to go in to understand some concepts. I started learn with pytorch and I have the next question: I constructed my own dataset that contains images ...
18 views

### Understanding deconvolutional network loss function

In the paper (1), there is a description of a deconvolutional network. The loss function (with only one layer) compares the colour channels of the orignal image with the colour channels of the ...
57 views

### Does gradient descent always find global minimum for specific regression type?

From my understanding, linear regression is used for predicting an output based on an input using a linear equation that is optimally fitted to some input data. We choose the best fitted linear ...
11 views

### selection of loss function to avoid overfitting by autoencoder in prediction a figure with a sharp rise

I have to select the loss function to avoid overfitting by autoencoder in prediction of this figure that has a sharp raise, I would like to find how to avoid overfitting by autoencoder in prediction a ...
42 views

### Loss in multi-class classification

I have a multi-class classification task. One of the standard approach in choosing loss function is to use a CrossEntropyLoss. It is a good option when classes are standonlone and not similar to each ...
12 views

### Deep autoencoder: validation loss doesn't change

I'm trying to understand autoencoders and reproduced some code from Keras documentation: ...
34 views

### Word embedding autoencoder

I'm trying to train a word embedding autoencoder, but it either doesn't train, or trains but doesn't make predictions. I know I'm doing something wrong, so any help is greatly appreciated. Here is my ...
9 views

### Continuous Bag of Words loss function and training objective

CBOW from what I understand, obtains a probability distribution $P(w|c)$ for all words $w$ in the vocabulary, given context $c$. Th loss function is: $-logP(w|c)$, which means this would be maximised ...
18 views

### loss function for simultaneous prediction of multiple OHE vectors

Hi I am trying to figure out how to set up the loss function for a model where the outputs are 3 one hot encoded vectors which are predicted simultaneously. This is a fully connected feed-forward ...
18 views

### Lovasz Softmax loss explanation

I would like to use Lovasz softmax for foreground background semantic segmentation because of its ability to improve segmentation with Jaccard index according to paper. I got the idea that its a ...
44 views

### Reducing Validation loss for Triplet Loss Embeddings

I'm trying to create a facial recognition detector using triplet loss followed by a kNN algorithm. I have roughly 10000 input images with 3 different classes, input size is 80x80. Model structure uses ...
17 views

### Significance of Convex Loss Function with Nonlinear Models

When used in a linear model, a convex loss function guarantees a unique global minimum for the parameters, which can be found by local optimization methods. However, when the model is nonlinear (e.g. ...
15 views

### why the accuracy result and the loss result of an ANN model is inconsistent?

I trained a model based on an ANN and the accuracy is 94.65% almost every time while the loss result is 12.06%. Now my question is shouldn't the loss of the model be (100-94 = 6%) or near it? Why it ...
24 views

### Why would the accuracy of a model change when the loss doesn't?

I've trained 8 models based on the same architecture (convolutional neural network), and each uses a different data augmentation method. The accuracy of the models fluctuates greatly while the loss ...
22 views

### Inverting a matrix using a convolutional neural network

Just for a fun exercise, I am trying to invert a matrix, say size 28x28 (or even 5x5) with a neural network. The way I approached this (quite naively) is as follows: I built a fully convolutional ...
38 views

### Custom keras callbacks and changing weight (beta) of regularization term in variational autoencoder loss function

The variational autoencoder loss function is this: Loss = Loss_reconstruction + Beta * Loss_kld. I am trying to efficiently implement Kullback-Liebler Divergence Cyclic Annealing--that is changing the ...
12 views

### Unable to train neural network for prediction

My data consists of a time series of values $\pm1$ and I am trying to apply a RBF NN as a function approximator. Essentially, the NN will take as input one data sample and predict the next sample (...
15 views

### Optimizing the Loss Function For Another Metric

Suppose I have a machine learning model which is used to improve the profitability of a business. One of the components of the model is a loss function, say for measuring the success of a ...
22 views

### Negative loss values for adaptive loss in tensorflow

I have used adaptive loss implementation on a neural network, however after training a model long enough, I am getting negative loss values. Any help/suggestion would be highly appreciated! Please let ...
119 views

### Can someone explain the solution to the following problem?

Q) We want to learn a function f(x) of the form f(x) = ax + b which is parameterized by (a, b). Using squared error as the loss function, which of the following parameters would you use to model this ...
45 views

### Loss drops to NaN after a short time for a time series classification

here is my model code for a binary classification of a time series: ...
39 views

### Which Neural Network or Gradient Boosting framework is the simplest for Custom Loss Functions?

I need to implement a custom loss function. The function is relatively simple: $$-\sum \limits_{i=1}^m [O_{1,i} \cdot y_i-1] \ \cdot \ \operatorname{ReLu}(O_{1,i} \cdot \hat{y_i} - 1)$$ With $O$ being ...
26 views

### How to train a neural network where computing the loss requires multiple object values?

I want to train a function that given metadata about an image produces hyper-parameters for an algorithm which operates on the image. My understanding is (please forgive me I'm a novice here) a neural ...
11 views

### Problem in convergence of hebbian learning approach for Fuzzy Cognitive Map

I was trying to learn Fuzzy Cognitive Map by Active Hebbian Learning approach from here. What I have understand is that the model learns iteratively, at each step a new concept values enters and tune ...
99 views

### Keras custom loss function with weight function

My LSTM neural network predicts nominal values between -1 and 1. I would like to set up a custom loss function in Keras that assigns a weight function depending on the predicted sign. If the predicted ...
97 views

### Worse performance after Hyperparameter tuning

I first construct a base model (using default parameters) and obtain MAE. ...
118 views

### SparseCategoricalcrossEntropy(from_logits=True) internally apply softmax?

Regarding Tensorflow/Keras SparseCategoricalcrossEntropy. SparseCategoricalcrossEntropy(from_logits=True) expects the logits that has not been normalized by softmax....
86 views

### Difference in performance Sigmoid vs. Softmax

For the same Binary Image Classification task, if in the final layer I use 1 node with Sigmoid activation function and ...
133 views

### How to interpreter Binary Cross Entropy loss function?

I saw some examples of Autoencoders (on images) which use sigmoid as output layer and BinaryCrossentropy as loss function. The ...