# Questions tagged [loss-function]

A function used to quantify the difference between observed data and predicted values according to a model. Minimization of loss functions is a way to estimate the parameters of the model.

496 questions
Filter by
Sorted by
Tagged with
5 views

### TF - KL divergence vs Cosine similarity

i was performing some experiments and noticed something..wanted to run it by the community to see if my understanding is correct y_true = [[0., 1.], [1., 1.]] y_pred = [[1., 0.], [1., 1.]] cosine_loss ...
1 vote
12 views

### Loss function for classification problem

So I'm working on a classification problem, I used convolutional neural networks to classify grayscale ECG beat images of dimension 200x200 (I had around 4000 images for each class in training and I ...
17 views

I've trained a model and saved it in .h5 format. when I try loading it I received this error ...
41 views

### How does the margin constant (alpha) in the triplet loss affect the training process when it is a constant?

How does the margin constant in the triplet loss formula affect the gradient calculation when its derivative will be zero?
1 vote
16 views

### Val Loss and manually calculated loss produce different values

I have a CNN classification model that uses loss: binary cross entropy: ...
10 views

### Trying to implement a loss function read from a journal-article in python

Computer science undergrad here. I am trying to understand Eqn 12 from this paper so that I can implement it in python code. In this paper, the NN model takes a blurred image as input and outputs a ...
7 views

### how to tune hyperparameters inn regression neural network

hope you are enjoying good health,i am trying to built a simple neural network which has to predict a shear wave well log values from other well logs,but my model's is stuck in mean absolute error of ...
19 views

### Regression sequence output loss function

I am fairly new to deep learning, and I have the following task. Based on an audio sequence of shape (200, 1024), I have to predict two sequences of shape (200, 1) of continuous values (for e.g 0.5687)...
8 views

### Large jumps in loss in simple transformer model?

As an exercise, I created a very simple transformer model that just sees the same simple batch of dummy data repeatedly and (one would assume) should quickly learn to fit it perfectly. And indeed, ...
19 views

### uncertainties in non-convex optimization problems (neural networks)

How do you treat statistical uncertainties coming from non-convex optimization problems? More specifically, suppose you have a neural network. It is well known that the loss is not convex; the ...
27 views

when using mini batch gradient descent , we perform backpropagation after each batch , ie we calculate the gradient after each batch , we also capture y-hat after each sample in the batch and finally ...
14 views

9 views

### Is it possible for the (Cross Entropy) test loss to increase for a few epochs while the test accuracy also increases?

I came across the question stated in the title: When training a model with the cross-entropy loss function, is it possible for the test loss to increase for a few epochs while the test accuracy also ...
60 views

### PyTorch: LSTM training loss not decreasing; starting at very high loss

I am training an LSTM to give counts of the number of items in buckets. There are 252 buckets. However, I am running into an issue with very large MSELoss that does not decrease in training (meaning ...
14 views

### How to improve L2 loss for generative autoencoder

I am working with a modified generative autoencoder and having issues getting the L2 sufficiently low. I think problem is that because my data is over a very large range and is standardized to values ...
18 views

### Custom multi-label cross-entropy loss that boosts weight of particular errors

I am using XGBoost for a multi-label classification problem (objective is 'multi:softmax' in XGBoost). In my case there are 16 discrete output labels where only one is correct. However, depending on ...
14 views

### How to vectorize this loop process

Hi guys I want to ask if anyone knows how to vectorize this code to make it more optimal and faster. ...
17 views

### From what function do come the gradients that I use to adjust weights?

I have a question about the loss function and the gradient. So I'm following the fastai (https://github.com/fastai/fastbook) course and at the end of 4th chapter, I got myself wondering. From what ...
21 views

### Is Loss value (e.g., MSE loss) used in the calculation for parameter update when doing gradient descent?

My question is really simple. I know the theory behind gradient descent and parameter updates, what I really haven't found clarity on is that is the loss value (e.g., MSE value) used, i.e., multiplied ...
45 views

### Regularization and loss function

I am currently trying to get a better understanding of regularization as a concept. This leads me to the following question: Will regularization change when we change the loss function? Is it correct ...
1 vote
112 views

### How to plot the computational graph and derive the update procedure of parameters using the backpropagation algorithm?

Please help me to solve this problem without a code (ps: this is a written problem): Given the following loss function, please plot the computational graph, and derive the update procedure of ...
18 views

### finding better threshold for the unsupervised problem

I am working on a unsupervised image reconstruction task. so i need to verify whether the reconstructed image by the model is good are not. I cant do that visually. so i was calculating the sum of ...
16 views

### weighted mse - weights as function of time

I am predicting timeseries data using LSTM (in tensorflow). Currently I am using MSE as my metric of choice. I would like to create my own custom Weighted MSE metric, such that the weights are a ...
49 views

### Difference between loss and cost function in the specific context of MAE in multiple-regression?

I've often met with the Mean Absolute Error loss function when dealing with regression problems in Artificial Neural Networks, but I'm still slightly confused about the difference between the word '...
10 views

### Increasing (negative) R2 coincident with decreasing (positive) MSE during hyper parameter optimisation

I have a densely connected NN and I'm running a hyper parameter optimisation for multi-target output. During hyper parameter optimisation training, each epoch KerasTuner focuses on val_loss. During ...
39 views

### Why does neural network need loss as scalar?

I have a loss function that's a weighted cross entropy loss for binary classification ...
121 views

### one hot encode or not for segmentation when using dice loss

I am trying to perform binary semantic segmentation and using Dice loss as my loss function. I used to perform one-hot encoding in most of my segmentation tasks, especially when using cross-entropy ...
65 views

### Why Should There Be Multiple Columns in Train Labels for One Model?

Going through the notebook on well known kaggle competition of favorita sales forecasting. One puzzle is, after the data is split for train and testing, it seems ...
21 views

### Classification with Fuzzy Class Labels

I am currently involved in a project involving fuzzy class labels. To be clear, whereas classes are discrete and mutually exclusive in a typical binary classification task, the classes I am working ...
35 views

### Distilling the knowledge of a binary cross entropy with sigmoid function model to a softmax model

I have a complex CNN architecture that uses a binary cross-entropy and sigmoid function for classification. However, due to hardware restraints I would like to compress my model using knowledge ...
13 views

### How come the cost function changed in linear regression in andrew Ng stanford course in two different videos?

I'm a little bit confused, here is the cost function Andrew gave at the first place to minimize but when he derived using probability, we minimize the same one but in a different sign here h(theta) ...
33 views

I have difficulty understanding the cross entropy function, my confusion arises when it get to implementation in the code. sor cross entrop is defined as $-\sum_{x} p(x)log(q(x))$ where $p(x)$ is the ...
1 vote
20 views

### Trainning CNN Metric Learning gradient is constantly 1

I'm training a CNN with shirts and bodycon photos. I have these two classes and about 15k photos. I'am trying to do Metric Learning with a Contrastive Loss, but my CNN is not learning because ...
1 vote
45 views

### Accuracy over 100%

I am using OpenFL, the Intel framework for Federated Learning. If I run their tutorial example, I have that loss decreases and accuracy is in range 0-100%, like this: ...
8 views

### Customized loss function for tail regions

I am try work with data that looks like below figure. Now my main regions of interest are towards the tails. Precisely, values < 70 and > 180. Considering Mean Squared error as the baseline. I ...
1 vote
61 views

### Can't understand an MSE loss function in a paper

I'm reading a paper published in nips 2021. There's a part in it that is confusing: This loss term is the mean squared error of the normalized feature vectors and can be written as what follows: ...
4 views

### The effects of Double Logarithms (Log Cross Entropy Loss) + Overfitting

My network involves two losses: one is a binary cross entropy, and the other is a multi-label cross entropy. The yellow graphs are the ones with double logarithm, meaning that we log(sum(ce_loss)). ...
10 views

### Training Set: Why Loss Flatten but Accuracy continues to increase?

I took a the Coursera course: Convolutional Neural Networks in TensorFlow, and 1 of the quiz qn is ...
56 views

### Adding a group specific penalty to binary cross-entropy

I want to implement a custom Keras loss function that consists of plain binary cross-entropy plus a penalty that increases the loss for false negatives from one class (each observation can belong to ...
24 views

### How do you create a custom loss in tensorflow which uses a external tensor?

I have a problem where I want to minimize the monetary cost associated with the prediction error (Mean Error, ME) from the feature I want to predict. The monetary cost is calculated by multiplying ME ...
42 views

### Loss value in Autoencoder neural network

We have an autoencoder neural network and we get a loss value (loss = binary cross entropy) convergent to a certain number. What can we deduce from this value ? Is this value strictly related to the ...
9 views

### give a 0 loss to predictions under some conditions

I am using CNN LSTM to predict a continuous quantity A one step ahead using features A and other features B, C and D with a lag of 10. I have 15000 points in my datasets. I use the first 10000 to ...
18 views

### Best loss function for baseN encoding LSTM model

everyone, I am trying to train an LSTM model for sequence prediction. I have X and Y which are two numpy arrays. X is a list of integers (integer encoded strings) while to encode Y I used a BaseN ...
63 views

### How to combine the loss in a multitask neural network?

when we train a model to learn two tasks at the same time (Multitask learning), we get losses from both tasks in the neural network and then we combine them. I've seen several works where they've done ...
23 views

### Awful predictions of RNN while MSE is very low

I have encountered a strange situation where the predictions of RNN are just awful despite the fact that NN has found a minimum of loss function at 0.002 for training and 0.0013-0.0015 for validation ...
55 views

### resnet50 based model classification loss increase

I'm trying to classify fonts in images into 7 classes. I wanted to use a pre-trained ResNet50 for the task and use its features to my classification. So I've followed some guide and came up with the ...