Questions tagged [loss-function]
A function used to quantify the difference between observed data and predicted values according to a model. Minimization of loss functions is a way to estimate the parameters of the model.
496
questions
0
votes
0
answers
5
views
TF - KL divergence vs Cosine similarity
i was performing some experiments and noticed something..wanted to run it by the community to see if my understanding is correct
y_true = [[0., 1.], [1., 1.]]
y_pred = [[1., 0.], [1., 1.]]
cosine_loss ...
- 188
1
vote
0
answers
12
views
Loss function for classification problem
So I'm working on a classification problem, I used convolutional neural networks to classify grayscale ECG beat images of dimension 200x200 (I had around 4000 images for each class in training and I ...
- 23
0
votes
0
answers
17
views
Loading saved model fails
I've trained a model and saved it in .h5 format. when I try loading it I received this error
...
- 1
0
votes
0
answers
41
views
How does the margin constant (alpha) in the triplet loss affect the training process when it is a constant?
How does the margin constant in the triplet loss formula affect the gradient calculation when its derivative will be zero?
- 101
1
vote
0
answers
16
views
Val Loss and manually calculated loss produce different values
I have a CNN classification model that uses loss: binary cross entropy:
...
- 201
0
votes
0
answers
10
views
Trying to implement a loss function read from a journal-article in python
Computer science undergrad here. I am trying to understand Eqn 12 from this paper so that I can implement it in python code. In this paper, the NN model takes a blurred image as input and outputs a ...
0
votes
0
answers
7
views
how to tune hyperparameters inn regression neural network
hope you are enjoying good health,i am trying to built a simple neural network which has to predict a shear wave well log values from other well logs,but my model's is stuck in mean absolute error of ...
- 1
0
votes
0
answers
19
views
Regression sequence output loss function
I am fairly new to deep learning, and I have the following task. Based on an audio sequence of shape (200, 1024), I have to predict two sequences of shape (200, 1) of continuous values (for e.g 0.5687)...
- 11
0
votes
0
answers
8
views
Large jumps in loss in simple transformer model?
As an exercise, I created a very simple transformer model that just sees the same simple batch of dummy data repeatedly and (one would assume) should quickly learn to fit it perfectly.
And indeed, ...
- 1
0
votes
0
answers
19
views
uncertainties in non-convex optimization problems (neural networks)
How do you treat statistical uncertainties coming from non-convex optimization problems?
More specifically, suppose you have a neural network. It is well known that the loss is not convex; the ...
- 3
0
votes
0
answers
27
views
calculating gradient descent
when using mini batch gradient descent , we perform backpropagation after each batch , ie we calculate the gradient after each batch , we also capture y-hat after each sample in the batch and finally ...
- 23
0
votes
0
answers
14
views
Converting a negative loss term to inverse
I'm training a classifier using this loss function:
$$ \mathcal{L} = \mathcal{L}_{CE} - \lambda_1 \mathcal{L}_{push} +\lambda_2 \mathcal{L}_{pull} $$
I need to maximize a certain value using $\...
- 239
1
vote
1
answer
28
views
how to calculate loss function?
i hope you are doing well ,
i want to ask a question regarding loss function in a neural network
i know that the loss function is calculated for each data point in the training set , and then the ...
- 23
1
vote
0
answers
12
views
If two functions are close apart can I proof the difference of their empirical loss is also small?
I am trying to understand the proof of Theorem 3 in the paper "A Universal Law of Robustness via isoperimetry" by Bubeck and Sellke.
Basically there exist atleast one $w_{L,e}$ in $\...
- 131
0
votes
0
answers
9
views
Is it possible for the (Cross Entropy) test loss to increase for a few epochs while the test accuracy also increases?
I came across the question stated in the title:
When training a model with the cross-entropy loss function, is it
possible for the test loss to increase for a few epochs while the
test accuracy also ...
- 1
0
votes
0
answers
60
views
PyTorch: LSTM training loss not decreasing; starting at very high loss
I am training an LSTM to give counts of the number of items in buckets. There are 252 buckets. However, I am running into an issue with very large MSELoss that does not decrease in training (meaning ...
- 101
0
votes
0
answers
14
views
How to improve L2 loss for generative autoencoder
I am working with a modified generative autoencoder and having issues getting the L2 sufficiently low.
I think problem is that because my data is over a very large range and is standardized to values ...
- 35
0
votes
0
answers
18
views
Custom multi-label cross-entropy loss that boosts weight of particular errors
I am using XGBoost for a multi-label classification problem (objective is 'multi:softmax' in XGBoost). In my case there are 16 discrete output labels where only one is correct. However, depending on ...
- 1
0
votes
0
answers
14
views
How to vectorize this loop process
Hi guys I want to ask if anyone knows how to vectorize this code to make it more optimal and faster.
...
- 11
0
votes
0
answers
17
views
From what function do come the gradients that I use to adjust weights?
I have a question about the loss function and the gradient.
So I'm following the fastai (https://github.com/fastai/fastbook) course and at the end of 4th chapter, I got myself wondering.
From what ...
0
votes
0
answers
21
views
Is Loss value (e.g., MSE loss) used in the calculation for parameter update when doing gradient descent?
My question is really simple. I know the theory behind gradient descent and parameter updates, what I really haven't found clarity on is that is the loss value (e.g., MSE value) used, i.e., multiplied ...
- 1
0
votes
1
answer
45
views
Regularization and loss function
I am currently trying to get a better understanding of regularization as a concept. This leads me to the following question:
Will regularization change when we change the loss function?
Is it correct ...
- 3
1
vote
0
answers
112
views
How to plot the computational graph and derive the update procedure of parameters using the backpropagation algorithm?
Please help me to solve this problem without a code (ps: this is a written problem):
Given the following loss function, please plot the computational graph, and derive the update procedure of ...
- 21
0
votes
0
answers
18
views
finding better threshold for the unsupervised problem
I am working on a unsupervised image reconstruction task. so i need to verify whether the reconstructed image by the model is good are not. I cant do that visually.
so i was calculating the sum of ...
- 101
0
votes
0
answers
16
views
weighted mse - weights as function of time
I am predicting timeseries data using LSTM (in tensorflow).
Currently I am using MSE as my metric of choice.
I would like to create my own custom Weighted MSE metric, such that the weights are a ...
- 1
2
votes
2
answers
49
views
Difference between loss and cost function in the specific context of MAE in multiple-regression?
I've often met with the Mean Absolute Error loss function when dealing with regression problems in Artificial Neural Networks, but I'm still slightly confused about the difference between the word '...
- 123
0
votes
0
answers
10
views
Increasing (negative) R2 coincident with decreasing (positive) MSE during hyper parameter optimisation
I have a densely connected NN and I'm running a hyper parameter optimisation for multi-target output. During hyper parameter optimisation training, each epoch KerasTuner focuses on val_loss.
During ...
- 71
0
votes
1
answer
39
views
Why does neural network need loss as scalar?
I have a loss function that's a weighted cross entropy loss for binary classification
...
- 1
0
votes
0
answers
121
views
one hot encode or not for segmentation when using dice loss
I am trying to perform binary semantic segmentation and using Dice loss as my loss function. I used to perform one-hot encoding in most of my segmentation tasks, especially when using cross-entropy ...
2
votes
1
answer
65
views
Why Should There Be Multiple Columns in Train Labels for One Model?
Going through the notebook on well known kaggle competition of favorita sales forecasting.
One puzzle is, after the data is split for train and testing, it seems ...
- 283
0
votes
0
answers
21
views
Classification with Fuzzy Class Labels
I am currently involved in a project involving fuzzy class labels.
To be clear, whereas classes are discrete and mutually exclusive in a typical binary classification task, the classes I am working ...
- 101
0
votes
0
answers
35
views
Distilling the knowledge of a binary cross entropy with sigmoid function model to a softmax model
I have a complex CNN architecture that uses a binary cross-entropy and sigmoid function for classification. However, due to hardware restraints I would like to compress my model using knowledge ...
0
votes
0
answers
13
views
How come the cost function changed in linear regression in andrew Ng stanford course in two different videos?
I'm a little bit confused, here is the cost function Andrew gave at the first place to minimize
but when he derived using probability, we minimize the same one but in a different sign
here h(theta) ...
0
votes
0
answers
33
views
Misconception about Cross entropy
I have difficulty understanding the cross entropy function, my confusion arises when it get to implementation in the code.
sor cross entrop is defined as $-\sum_{x} p(x)log(q(x))$ where $p(x)$ is the ...
1
vote
0
answers
20
views
Trainning CNN Metric Learning gradient is constantly 1
I'm training a CNN with shirts and bodycon photos. I have these two classes and about 15k photos. I'am trying to do Metric Learning with a Contrastive Loss, but my CNN is not learning because ...
- 11
1
vote
1
answer
45
views
Accuracy over 100%
I am using OpenFL, the Intel framework for Federated Learning.
If I run their tutorial example, I have that loss decreases and accuracy is in range 0-100%, like this:
...
- 57
0
votes
0
answers
8
views
Customized loss function for tail regions
I am try work with data that looks like below figure. Now my main regions of interest are towards the tails. Precisely, values < 70 and > 180.
Considering Mean Squared error as the baseline. I ...
- 11
1
vote
1
answer
61
views
Can't understand an MSE loss function in a paper
I'm reading a paper published in nips 2021.
There's a part in it that is confusing:
This loss term is the mean squared error of the normalized feature
vectors and can be written as what follows:
...
- 239
0
votes
0
answers
4
views
The effects of Double Logarithms (Log Cross Entropy Loss) + Overfitting
My network involves two losses: one is a binary cross entropy, and the other is a multi-label cross entropy.
The yellow graphs are the ones with double logarithm, meaning that we log(sum(ce_loss)). ...
- 1
0
votes
0
answers
10
views
Training Set: Why Loss Flatten but Accuracy continues to increase?
I took a the Coursera course: Convolutional Neural Networks in TensorFlow, and 1 of the quiz qn is
...
- 1
0
votes
0
answers
56
views
Adding a group specific penalty to binary cross-entropy
I want to implement a custom Keras loss function that consists of plain binary cross-entropy plus a penalty that increases the loss for false negatives from one class (each observation can belong to ...
- 1
0
votes
0
answers
24
views
How do you create a custom loss in tensorflow which uses a external tensor?
I have a problem where I want to minimize the monetary cost associated with the prediction error (Mean Error, ME) from the feature I want to predict. The monetary cost is calculated by multiplying ME ...
- 1
0
votes
0
answers
42
views
Loss value in Autoencoder neural network
We have an autoencoder neural network and we get a loss value (loss = binary cross entropy) convergent to a certain number.
What can we deduce from this value ? Is this value strictly related to the ...
0
votes
0
answers
9
views
give a 0 loss to predictions under some conditions
I am using CNN LSTM to predict a continuous quantity A one step ahead using features A and other features B, C and D with a lag of 10.
I have 15000 points in my datasets. I use the first 10000 to ...
- 1
0
votes
0
answers
18
views
Best loss function for baseN encoding LSTM model
everyone, I am trying to train an LSTM model for sequence prediction.
I have X and Y which are two numpy arrays. X is a list of integers (integer encoded strings) while to encode Y I used a BaseN ...
0
votes
0
answers
63
views
How to combine the loss in a multitask neural network?
when we train a model to learn two tasks at the same time (Multitask learning), we get losses from both tasks in the neural network and then we combine them.
I've seen several works where they've done ...
- 240
0
votes
1
answer
23
views
Awful predictions of RNN while MSE is very low
I have encountered a strange situation where the predictions of RNN are just awful despite the fact that NN has found a minimum of loss function at 0.002 for training and 0.0013-0.0015 for validation ...
0
votes
1
answer
55
views
resnet50 based model classification loss increase
I'm trying to classify fonts in images into 7 classes.
I wanted to use a pre-trained ResNet50 for the task and use its features to my classification. So I've followed some guide and came up with the ...
- 101
0
votes
0
answers
14
views
Custom Tensorflow loss function that disincentivizes all black pixels
I'm training a Tensorflow model that receives an image and segments the image into foreground and background. That is, if the input image is w x h x 3, then the ...
- 1
0
votes
0
answers
35
views
Custom loss function for out of distribution detection using CNN in Tensorflow 2.0+
My question is in reference to the paper "Learning Confidence for Out-of-Distribution Detection in Neural Networks".
I need help in creating a custom loss function in TensorFlow 2.0+ as per ...