Questions tagged [loss-function]

A function used to quantify the difference between observed data and predicted values according to a model. Minimization of loss functions is a way to estimate the parameters of the model.

Filter by
Sorted by
Tagged with
0 votes
0 answers
5 views

TF - KL divergence vs Cosine similarity

i was performing some experiments and noticed something..wanted to run it by the community to see if my understanding is correct y_true = [[0., 1.], [1., 1.]] y_pred = [[1., 0.], [1., 1.]] cosine_loss ...
user avatar
1 vote
0 answers
12 views

Loss function for classification problem

So I'm working on a classification problem, I used convolutional neural networks to classify grayscale ECG beat images of dimension 200x200 (I had around 4000 images for each class in training and I ...
user avatar
  • 23
0 votes
0 answers
17 views

Loading saved model fails

I've trained a model and saved it in .h5 format. when I try loading it I received this error ...
user avatar
  • 1
0 votes
0 answers
41 views

How does the margin constant (alpha) in the triplet loss affect the training process when it is a constant?

How does the margin constant in the triplet loss formula affect the gradient calculation when its derivative will be zero?
user avatar
  • 101
1 vote
0 answers
16 views

Val Loss and manually calculated loss produce different values

I have a CNN classification model that uses loss: binary cross entropy: ...
user avatar
  • 201
0 votes
0 answers
10 views

Trying to implement a loss function read from a journal-article in python

Computer science undergrad here. I am trying to understand Eqn 12 from this paper so that I can implement it in python code. In this paper, the NN model takes a blurred image as input and outputs a ...
user avatar
0 votes
0 answers
7 views

how to tune hyperparameters inn regression neural network

hope you are enjoying good health,i am trying to built a simple neural network which has to predict a shear wave well log values from other well logs,but my model's is stuck in mean absolute error of ...
user avatar
0 votes
0 answers
19 views

Regression sequence output loss function

I am fairly new to deep learning, and I have the following task. Based on an audio sequence of shape (200, 1024), I have to predict two sequences of shape (200, 1) of continuous values (for e.g 0.5687)...
user avatar
0 votes
0 answers
8 views

Large jumps in loss in simple transformer model?

As an exercise, I created a very simple transformer model that just sees the same simple batch of dummy data repeatedly and (one would assume) should quickly learn to fit it perfectly. And indeed, ...
user avatar
0 votes
0 answers
19 views

uncertainties in non-convex optimization problems (neural networks)

How do you treat statistical uncertainties coming from non-convex optimization problems? More specifically, suppose you have a neural network. It is well known that the loss is not convex; the ...
user avatar
  • 3
0 votes
0 answers
27 views

calculating gradient descent

when using mini batch gradient descent , we perform backpropagation after each batch , ie we calculate the gradient after each batch , we also capture y-hat after each sample in the batch and finally ...
user avatar
  • 23
0 votes
0 answers
14 views

Converting a negative loss term to inverse

I'm training a classifier using this loss function: $$ \mathcal{L} = \mathcal{L}_{CE} - \lambda_1 \mathcal{L}_{push} +\lambda_2 \mathcal{L}_{pull} $$ I need to maximize a certain value using $\...
user avatar
1 vote
1 answer
28 views

how to calculate loss function?

i hope you are doing well , i want to ask a question regarding loss function in a neural network i know that the loss function is calculated for each data point in the training set , and then the ...
user avatar
  • 23
1 vote
0 answers
12 views

If two functions are close apart can I proof the difference of their empirical loss is also small?

I am trying to understand the proof of Theorem 3 in the paper "A Universal Law of Robustness via isoperimetry" by Bubeck and Sellke. Basically there exist atleast one $w_{L,e}$ in $\...
user avatar
  • 131
0 votes
0 answers
9 views

Is it possible for the (Cross Entropy) test loss to increase for a few epochs while the test accuracy also increases?

I came across the question stated in the title: When training a model with the cross-entropy loss function, is it possible for the test loss to increase for a few epochs while the test accuracy also ...
user avatar
0 votes
0 answers
60 views

PyTorch: LSTM training loss not decreasing; starting at very high loss

I am training an LSTM to give counts of the number of items in buckets. There are 252 buckets. However, I am running into an issue with very large MSELoss that does not decrease in training (meaning ...
user avatar
0 votes
0 answers
14 views

How to improve L2 loss for generative autoencoder

I am working with a modified generative autoencoder and having issues getting the L2 sufficiently low. I think problem is that because my data is over a very large range and is standardized to values ...
user avatar
  • 35
0 votes
0 answers
18 views

Custom multi-label cross-entropy loss that boosts weight of particular errors

I am using XGBoost for a multi-label classification problem (objective is 'multi:softmax' in XGBoost). In my case there are 16 discrete output labels where only one is correct. However, depending on ...
user avatar
0 votes
0 answers
14 views

How to vectorize this loop process

Hi guys I want to ask if anyone knows how to vectorize this code to make it more optimal and faster. ...
user avatar
0 votes
0 answers
17 views

From what function do come the gradients that I use to adjust weights?

I have a question about the loss function and the gradient. So I'm following the fastai (https://github.com/fastai/fastbook) course and at the end of 4th chapter, I got myself wondering. From what ...
user avatar
0 votes
0 answers
21 views

Is Loss value (e.g., MSE loss) used in the calculation for parameter update when doing gradient descent?

My question is really simple. I know the theory behind gradient descent and parameter updates, what I really haven't found clarity on is that is the loss value (e.g., MSE value) used, i.e., multiplied ...
user avatar
  • 1
0 votes
1 answer
45 views

Regularization and loss function

I am currently trying to get a better understanding of regularization as a concept. This leads me to the following question: Will regularization change when we change the loss function? Is it correct ...
user avatar
1 vote
0 answers
112 views

How to plot the computational graph and derive the update procedure of parameters using the backpropagation algorithm?

Please help me to solve this problem without a code (ps: this is a written problem): Given the following loss function, please plot the computational graph, and derive the update procedure of ...
user avatar
  • 21
0 votes
0 answers
18 views

finding better threshold for the unsupervised problem

I am working on a unsupervised image reconstruction task. so i need to verify whether the reconstructed image by the model is good are not. I cant do that visually. so i was calculating the sum of ...
user avatar
  • 101
0 votes
0 answers
16 views

weighted mse - weights as function of time

I am predicting timeseries data using LSTM (in tensorflow). Currently I am using MSE as my metric of choice. I would like to create my own custom Weighted MSE metric, such that the weights are a ...
user avatar
  • 1
2 votes
2 answers
49 views

Difference between loss and cost function in the specific context of MAE in multiple-regression?

I've often met with the Mean Absolute Error loss function when dealing with regression problems in Artificial Neural Networks, but I'm still slightly confused about the difference between the word '...
user avatar
0 votes
0 answers
10 views

Increasing (negative) R2 coincident with decreasing (positive) MSE during hyper parameter optimisation

I have a densely connected NN and I'm running a hyper parameter optimisation for multi-target output. During hyper parameter optimisation training, each epoch KerasTuner focuses on val_loss. During ...
user avatar
  • 71
0 votes
1 answer
39 views

Why does neural network need loss as scalar?

I have a loss function that's a weighted cross entropy loss for binary classification ...
user avatar
0 votes
0 answers
121 views

one hot encode or not for segmentation when using dice loss

I am trying to perform binary semantic segmentation and using Dice loss as my loss function. I used to perform one-hot encoding in most of my segmentation tasks, especially when using cross-entropy ...
user avatar
2 votes
1 answer
65 views

Why Should There Be Multiple Columns in Train Labels for One Model?

Going through the notebook on well known kaggle competition of favorita sales forecasting. One puzzle is, after the data is split for train and testing, it seems ...
user avatar
  • 283
0 votes
0 answers
21 views

Classification with Fuzzy Class Labels

I am currently involved in a project involving fuzzy class labels. To be clear, whereas classes are discrete and mutually exclusive in a typical binary classification task, the classes I am working ...
user avatar
0 votes
0 answers
35 views

Distilling the knowledge of a binary cross entropy with sigmoid function model to a softmax model

I have a complex CNN architecture that uses a binary cross-entropy and sigmoid function for classification. However, due to hardware restraints I would like to compress my model using knowledge ...
user avatar
0 votes
0 answers
13 views

How come the cost function changed in linear regression in andrew Ng stanford course in two different videos?

I'm a little bit confused, here is the cost function Andrew gave at the first place to minimize but when he derived using probability, we minimize the same one but in a different sign here h(theta) ...
user avatar
0 votes
0 answers
33 views

Misconception about Cross entropy

I have difficulty understanding the cross entropy function, my confusion arises when it get to implementation in the code. sor cross entrop is defined as $-\sum_{x} p(x)log(q(x))$ where $p(x)$ is the ...
user avatar
1 vote
0 answers
20 views

Trainning CNN Metric Learning gradient is constantly 1

I'm training a CNN with shirts and bodycon photos. I have these two classes and about 15k photos. I'am trying to do Metric Learning with a Contrastive Loss, but my CNN is not learning because ...
user avatar
1 vote
1 answer
45 views

Accuracy over 100%

I am using OpenFL, the Intel framework for Federated Learning. If I run their tutorial example, I have that loss decreases and accuracy is in range 0-100%, like this: ...
user avatar
0 votes
0 answers
8 views

Customized loss function for tail regions

I am try work with data that looks like below figure. Now my main regions of interest are towards the tails. Precisely, values < 70 and > 180. Considering Mean Squared error as the baseline. I ...
user avatar
1 vote
1 answer
61 views

Can't understand an MSE loss function in a paper

I'm reading a paper published in nips 2021. There's a part in it that is confusing: This loss term is the mean squared error of the normalized feature vectors and can be written as what follows: ...
user avatar
0 votes
0 answers
4 views

The effects of Double Logarithms (Log Cross Entropy Loss) + Overfitting

My network involves two losses: one is a binary cross entropy, and the other is a multi-label cross entropy. The yellow graphs are the ones with double logarithm, meaning that we log(sum(ce_loss)). ...
user avatar
  • 1
0 votes
0 answers
10 views

Training Set: Why Loss Flatten but Accuracy continues to increase?

I took a the Coursera course: Convolutional Neural Networks in TensorFlow, and 1 of the quiz qn is ...
user avatar
  • 1
0 votes
0 answers
56 views

Adding a group specific penalty to binary cross-entropy

I want to implement a custom Keras loss function that consists of plain binary cross-entropy plus a penalty that increases the loss for false negatives from one class (each observation can belong to ...
user avatar
  • 1
0 votes
0 answers
24 views

How do you create a custom loss in tensorflow which uses a external tensor?

I have a problem where I want to minimize the monetary cost associated with the prediction error (Mean Error, ME) from the feature I want to predict. The monetary cost is calculated by multiplying ME ...
user avatar
  • 1
0 votes
0 answers
42 views

Loss value in Autoencoder neural network

We have an autoencoder neural network and we get a loss value (loss = binary cross entropy) convergent to a certain number. What can we deduce from this value ? Is this value strictly related to the ...
user avatar
0 votes
0 answers
9 views

give a 0 loss to predictions under some conditions

I am using CNN LSTM to predict a continuous quantity A one step ahead using features A and other features B, C and D with a lag of 10. I have 15000 points in my datasets. I use the first 10000 to ...
user avatar
  • 1
0 votes
0 answers
18 views

Best loss function for baseN encoding LSTM model

everyone, I am trying to train an LSTM model for sequence prediction. I have X and Y which are two numpy arrays. X is a list of integers (integer encoded strings) while to encode Y I used a BaseN ...
user avatar
0 votes
0 answers
63 views

How to combine the loss in a multitask neural network?

when we train a model to learn two tasks at the same time (Multitask learning), we get losses from both tasks in the neural network and then we combine them. I've seen several works where they've done ...
user avatar
  • 240
0 votes
1 answer
23 views

Awful predictions of RNN while MSE is very low

I have encountered a strange situation where the predictions of RNN are just awful despite the fact that NN has found a minimum of loss function at 0.002 for training and 0.0013-0.0015 for validation ...
user avatar
0 votes
1 answer
55 views

resnet50 based model classification loss increase

I'm trying to classify fonts in images into 7 classes. I wanted to use a pre-trained ResNet50 for the task and use its features to my classification. So I've followed some guide and came up with the ...
user avatar
  • 101
0 votes
0 answers
14 views

Custom Tensorflow loss function that disincentivizes all black pixels

I'm training a Tensorflow model that receives an image and segments the image into foreground and background. That is, if the input image is w x h x 3, then the ...
user avatar
0 votes
0 answers
35 views

Custom loss function for out of distribution detection using CNN in Tensorflow 2.0+

My question is in reference to the paper "Learning Confidence for Out-of-Distribution Detection in Neural Networks". I need help in creating a custom loss function in TensorFlow 2.0+ as per ...
user avatar

1
2 3 4 5
10