Data Science

Questions tagged [loss-function]

Ask Question

A function used to quantify the difference between observed data and predicted values according to a model. Minimization of loss functions is a way to estimate the parameters of the model.

292 questions
Newest
Active
Bountied
Unanswered
Filter by
Sorted by
Tagged with
1
vote
0answers
16 views

If mean absolute loss is not differentiable, how it can be used in neural networks? which majorly are trained using back-propagation

If Mean Absolute Error (MAE) loss is not differentiable, how can it be used in neural networks? which majorly are trained using back-propagation I am wondering if MAE is not differentiable how they ...
1
vote
0answers
38 views

Keras: do `sample weights` take part in the derivatives

According to Keras documentation, sample_weight can be used in order to give any sample in the training data a different importance in the loss. I have googled ...
0
votes
1answer
16 views

PyTorch MultiLayer Perceptron Classification Size of Features vs Labels Wrong

I am getting the following error: ...
1
vote
2answers
25 views

Non-categorical loss in Keras

I am training a neural network (arbitrary architecture) and I have a label space that is not one-hot encoded, but continuous. The reason is that for the given problem, it is not possible to assign a ...
0
votes
0answers
7 views

Picking a model to go ahead with for a WGAN

I have been trying to train a model using WGAN loss functions, working different learning rates to choose my hyper parameters based on advice. I was told to try looking into keeping everything simple ...
0
votes
0answers
24 views

Loss and Metrics for COCO Keypoints

I am using Keras to train different models on the COCO keypoints dataset. All of the models I am working with are used for image segmentation, so they output heatmaps corresponding to the labels. All ...
0
votes
1answer
25 views

Generator losses in WGAN and potential convergence failure

I have been training a WGAN for a while now, with my generator training once in every five epochs. I have tried several model architectures(no of filters) and also tried varying the relationship with ...
0
votes
0answers
21 views

U-Net: Accuracy drops to zero - How do I prevent this from happening?

I have a problem, when training a U-Net. When starting the Training, the Accuracy increases and the loss steadily goes down. At epoch 40, in my example, the validation loss jumps to the maximum and ...
0
votes
1answer
29 views

Multi-class classification with discrete output: Which loss function and activation to choose?

I'm working with a multi-class classification problem, using Keras Sequential models. In my dataset, the output class has one of the following values: ...
1
vote
0answers
9 views

How to update edge features in a graph using a loss function?

Given a directed, edge attributed graph G, where the edge attribute is a probability value, and a particular node N (with binary features f1 and f2) in G, the algorithm that I want to implement is as ...
1
vote
0answers
36 views

Constraining a neural net during training

I have simple "image-like" objects that contain non-linearly encoded information about real images. Each real image is zero except for two pixels, whose float values sum to unity. I created a simple ...
0
votes
1answer
21 views

Multi-Label Loss function and model training

I'm working on Multi-Label problem i.e output can predict 1 or more label as an output and hence training data also have multiple labels. Somehow I'm not able to map such ML model training. Please ...
2
votes
1answer
22 views

Why does this paper say that 0-1 loss is insensitive to scaling of weights in a neural network?

When discussing capacity control using norms of weights in a neural network,this paper says the following(see P4): Capacity control in terms of norm, when using a zero/one loss (i.e. counting ...
0
votes
0answers
13 views

Loss per batch for semi-supervised learning

I have a semi-supervised problem as follows: I only know ground-truth for batches of examples, e.g. for batch 1 with examples b1=(e1,e2,…) there should be at least one high value from the outputs o1=(...
1
vote
1answer
61 views

Keras Sequential model returns loss 'nan'

I'm implementing a neural network with Keras, but the Sequential model returns nan as loss value. I have sigmoid activation ...
1
vote
0answers
13 views

Differentiable loss function for ranking problem in regression model

In regression problem, we may need a loss function to measure the relative ranking accuracy between targets $y$ and predicted values $y_{pred}$. Abviously, the simple MSE does not consider such ...
0
votes
0answers
11 views

Initial loss for a multi-label prediction problem

I have seen this: Is there a rule of thumb for the initial value of loss function in a CNN? My question is: how can I know what my initial loss should be in, say, the following situation: 10 ...
0
votes
0answers
19 views

Use of Active contour loss funtion in U-net

I am using a custom loss function named 'Active contour loss'.(https://github.com/xuuuuuuchen/Active-Contour-Loss/blob/master/Active-Contour-Loss.py) I am getting an issue while segmentation. I have ...
1
vote
2answers
99 views

Loss being outputed as nan in keras RNN

Since the first Epoch of the RNN, the loss value is being outputted as nan. Epoch 1/100 9787/9787 [==============================] - 22s 2ms/step - loss: nan I have normalized the data. ...
0
votes
0answers
170 views

Tensorflow2 graph tensor leaking when using layer.output

I am trying to build a Contractive Auto Encoder using Tensorflow 2.0. The model's loss function uses the encoder output in its calculations. The problem is that every time i retrieve the output, it ...
2
votes
1answer
73 views

Interpreting Gradients and Partial Derivatives when training Neural Networks

I am trying to understand of purpose of partial differentiation in NN training by knowing how to interpret gradients and their partial derivatives. Below is my way of interpreting them so I would like ...
2
votes
1answer
120 views

Is it possible to make F1_Score differentiable and use it directly as a Loss function?

One of the metrics that is widely used in binary classification is the F1 score: $F_1 = 2\cdot \frac{recall \cdot precision}{recall+precision}$ The problem of the F1-score is that it is not ...
2
votes
1answer
43 views

batched CrossEntropyLoss in pytorch

I'm wondering how to implement this with pytorch built-ins. I've got a 3 dimensional input of uints called policy. Most of the entries are zero, and if I were to L1 normalize this I would have a (...
1
vote
0answers
86 views

Non differentiable loss function

I have a loss function that minimizes the error according to what I want the neural network to do. The problem is, that it is a nondifferentiable function. How can I handle this? the loss function: $(...
1
vote
1answer
118 views

Keras ResNet-50 not performing as expected

I am trying to build a neural network that is capable of classifying the make and model of a car. I am using the VMMR dataset to verify that the network is working, at which point I would like to ...
3
votes
0answers
65 views

Unable to transform (greatly performing) Autoencoder into Variational Autoencoder

Following the procedure described in this SO question, I am trying to transform my (greatly performing) convolutional Autoencoder into a Variational version of the same Autoencoder. As explained in ...
0
votes
2answers
25 views

How to assess that a cross entropy based model has converged

I have a question regarding cross entropy convergence using Stochastic Gradient Descent. I am a little bit confused about how the convergence should be assessed. Should we treat the model as converged ...
2
votes
0answers
35 views

Loss function for Autoencoder of sparse 3D Image

I have 3D structure data of molecules. I represented the atoms as points in a 100*100*100 grid and applied a gaussian blur to counter the sparseness. (nearly all of the grid cells contain zeros) I am ...
0
votes
0answers
17 views

Regression for Deskew Document problem

I am currently at an impasse regarding my regression problem. My goal is to generate a model that rotates correctly an image. My images are documents (invoices for example). Each document is either ...
1
vote
1answer
90 views

Overfitting model

I'm training two ResNet models on an image dataset. The first one has been trained with random weights, while the other has been pre-trained on ImageNet before. The second model starts overfitting ...
2
votes
1answer
49 views

What is the effect of KL divergence between two Gaussian distributions as a loss function in neural networks?

In many deep neural networks, especially those based on VAE architecture, a KL divergence term is added to the loss function. The divergence is computed between the estimated Gaussian distribution and ...
0
votes
0answers
10 views

May I please get to know which paper first proposed/mentioned to use Shannon's entropy in deep learning/machine learning in neural networks?

I have tried to investigate as to who had first mentioned the use of cross-entropy in deep learning -\sum p(X)\log q(X)
1
vote
0answers
39 views

Place of backward and its relation with batches in PyTorch

I am implementing a dependency parsing model using PyTorch and little bit confused about the situation that I explained below. When calculating loss and backward the model; I tried different things. ...
0
votes
0answers
42 views

Cross Entropy loss function performs far worse than Square Mean Error for Multi-class Classification

Relevant Information: I am building an opponent modeler for a poker application where the classification problem is predicting what move the opponent will make next. These moves are classified into ...
4
votes
2answers
117 views

Loss function for multivariate regression where relationship between outputs matters

I am attempting to build a sequential model with Keras (Tensorflow backend) that has multiple outputs. My targets are proportions of a whole so each observation is an array like [0.5, 0.25, 0.15, 0.1]....
0
votes
0answers
27 views

What will cause high accuracy but a big loss?

In the question of What is the relationship between the accuracy and the loss in deep learning?, @Jérémy Blain gave a fantastic interpretation of 'relationship' between accuracy and loss: 1 - low ...
1
vote
1answer
33 views

Dense output from neural network

I would like to create a loss function that encourages the output of the embedding of an autoencoder to be dense. I don't have an explicit condition for how density is defined, but one option would be ...
0
votes
2answers
24 views

Epochs or Loss Convergence

I was training a Deep Neural Network using transfer learning and the loss value as of now is close to 1.7. I am using a dataset of 31,000 images and I do apply data augmentation techniques to help the ...
0
votes
0answers
9 views

modified mean squared error with sign being taken into consideration

Suppose we have a regression model y_pred=f(X) with corresponding label y. Typical MSE is ...
1
vote
1answer
42 views

Creating a loss function with two minima?

I'm using keras to build a model. The model has one output, and I want a loss similar to mse loss, but there are two values to predict, and I'm fine if the model predicts either one of them, so I want ...
0
votes
1answer
13 views

Understanding AC_errorRate loss function

I'm reading an article about Rolling Window Regression: a Simple Approach for Time Series Next value Predictions. He explains about 5 different loss functions. I managed to understand the first four, ...
0
votes
0answers
35 views

How model loss is computed from multiple outputs when loss_weights are set to zero?

If I set loss_weights = 0 for all multiple outputs of a Keras model, how the loss function is computed? I am aware from Keras documentation about the definition: "...
1
vote
0answers
13 views

Can someone please explain Lovasz softmax loss? as its a bit difficult to understand why it works well from the original paper [duplicate]

Lovasz Softmax is used a lot these days for segmentation problem and the original paper is really bad at explaining why it works.
0
votes
1answer
19 views

How is loss computed for multiclass CNN with an output layer larger than the number of classes?

I have built a CNN in pytorch for classifying the Fashion-MNIST dataset (10 classes). The images are 28x28. I have constructed the final layer in my model as an output of 50. (i.e. $nn.Linear(...
1
vote
0answers
23 views

Balance two crossentropy losses with different number of neurons

I have a model with a few outputs, each output with shape: Shape: (batch_size, labels_1) -> softmax -> ...
0
votes
2answers
43 views

How to apply a different Loss function to one specific Label?

I got a recurrent neural network in Keras, which classifies on 14 labels. The first label is the most important one and should be predicted with the highest accuracy. The other labels don't have to be ...
0
votes
0answers
36 views

Customize loss function for Music Generation LSTM (?)

I have to carry out a Music Generation project for a Deep Learning course I have this semester and I am using Pytorch. The dataset is songs in midi format and I use the python library mido to extract ...
1
vote
1answer
22 views

why is MSE of prediction way different from loss over batches

I am new to machine learning so forgive me if i ask stupid question. I have a time series data and i split it into training and test set. This is my code: ...
0
votes
0answers
31 views

One part of my loss function overfits. How do I fix this?

I am working on an object detection problem where the final loss that is being optimized is the sum of an L2 loss (for the error in the predicted w, h values), and three binary cross entropy losses (...

1
2 3 4 5 6