Data Science

Questions tagged [loss-function]

Ask Question

A function used to quantify the difference between observed data and predicted values according to a model. Minimization of loss functions is a way to estimate the parameters of the model.

278 questions
Newest
Active
Bountied
Unanswered
Filter by
Sorted by
Tagged with
0
votes
1answer
10 views

Interpreting Gradients and Partial Derivatives when training Neural Networks

I am trying to understand of purpose of partial differentiation in NN training by knowing how to interpret gradients and their partial derivatives. Below is my way of interpreting them so I would like ...
1
vote
0answers
38 views

Is it possible to make F1_Score differentiable and use it directly as a Loss function?

One of the metrics that is widely used in binary classification is the F1 score: $F_1 = 2\cdot \frac{recall \cdot precision}{recall+precision}$ The problem of the F1-score is that it is not ...
2
votes
0answers
29 views

batched CrossEntropyLoss in pytorch

I'm wondering how to implement this with pytorch built-ins. I've got a 3 dimensional input of uints called policy. Most of the entries are zero, and if I were to L1 normalize this I would have a (...
1
vote
0answers
71 views

Non differentiable loss function

I have a loss function that minimizes the error according to what I want the neural network to do. The problem is, that it is a nondifferentiable function. How can I handle this? the loss function: $(...
0
votes
1answer
39 views

Keras ResNet-50 not performing as expected

I am trying to build a neural network that is capable of classifying the make and model of a car. I am using the VMMR dataset to verify that the network is working, at which point I would like to ...
3
votes
0answers
51 views

Unable to transform (greatly performing) Autoencoder into Variational Autoencoder

Following the procedure described in this SO question, I am trying to transform my (greatly performing) convolutional Autoencoder into a Variational version of the same Autoencoder. As explained in ...
0
votes
2answers
22 views

How to assess that a cross entropy based model has converged

I have a question regarding cross entropy convergence using Stochastic Gradient Descent. I am a little bit confused about how the convergence should be assessed. Should we treat the model as converged ...
2
votes
0answers
15 views

Loss function for Autoencoder of sparse 3D Image

I have 3D structure data of molecules. I represented the atoms as points in a 100*100*100 grid and applied a gaussian blur to counter the sparseness. (nearly all of the grid cells contain zeros) I am ...
0
votes
0answers
15 views

Regression for Deskew Document problem

I am currently at an impasse regarding my regression problem. My goal is to generate a model that rotates correctly an image. My images are documents (invoices for example). Each document is either ...
2
votes
1answer
70 views

Overfitting model

I'm training two ResNet models on an image dataset. The first one has been trained with random weights, while the other has been pre-trained on ImageNet before. The second model starts overfitting ...
2
votes
1answer
36 views

What is the effect of KL divergence between two Gaussian distributions as a loss function in neural networks?

In many deep neural networks, especially those based on VAE architecture, a KL divergence term is added to the loss function. The divergence is computed between the estimated Gaussian distribution and ...
0
votes
0answers
10 views

May I please get to know which paper first proposed/mentioned to use Shannon's entropy in deep learning/machine learning in neural networks?

I have tried to investigate as to who had first mentioned the use of cross-entropy in deep learning -\sum p(X)\log q(X)
1
vote
0answers
25 views

Place of backward and its relation with batches in PyTorch

I am implementing a dependency parsing model using PyTorch and little bit confused about the situation that I explained below. When calculating loss and backward the model; I tried different things. ...
0
votes
0answers
38 views

Cross Entropy loss function performs far worse than Square Mean Error for Multi-class Classification

Relevant Information: I am building an opponent modeler for a poker application where the classification problem is predicting what move the opponent will make next. These moves are classified into ...
5
votes
2answers
60 views

Loss function for multivariate regression where relationship between outputs matters

I am attempting to build a sequential model with Keras (Tensorflow backend) that has multiple outputs. My targets are proportions of a whole so each observation is an array like [0.5, 0.25, 0.15, 0.1]....
0
votes
0answers
26 views

What will cause high accuracy but a big loss?

In the question of What is the relationship between the accuracy and the loss in deep learning?, @Jérémy Blain gave a fantastic interpretation of 'relationship' between accuracy and loss: 1 - low ...
1
vote
1answer
33 views

Dense output from neural network

I would like to create a loss function that encourages the output of the embedding of an autoencoder to be dense. I don't have an explicit condition for how density is defined, but one option would be ...
0
votes
2answers
16 views

Epochs or Loss Convergence

I was training a Deep Neural Network using transfer learning and the loss value as of now is close to 1.7. I am using a dataset of 31,000 images and I do apply data augmentation techniques to help the ...
0
votes
0answers
9 views

modified mean squared error with sign being taken into consideration

Suppose we have a regression model y_pred=f(X) with corresponding label y. Typical MSE is ...
1
vote
1answer
30 views

Creating a loss function with two minima?

I'm using keras to build a model. The model has one output, and I want a loss similar to mse loss, but there are two values to predict, and I'm fine if the model predicts either one of them, so I want ...
0
votes
1answer
11 views

Understanding AC_errorRate loss function

I'm reading an article about Rolling Window Regression: a Simple Approach for Time Series Next value Predictions. He explains about 5 different loss functions. I managed to understand the first four, ...
0
votes
0answers
19 views

How model loss is computed from multiple outputs when loss_weights are set to zero?

If I set loss_weights = 0 for all multiple outputs of a Keras model, how the loss function is computed? I am aware from Keras documentation about the definition: "...
1
vote
0answers
13 views

Can someone please explain Lovasz softmax loss? as its a bit difficult to understand why it works well from the original paper [duplicate]

Lovasz Softmax is used a lot these days for segmentation problem and the original paper is really bad at explaining why it works.
0
votes
1answer
16 views

How is loss computed for multiclass CNN with an output layer larger than the number of classes?

I have built a CNN in pytorch for classifying the Fashion-MNIST dataset (10 classes). The images are 28x28. I have constructed the final layer in my model as an output of 50. (i.e. $nn.Linear(...
1
vote
0answers
18 views

Balance two crossentropy losses with different number of neurons

I have a model with a few outputs, each output with shape: Shape: (batch_size, labels_1) -> softmax -> ...
0
votes
2answers
40 views

How to apply a different Loss function to one specific Label?

I got a recurrent neural network in Keras, which classifies on 14 labels. The first label is the most important one and should be predicted with the highest accuracy. The other labels don't have to be ...
0
votes
0answers
28 views

Customize loss function for Music Generation LSTM (?)

I have to carry out a Music Generation project for a Deep Learning course I have this semester and I am using Pytorch. The dataset is songs in midi format and I use the python library mido to extract ...
1
vote
1answer
19 views

why is MSE of prediction way different from loss over batches

I am new to machine learning so forgive me if i ask stupid question. I have a time series data and i split it into training and test set. This is my code: ...
0
votes
0answers
30 views

One part of my loss function overfits. How do I fix this?

I am working on an object detection problem where the final loss that is being optimized is the sum of an L2 loss (for the error in the predicted w, h values), and three binary cross entropy losses (...
0
votes
0answers
14 views

Is using cross-entropy enough to ensure the output is a distribution probability?

I am following along https://pytorch.org/tutorials/beginner/finetuning_torchvision_models_tutorial.html. In this code, the last layers of the pretrained networks are linear. The loss used in this ...
1
vote
0answers
24 views

Unsupervised Function Optimization using Input and Output for Loss Function?

I have some vectors {$\mathbf{X_1 ... X_n}$} and they are all of dimension 1 x N. Vectors {$\mathbf{X_1' ... X_n'}$} are also 1 x N and are related to {$\mathbf{X_1 ... X_n}$}, but the relation cannot ...
0
votes
0answers
70 views

Statistical loss function for categorical distributions

For training an autoencoder model whose outputs (and inputs) are parameters from a categorical distribution $[q_1, q_2, \ldots, q_n]$, I have to define a proper loss function measuring the distance ...
2
votes
1answer
266 views

Keras - Implementation of custom loss function with multiple outputs

I am trying to replicate (a way smaller version) of the AlphaGo Zero system. However, in the network model, I am having a problem. The loss function I am supposed to implement is the following: $$l = ...
0
votes
1answer
26 views

Help making a custom categorical loss function in Keras

I am a bit new to machine learning, and I'm trying to get the basics working towards a bigger project using a very simple encoder-decoder model. It looks like this: ...
1
vote
1answer
84 views

AlphaGo Zero loss function

As far as I understood from the AlphaGo Zero system: During the self-play part, the MCTS algorithm stores a tuple ($s$, $\pi$, $z$) where $s$ is the state, $\pi$ is the distribution probability over ...
0
votes
0answers
161 views

Derivative of logarithm of loss function. Logistic regression

I am reading machine learning literature. I found the log-loss function of logistic regression algorithm: $$ l(w) = \sum_{n=0}^{N-1}\ln(1+e^{-y_nw^Tx_n}) $$ Where $ y \in {-1;1}, w \in R^P, x_n \in R^...
1
vote
2answers
284 views

what is the difference between euclidean distance and RMSE?

I'm searching for a loss function that fits my Project. Actually I have two question but they are in the same direction. I take a look at the definition of the root mean squared error and the ...
0
votes
0answers
22 views

Should the LightGBM score match the regularization?

If I set the parameter objective to regression_l1 and set the metric to mean absolute error in ...
0
votes
0answers
12 views

Categorical loss functions with similar properties to Kullback-Leibler loss function

When using the Kullback-Leibler divergence as loss function for predicting the probabilities of a categorical (multinomial) distribution, one of the properties is that the difference between $a$ and $...
2
votes
2answers
67 views

Classifying points exactly on decision boundary

For calculating loss arose by classification we do this: If $y (w \cdot x + b) > 0$: $\text{no loss}$ If $y (w \cdot x + b) < 0$: $\text{loss} = −y (w \cdot x + b)$ So here what about the ...
0
votes
0answers
5 views

Implementation of the paper 'A Comprehensive Study for Center Loss'

I have studied the research paper A Comprehensive Study for Center Loss. The implementation in Caffe also exists in this github repo. In the paper, the author talks about a generalized implementation ...
0
votes
0answers
19 views

What kind of loss function should be used for a problem like this?

My dataset consists of hierarchical timeseries. One could imagine it as "total sales" and segmentation per product. Something like this: ...
0
votes
0answers
28 views

Multi-class classification with custom loss matrix?

Suppose I have classes A,B,C and some predictors. I want to minimize the loss function where the loss penalties are arbitrary penalties applied to each possible misclassification e.g.: $$L = \begin{...
0
votes
0answers
13 views

binary_crossentropy and sigmoid with InceptionV3 gives pred [[1.]] for all test inputs. How to interpret this result?

Keras 2.1.2 with Tensorflow backend. The train.py looks like below: ...
1
vote
0answers
21 views

Keras Custom Loss Function

I am looking to design a custom loss function for Keras model. The model itself is neural network that accepts a set of images and is supposed to run a regression to get an output, which is a value. ...
0
votes
0answers
38 views

sess.run in losses.py

now I want to use sess.run in losses.py so that complex codes can be written by python style for customizing error function as below; ...
0
votes
0answers
24 views

Why don't we use BCE(Binary Cross Entropy) for language modeling?

I've seen a lot of RNN/Transformer models use cross entropy loss with softmax. but isn't language modeling a multilabel classification task? what happens if we replace cross entropy loss with binary ...
0
votes
0answers
50 views

For loop and assignment procedure in losses.py for customizing loss function of ML model

I want to customize a loss function for neural networks. A computation of the loss function requires the inclusion of a for loop like below. However, it is super difficult to implement because ...
0
votes
0answers
65 views

Custom Loss function - An operation has `None` for gradient

i want to write my own loss function to train a GAN in Keras. The Generator should learn to write word images. Therefore i use a Discriminator and a Text Recognition. The Generator should learn from ...

1 2 3 4 5 6