Questions tagged [loss-function]
A function used to quantify the difference between observed data and predicted values according to a model. Minimization of loss functions is a way to estimate the parameters of the model.
257
questions
0
votes
0answers
10 views
How model loss is computed from multiple outputs when loss_weights are set to zero?
If I set loss_weights = 0 for all multiple outputs of a Keras model, how the loss function is computed? I am aware from Keras documentation about the definition: "...
1
vote
0answers
13 views
Can someone please explain Lovasz softmax loss? as its a bit difficult to understand why it works well from the original paper [duplicate]
Lovasz Softmax is used a lot these days for segmentation problem and the original paper is really bad at explaining why it works.
0
votes
1answer
12 views
How is loss computed for multiclass CNN with an output layer larger than the number of classes?
I have built a CNN in pytorch for classifying the Fashion-MNIST dataset (10 classes). The images are 28x28.
I have constructed the final layer in my model as an output of 50. (i.e. $nn.Linear(...
1
vote
0answers
13 views
Balance two crossentropy losses with different number of neurons
I have a model with a few outputs, each output with shape:
Shape: (batch_size, labels_1) -> softmax -> ...
0
votes
1answer
25 views
How to apply a different Loss function to one specific Label?
I got a recurrent neural network in Keras, which classifies on 14 labels.
The first label is the most important one and should be predicted with the highest accuracy. The other labels don't have to be ...
0
votes
0answers
19 views
Customize loss function for Music Generation LSTM (?)
I have to carry out a Music Generation project for a Deep Learning course I have this semester and I am using Pytorch. The dataset is songs in midi format and I use the python library mido to extract ...
1
vote
1answer
16 views
why is MSE of prediction way different from loss over batches
I am new to machine learning so forgive me if i ask stupid question. I have a time series data and i split it into training and test set.
This is my code:
...
0
votes
0answers
28 views
One part of my loss function overfits. How do I fix this?
I am working on an object detection problem where the final loss that is being optimized is the sum of an L2 loss (for the error in the predicted w, h values), and three binary cross entropy losses (...
0
votes
0answers
10 views
Is using cross-entropy enough to ensure the output is a distribution probability?
I am following along https://pytorch.org/tutorials/beginner/finetuning_torchvision_models_tutorial.html.
In this code, the last layers of the pretrained networks are linear. The loss used in this ...
-1
votes
0answers
19 views
Which is the best automatic differentiation tool to write multi-layered neural networks in c++? [closed]
I'm triyng to write a custom multi-layered neural network with a loss that takes in consideration correlations between patterns:
https://datascience.stackexchange.com/questions/62012/a-generalized-...
1
vote
0answers
24 views
Unsupervised Function Optimization using Input and Output for Loss Function?
I have some vectors {$\mathbf{X_1 ... X_n}$} and they are all of dimension 1 x N. Vectors {$\mathbf{X_1' ... X_n'}$} are also 1 x N and are related to {$\mathbf{X_1 ... X_n}$}, but the relation cannot ...
0
votes
0answers
19 views
Statistical loss function for categorical distributions
For training an autoencoder model whose outputs (and inputs) are parameters from a categorical distribution $[q_1, q_2, \ldots, q_n]$, I have to define a proper loss function measuring the distance ...
2
votes
0answers
59 views
Keras - Implementation of custom loss function with multiple outputs
I am trying to replicate (a way smaller version) of the AlphaGo Zero system. However, in the network model, I am having a problem. The loss function I am supposed to implement is the following:
$$l = ...
0
votes
1answer
18 views
Help making a custom categorical loss function in Keras
I am a bit new to machine learning, and I'm trying to get the basics working towards a bigger project using a very simple encoder-decoder model. It looks like this:
...
1
vote
1answer
71 views
AlphaGo Zero loss function
As far as I understood from the AlphaGo Zero system:
During the self-play part, the MCTS algorithm stores a tuple ($s$, $\pi$, $z$) where $s$ is the state, $\pi$ is the distribution probability over ...
0
votes
0answers
22 views
Derivative of logarithm of loss function. Logistic regression
I am reading machine learning literature. I found the log-loss function of logistic regression algorithm:
$$
l(w) = \sum_{n=0}^{N-1}\ln(1+e^{-y_nw^Tx_n})
$$
Where $ y \in {-1;1}, w \in R^P, x_n \in R^...
0
votes
2answers
70 views
what is the difference between euclidean distance and RMSE?
I'm searching for a loss function that fits my Project. Actually I have two question but they are in the same direction. I take a look at the definition of the root mean squared error and the ...
0
votes
0answers
17 views
Should the LightGBM score match the regularization?
If I set the parameter objective to regression_l1 and set the metric to mean absolute error in ...
0
votes
0answers
10 views
Categorical loss functions with similar properties to Kullback-Leibler loss function
When using the Kullback-Leibler divergence as loss function for predicting the probabilities of a categorical (multinomial) distribution, one of the properties is that the difference between $a$ and $...
2
votes
2answers
64 views
Classifying points exactly on decision boundary
For calculating loss arose by classification we do this:
If $y (w \cdot x + b) > 0$: $\text{no loss}$
If $y (w \cdot x + b) < 0$: $\text{loss} = −y (w \cdot x + b)$
So here what about the ...
0
votes
0answers
5 views
Implementation of the paper 'A Comprehensive Study for Center Loss'
I have studied the research paper A Comprehensive Study for Center Loss. The implementation in Caffe also exists in this github repo. In the paper, the author talks about a generalized implementation ...
0
votes
0answers
19 views
What kind of loss function should be used for a problem like this?
My dataset consists of hierarchical timeseries. One could imagine it as "total sales" and segmentation per product. Something like this:
...
0
votes
0answers
26 views
Multi-class classification with custom loss matrix?
Suppose I have classes A,B,C and some predictors.
I want to minimize the loss function where the loss penalties are arbitrary penalties applied to each possible misclassification e.g.:
$$L = \begin{...
0
votes
0answers
12 views
binary_crossentropy and sigmoid with InceptionV3 gives pred [[1.]] for all test inputs. How to interpret this result?
Keras 2.1.2 with Tensorflow backend.
The train.py looks like below:
...
1
vote
0answers
18 views
Keras Custom Loss Function
I am looking to design a custom loss function for Keras model. The model itself is neural network that accepts a set of images and is supposed to run a regression to get an output, which is a value. ...
0
votes
0answers
35 views
sess.run in losses.py
now I want to use sess.run in losses.py so that complex codes can be written by python style for customizing error function as below;
...
0
votes
0answers
21 views
Why don't we use BCE(Binary Cross Entropy) for language modeling?
I've seen a lot of RNN/Transformer models use cross entropy loss with softmax. but isn't language modeling a multilabel classification task? what happens if we replace cross entropy loss with binary ...
0
votes
0answers
50 views
For loop and assignment procedure in losses.py for customizing loss function of ML model
I want to customize a loss function for neural networks.
A computation of the loss function requires the inclusion of a for loop like below. However, it is super difficult to implement because ...
0
votes
0answers
41 views
Custom Loss function - An operation has `None` for gradient
i want to write my own loss function to train a GAN in Keras.
The Generator should learn to write word images. Therefore i use a Discriminator and a Text Recognition. The Generator should learn from ...
0
votes
0answers
13 views
Conditional loss for bounding boxes in Tensorflow
I want to build a classifier plus localizer in Tensorflow. I wonder how I can implement a loss function with a condition that checks the accuracy of the bounding box only of the object is detected.
...
1
vote
1answer
38 views
0
votes
0answers
16 views
How to optimize neural network with mutile losses
There is a multi-task problem that I try to solve with a single neutral network. In general it works fine, but it seems like there is a room for improvement.
The final loss to optimize looks like ...
0
votes
1answer
28 views
The loss and accuracy of this LSTM both drop to nearly 0 at the same epoch
I'm trying to train an LSTM to predict the the Nth token using the N-1 tokens preceding it
For each One-Hot encoded token, I ...
0
votes
0answers
10 views
Why is Dice loss neglecting random classes?
I implemented Dice loss for a semantic segmentation problem (with a severe class imbalance in my dataset) as follows:
...
2
votes
1answer
41 views
What is auxiliary loss in Character-level Transformer model?
I am reading Character-Level Language Modeling with Deeper Self-Attention from Rami Al-Rfou. In the second page, they had mentioned about Auxiliary Losses which can speed-up the model convergence and ...
0
votes
0answers
16 views
Combining multiple loss functions in the right way
I am trying to build an (variational) Autoencoder to generate fake but representative data from a generic data set with a couple of numeric and categorical columns.
So far I have built functions that ...
0
votes
0answers
26 views
Plotting an loss/cost function wrt parameters
What would be the most sensible way to plot, for example the cross-entropy loss function (or the quadratic error function) for a binary classification algorithm (e.g. in this case logistic regression) ...
0
votes
0answers
7 views
Derive main prediction for zero-one loss function
Intuitively I can see why mode of predictions is the main prediction of a zero-one loss function, but mathematically I am not sure how it is derived?
Main prediction $= argmin_{y'}E_D[L(\widehat y, y'...
0
votes
0answers
55 views
Cross entropy loss for sigmoid function
Suppose instead of squared error loss I take cross entropy loss:
$$H(y) = - \sum y' \log(y) $$ ( where y' is the actual distribution)
. I read somewhere that this loss function converges faster ...
0
votes
0answers
120 views
Interpreting Gensim Word2Vec Training Loss
I am using Gensim to build a Word2Vec model and identify the convergence of training loss, so that I can figure out the optimal number of iterations. For understanding this since gensim's ...
0
votes
1answer
35 views
Gradient Descent - how many values are calculated in loss function?
I'm a little bit confused how loss function is calculated in neural network training. There's is said that in theory when using Grid Search or Monte Carlo methods we can calculate all the possible ...
0
votes
1answer
22 views
Explanation behind the calculation of training loss in deep learning model
I am trying to model an image classification problem using convolution neural network. I came across a code on Github in which I am not able to understand the meaning of following line for loss ...
1
vote
1answer
24 views
Metric/loss for bin classification
I have a model that has to classify inputs into one of 45 categories but those categories actually represent bins (e.g. bins 1, 2 and 3 are between 1 and 10, 11 and 20, 21 and 30 respectively). What I ...
0
votes
0answers
24 views
Difference between binary cross_entropy and mse
model = Sequential()
model.add(Dense(16, activation="selu"))
model.add(Dense(1, activation="sigmoid"))
I am using Keras to create a binary classifier.
My data ...
0
votes
1answer
42 views
sklearn: sklearn.linear_model.HuberRegressor vs sklearn.linear_model.ElasticNet
I am experimenting different loss functions for my regression model. I noticed that in the sklearn, there are:
sklearn.linear_model.HuberRegressor and sklearn.linear_model.ElasticNet
To me, both use ...
0
votes
0answers
11 views
Nan in target variables Neural Network
Is it possible to train on a dataset with some nan in the target variables?
I imagine a sort of loss calculation only for the given target data.
Is this Doable in Tensorflow/Keras =?
1
vote
1answer
116 views
Why $L2$ loss is strictly convex if number of samples $N$ is larger than input dimension $d$?
I am using $L2$ loss in my linear regression problem and I have to prove that my $L2$ loss is strictly convex if number of samples $N$ is larger than input dimension $d$.
I think, if I can prove ...
0
votes
1answer
183 views
Can we use Binary Cross Entropy for Multiclass Classification?
In this link, the author has implemented a CNN which classifies 15 classes and has used Binary Cross Entropy as the loss function. But since it's multiclass classification, is it valid to use Binary ...
0
votes
2answers
135 views
What can be the cause of a sudden explosion in the loss when training a CNN (Deeplab)
I am training the following deeplab CNN:
https://github.com/tensorflow/models/tree/master/research/deeplab
During training I see the following loss:
The first 50k steps of the training the loss is ...
0
votes
0answers
14 views
Hierarchical prediction
Suppose the problem is the following: there is, say, binary target variable $x$, and real-valued target variable $y$, which is only relevant if $x = 1$. What is the best way to train a model to ...
