Questions tagged [loss-function]
The loss-function tag has no usage guidance.
0
votes
0answers
6 views
What is the difference between SSIM and MS-SSIM?
I would like to know what is the difference between SSIM and MS-SSIM? Also, there is a built-in function in Tensorflow for both of them, I am curious to know when should I use SSIM and when MS-SSIM?
...
0
votes
0answers
23 views
Continuous Function Input
If a continuous function has trouble differentiating between small close numbers (i.e. 0.1 and 0.001) why can we train and learn such small numbers in something like word2vec?
0
votes
0answers
25 views
Is empirical risk the same thing as loss function?
I am reading the article Stochastic Gradient Descent Tricks by Léon Bottou (avaible here) and on the very first page they introduce empirical risk
$E_n(f) = \frac{1}{n} \sum_{i=1}^{n} l(f(x_i),y_i),$
...
0
votes
0answers
23 views
Intention behind of weight loss function
I am curious to know the intention behind of weight loss functions. In some model, loss function multiply by a small number or sometimes subtract 1 from loss function of the model. I want to know the ...
0
votes
1answer
25 views
Which Loss function is correct for binary mapping?
I have built a 3 layer neural network to perform a binary mapping (2016 inputs, 288 outputs.) I am getting decent results with mean square error and stochastic gradient decent. My question is: Is ...
1
vote
0answers
9 views
Strangeness in validation loss between CPU vs GPU when training CNN
I've been training an implementation of Mask R-CNN and it was training very successfully on my CPU but I've just set up my GPU and it is giving some strange results when looking at my validation loss. ...
0
votes
0answers
17 views
Backpropagation through LSTM and MLP layers
For didactic reason, I am currently implementing in numpy an LSTM network for classifications. I need to add on top of the LSTM another fully connected layer, because I don't want the output to have ...
0
votes
0answers
32 views
Triplet loss training problem
My results are very poor and I cannot make out the reason on why is it so?
I am using euclidean distance measure for hard mining of triplets. It is prior to training with the initial random set of ...
1
vote
0answers
19 views
Unbalanced multi-label multi-class classification
What are common approaches in order to deal with unbalanced multi-label multi-class classification problems in deep learning? Furthermore there is correlation between the labels. I tried two ...
0
votes
1answer
25 views
Classification loss function: how to implement individual weights for each observation and class
The problem I have to solve is a classification problem. The costs of a misclassification are very different (but known) for the various observations, so I plan to include them by assigning weights to ...
0
votes
0answers
12 views
Multi-Task Loss: Bischke, et al
I'm trying to understand the multi-task loss function from paper Bischke, et al. Specifically, I'm stumped at Equation 8 and how this is actually computed given ...
1
vote
1answer
137 views
custom loss in keras, problem with batch size
I am trying to create a custom loss function,custom_loss(y_true, y_pred). I understand that y_pred is calculated by my model but ...
0
votes
1answer
23 views
Enable to reproduce the loss of training while predicting
i use CNN model for a regression problem with a custom loss
...
1
vote
1answer
49 views
What is non-decomposable and/or non-differentiable loss function?
I have been reading some deep learning literature and came up with these concepts of non-decomposable and non-differentiable loss functions. My question is are these same thing? if not how are they ...
1
vote
1answer
228 views
Sparse_categorical_crossentropy vs categorical_crossentropy (keras, accuracy)
Which is better for accuracy or are they the same?
Of course, if you use categorical_crossentropy you use one hot encoding, and if you use sparse_categorical_crossentropy you encode as normal integers....
1
vote
1answer
30 views
Loss is bad, but accuracy increases?
I have a multicategorial classification problem for images. There are 5 (imbalanced) classes for which i use different class weights. In general there are only a few training images per class: ~56-238
...
2
votes
0answers
88 views
Custom Loss Function on a Keras Neural Network
I'm training a Neural Network on Keras to predict class as a triplet of the form S,P,T, where S, ...
0
votes
1answer
37 views
What loss function avoids overconfidence?
In the case of a neural net with a relatively small training data set, doing simple classification with categorical cross entropy (log loss), it is very easy for the results of the network to be "...
1
vote
0answers
17 views
issue with early-stopping on f1 score with imbalanced data
I have a highly imbalanced dataset with less than 0.5% of the minor class. Using Keras, I'm training DNN on the training set and evaluate performance on validation set. Loss function is ...
-1
votes
0answers
26 views
Why my val loss is fluctuating?
I am using resnet50 with preloaded weights of imagines. set the learning rate as 2e-2, which I figured from learner.lr_find. Any suggestions
on why its fluctuating?
[![run ouput][1]][1]
[1]: https:/...
0
votes
0answers
8 views
High loss value with good classification result
I have a dataset which contains news articles (the articles are long, where for each record I have about 1500 words). During the training of my lstm network, I noticed when I get the best macro F1 ...
1
vote
0answers
61 views
Using SMAPE as a loss function for an LSTM
I am currently working on a time series forecasting problem and am looking into using an LSTM.
My final accuracy metric that I use to determine whether or not the forecast is good or not is defined ...
0
votes
1answer
90 views
Should the minimum value of a cost (loss) function be equal to zero?
We know optimization techniques search in the space of all
the possible parameters for a parameter set that minimizes the cost function of the model. The most well-known loss functions, like MSE or ...
0
votes
1answer
102 views
What is the advantage of using log softmax instead of softmax
i am wondering if there are any advantages of log softmax over softmax. And also, when i should use softmax or log-softmax. is there any specific reason for choosing one over another?
0
votes
3answers
249 views
What Base Should Be Used For Negative Log Likelihood?
When calculating the negative log likelihood loss, what base of log are we supposed to use?
2
votes
1answer
24 views
“help” decision tree by tying 2 features together
Assuming I have in my dataset 2 (or more) features that are for sure linked (for example: feature B indicates the amount of relevance of feature A), is there a way I could design a decision tree that ...
1
vote
0answers
19 views
Should I rescale losses before combining them for multitask learning?
I have a multitask network taking one input and trying to achieve two tasks (with several shared layers, and then separate layers).
One task is multiclass classification using the CrossEntropy loss, ...
1
vote
0answers
31 views
Tuning neural network loss function for space physics [closed]
I am trying to work on neural network classification (with python, Keras) for space physics purposes, where I want to identify specific planetary regions based on multivariate temporal data and multi ...
0
votes
1answer
30 views
Validation loss differs on GPU vs CPU
I am consistently seeing higher validation loss when I train & evaluate a model on AWS GPU vs local CPU.
I am using the exact same train/eval datasets and the exact same Tensorflow code and ...
0
votes
1answer
212 views
Validation loss is lower than the training loss
I am using autoencoder for anomaly detection in warranty data.
Architecture 1:
The plot shows the training vs validation loss based on Architecture 1.
As we see in the plot, validation loss is ...
0
votes
0answers
35 views
Which loss and activation function at output layer is suitable for Multi target classification problem?
I am modelling a Multi-target classification problem which has 220 input features and 132 output features. Each output target has an integer value in between [0,1,2,3,4] .And for this I have applied ...
0
votes
0answers
97 views
Calculating gradient and hessian for a custom loss function to use in xgboost
I want to use a cost function which rewards true positives, true negatives, and penalizes false positives and false negatives differently.
Something like the one below
...
4
votes
2answers
80 views
Why Gradient methods work in finding the parameters in Neural Networks?
After reading quite a lot of papers (20-30 or so), I feel that I am quite not understanding things.
Let us focus on the supervised learnings (for example).
Given a set of data $\mathcal{D}_{train}=\{...
0
votes
1answer
58 views
Loss function when the output is a single probability
I have a regression problem where the output y is a single probability, i.e. real number that varies in the interval [0, 1]
...
1
vote
2answers
88 views
What are the differences between logistic and linear regression?
I know that linear regression does "regression" and logistic regression does "classification". When we implement these two methods, the only difference I could notice is the loss function: linear ...
2
votes
1answer
99 views
Constant validation loss & accuracy, training accuracy fluctuates
I am training a Squeeze-net model for binary classification of images. I have 79968 images for training (50:50 for and against) and 8892 images in the validation set. After 35000 iterations my ...
1
vote
1answer
896 views
What is the difference between SGD classifier and the Logisitc regression?
To my understanding, the SGD classifier, and Logistic regression seems similar.
An SGD classifier with loss = 'log' implements Logistic regression and loss = 'hinge' implements Linear SVM. I also ...
2
votes
1answer
113 views
Purpose of backpropagation in neural networks
I've just finished conceptually studying linear and logistic regression functions and their optimization as preparation for neural networks.
For example, say we are performing binary classification ...
0
votes
0answers
34 views
Train the model to increase accuracy rather than to minimise loss
I am currently in a situation of seq2seq training where the cross entropy loss is very low (near zero) but the accuracy is also very low. This made me wondering if there were any loss functions that ...
1
vote
1answer
42 views
Validation loss
I am having trouble wrapping my brain around validation loss. It's my understanding that loss is calculated at the end of the feed forward in a NeuralNet and is used in back propagation to update the ...
1
vote
0answers
188 views
Why is my loss function for DQN converging too quickly?
I'm still relatively new to deep learning and am experiencing an issue that I can't seem to find a solution/explanation for. I've developed a DQN model in tensorflow, as described by DeepMind, and am ...
2
votes
0answers
25 views
4
votes
1answer
778 views
Interpreting the Root Mean Squared Error (RMSE)!
I read all about pros and cons of RMSE vs. other absolute errors namely mean absolute error (MAE). See the the following references:
MAE and RMSE — Which Metric is Better?
What's the bottom line? How ...
2
votes
1answer
31 views
Why are optimization algorithms slower at critical points?
I just found the animation below from Alec Radford's presentation:
As visible, all algorithms are considerably slowed down at saddle point (where derivative is 0) and quicken up once they get out of ...
2
votes
0answers
104 views
Validation loss keeps fluctuating about training loss
I am training a Keras model for multi-target regression by using a custom loss function with the goal of getting predictions accurate to below 0.01 with respect to ...
0
votes
1answer
27 views
Differentiating roadmap of a loss function
Let's say I'm performing Stochastic Gradient Descent (SGD) on binary cross entropy error while optimizing weight $w_{2}$.
Binary cross entropy error:
$$L(y|p(x_{i}))=-y_{i}*ln(p(x_{i}))-(1-y_{i})*...
1
vote
0answers
78 views
Deep RL: Proximal policy optimization gradient calculation
Case:
Continuous action domain with 4 outputs (control problem)
Policy and Value function approximation with fully connected neural networks
I understand that the loss function for PPO for the ...
-1
votes
1answer
179 views
What's the difference between Error, Risk and Loss?
When we talk about 'Minimizing Loss', we often talk about loss functions such as Mean Squared Error (MSE); the term 'Empirical Risk Minimization' is often used interchangeably. So what's the ...
0
votes
0answers
63 views
Keras update Layers which are only Part of the Loss Function
I'm implementing a variational inference model. There are two main structures a inference network and a generative model. The first part is only used for training and only appears in the loss function....
1
vote
1answer
839 views
focal loss function help
I am working on a relation extraction and classification problem. The data is in the form of text files. The data is imbalanced. I want to use focal loss function to address class imbalance problem in ...
