Questions tagged [loss-function]
The loss-function tag has no usage guidance.
0
votes
0answers
2 views
issue with early-stopping on f1 score with imbalanced data
I have a highly imbalanced dataset with less than 0.5% of the minor class. Using Keras, I'm training DNN on the training set and evaluate performance on validation set. Loss function is ...
0
votes
0answers
19 views
Why my val loss is fluctuating?
I am using resnet50 with preloaded weights of imagines. set the learning rate as 2e-2, which I figured from learner.lr_find. Any suggestions
on why its fluctuating?
[![run ouput][1]][1]
[1]: https:/...
0
votes
0answers
4 views
High loss value with good classification result
I have a dataset which contains news articles (the articles are long, where for each record I have about 1500 words). During the training of my lstm network, I noticed when I get the best macro F1 ...
0
votes
0answers
13 views
Using SMAPE as a loss function for an LSTM
I am currently working on a time series forecasting problem and am looking into using an LSTM.
My final accuracy metric that I use to determine whether or not the forecast is good or not is defined ...
0
votes
1answer
39 views
Should the minimum value of a cost (loss) function be equal to zero?
We know optimization techniques search in the space of all
the possible parameters for a parameter set that minimizes the cost function of the model. The most well-known loss functions, like MSE or ...
0
votes
1answer
26 views
What is the advantage of using log softmax instead of softmax
i am wondering if there are any advantages of log softmax over softmax. And also, when i should use softmax or log-softmax. is there any specific reason for choosing one over another?
0
votes
3answers
226 views
What Base Should Be Used For Negative Log Likelihood?
When calculating the negative log likelihood loss, what base of log are we supposed to use?
1
vote
1answer
17 views
“help” decision tree by tying 2 features together
Assuming I have in my dataset 2 (or more) features that are for sure linked (for example: feature B indicates the amount of relevance of feature A), is there a way I could design a decision tree that ...
1
vote
0answers
11 views
Should I rescale losses before combining them for multitask learning?
I have a multitask network taking one input and trying to achieve two tasks (with several shared layers, and then separate layers).
One task is multiclass classification using the CrossEntropy loss, ...
1
vote
0answers
21 views
Tuning neural network loss function for space physics [closed]
I am trying to work on neural network classification (with python, Keras) for space physics purposes, where I want to identify specific planetary regions based on multivariate temporal data and multi ...
0
votes
1answer
21 views
Validation loss differs on GPU vs CPU
I am consistently seeing higher validation loss when I train & evaluate a model on AWS GPU vs local CPU.
I am using the exact same train/eval datasets and the exact same Tensorflow code and ...
0
votes
1answer
61 views
Validation loss is lower than the training loss
I am using autoencoder for anomaly detection in warranty data.
Architecture 1:
The plot shows the training vs validation loss based on Architecture 1.
As we see in the plot, validation loss is ...
0
votes
0answers
22 views
Which loss and activation function at output layer is suitable for Multi target classification problem?
I am modelling a Multi-target classification problem which has 220 input features and 132 output features. Each output target has an integer value in between [0,1,2,3,4] .And for this I have applied ...
0
votes
0answers
32 views
Calculating gradient and hessian for a custom loss function to use in xgboost
I want to use a cost function which rewards true positives, true negatives, and penalizes false positives and false negatives differently.
Something like the one below
...
4
votes
2answers
70 views
Why Gradient methods work in finding the parameters in Neural Networks?
After reading quite a lot of papers (20-30 or so), I feel that I am quite not understanding things.
Let us focus on the supervised learnings (for example).
Given a set of data $\mathcal{D}_{train}=\{...
0
votes
1answer
31 views
Loss function when the output is a single probability
I have a regression problem where the output y is a single probability, i.e. real number that varies in the interval [0, 1]
...
1
vote
2answers
80 views
What are the differences between logistic and linear regression?
I know that linear regression does "regression" and logistic regression does "classification". When we implement these two methods, the only difference I could notice is the loss function: linear ...
2
votes
1answer
36 views
Constant validation loss & accuracy, training accuracy fluctuates
I am training a Squeeze-net model for binary classification of images. I have 79968 images for training (50:50 for and against) and 8892 images in the validation set. After 35000 iterations my ...
1
vote
1answer
240 views
What is the difference between SGD classifier and the Logisitc regression?
To my understanding, the SGD classifier, and Logistic regression seems similar.
An SGD classifier with loss = 'log' implements Logistic regression and loss = 'hinge' implements Linear SVM. I also ...
1
vote
1answer
81 views
Purpose of backpropagation in neural networks
I've just finished conceptually studying linear and logistic regression functions and their optimization as preparation for neural networks.
For example, say we are performing binary classification ...
0
votes
0answers
27 views
Train the model to increase accuracy rather than to minimise loss
I am currently in a situation of seq2seq training where the cross entropy loss is very low (near zero) but the accuracy is also very low. This made me wondering if there were any loss functions that ...
1
vote
1answer
31 views
Validation loss
I am having trouble wrapping my brain around validation loss. It's my understanding that loss is calculated at the end of the feed forward in a NeuralNet and is used in back propagation to update the ...
1
vote
0answers
94 views
Why is my loss function for DQN converging too quickly?
I'm still relatively new to deep learning and am experiencing an issue that I can't seem to find a solution/explanation for. I've developed a DQN model in tensorflow, as described by DeepMind, and am ...
2
votes
0answers
22 views
3
votes
1answer
208 views
Interpreting the Root Mean Squared Error (RMSE)!
I real all about pros and cons of RMSE vs. other absolute errors namely mean absolute error (MAE). See the the following references:
MAE and RMSE — Which Metric is Better?
What's the bottom line? How ...
2
votes
1answer
28 views
Why are optimization algorithms slower at critical points?
I just found the animation below from Alec Radford's presentation:
As visible, all algorithms are considerably slowed down at saddle point (where derivative is 0) and quicken up once they get out of ...
2
votes
0answers
47 views
Validation loss keeps fluctuating about training loss
I am training a Keras model for multi-target regression by using a custom loss function with the goal of getting predictions accurate to below 0.01 with respect to ...
0
votes
1answer
22 views
Differentiating roadmap of a loss function
Let's say I'm performing Stochastic Gradient Descent (SGD) on binary cross entropy error while optimizing weight $w_{2}$.
Binary cross entropy error:
$$L(y|p(x_{i}))=-y_{i}*ln(p(x_{i}))-(1-y_{i})*...
1
vote
0answers
50 views
Deep RL: Proximal policy optimization gradient calculation
Case:
Continuous action domain with 4 outputs (control problem)
Policy and Value function approximation with fully connected neural networks
I understand that the loss function for PPO for the ...
-1
votes
1answer
67 views
What's the difference between Error, Risk and Loss?
When we talk about 'Minimizing Loss', we often talk about loss functions such as Mean Squared Error (MSE); the term 'Empirical Risk Minimization' is often used interchangeably. So what's the ...
0
votes
0answers
27 views
Keras update Layers which are only Part of the Loss Function
I'm implementing a variational inference model. There are two main structures a inference network and a generative model. The first part is only used for training and only appears in the loss function....
1
vote
1answer
381 views
focal loss function help
I am working on a relation extraction and classification problem. The data is in the form of text files. The data is imbalanced. I want to use focal loss function to address class imbalance problem in ...
0
votes
2answers
760 views
Custom loss function which is included gradient in Keras
I want to make a custom loss function.
Concretely, I use a 2D Convolutional neural network in Keras.
So far, I've made various custom loss function by adding to losses.py.
However, in this case, I ...
1
vote
2answers
351 views
How the combination of cross entropy loss and gradient descent penalizes and rewards
For a simple problem of classification (C classes) using the softmax classifier, most people use the cross-entropy loss function to quantify the objective. The cross-entropy loss is:
$$L = -\sum_{i=1}...
0
votes
1answer
320 views
Loss function for an RNN used for binary classification
I'm using an RNN consisting of GRU cells to compare two bounding box trajectories and determine whether they belong to the same agent or not. In other words, I am only interested in a single final ...
0
votes
1answer
24 views
Loss function range normalization
This is from a referee report in a conference to which I submitted my paper - I don't quite get it and I'm not sure what I need to do about it.
I use Euclidean loss and Softmax cross-entropy (...
0
votes
0answers
175 views
Tensorflow - Clarification on how to predict more than 1 step ahead using LSTM
I'm looking for some clarification on how to perform more than one prediction ahead using long-short term memory in Tensorflow for univariate time series. The reason is that I'm having problems when I ...
1
vote
0answers
14 views
neural network function approximation with constraints
I would like to approximate a function $f(\cdot)$ by means of a neural network given a finite set of observations $f(x_i)$ where $x_i\in\mathbb{R}^n$ and $i=1\dots,N$. However, I have some prior ...
0
votes
0answers
85 views
Can a Warm-up loss period easily be implemented in Keras?
I am creating a RNN in Keras. It was suggested that I utilize a warm-up period before loss is calculated to increase accuracy down the line.
I saw some people achieved this by creating a loss ...
0
votes
0answers
16 views
Decision Tree problem with dynamic loss function
I have a marketing problem, we have service lines, and margin on each line.
For lines that disconnect and lines that do not, we want to identify features that help us maximize the difference between ...
3
votes
1answer
38 views
Can pinball loss be used to construct a prediction interval?
I'm modeling some time series data ($\{y_t\}_t$) and would like to construct a model that is able to return not just a single-value prediction $\hat{y_t}$, but an interval $C_t=(\hat{y}_{t, lower}, \...
3
votes
0answers
395 views
Precision recall loss function
I've been using precision and recall as my metrics, as per keras-team/keras/pull/9393/files
Sensitivity & specificity is what I want to optimise for. Every epoch I output it:
...
3
votes
1answer
316 views
Understanding LSTM behaviour: Validation loss smaller than training loss throughout training for regression problem
I'm building a lstm model for regression on timeseries. To verify my implementation of the model and understand keras, I'm using a toyproblem to make sure I understand what's going on. Problem is I do ...
1
vote
1answer
349 views
How to create a custom loss function from sklearn metrics in Keras? [closed]
I'd like to use the mutual information metric from sklearn as a loss function for a neural network in Keras, but I'm not sure how to do it. I'd like to try this because relationships in my dataset are ...
4
votes
2answers
159 views
What algorithms will stuck in the local minimum?
Algorithms like neural network are easily getting stuck in local minimum because the shape of the loss function (so there are parameters like momentum are designed to solve this type of problem). ...
1
vote
2answers
20 views
Weighting influence of two neural networks on classification
I'm training a model that has two neural networks. One of them is a resnet18 CNN which has as it's input images. The other one is a small one hidden layer network that has as it's input four other ...
2
votes
0answers
44 views
Cross error loss function cause division by zero error
How to calculate cross entropy when actual output is 0? Would not it give indf brcause of log(0) and the cross entropy for binary classification is given by:
-(ylog(actual_output)+(1-y)*(1-...
3
votes
2answers
140 views
Why validation loss worsens while precision/recall continue to improve?
I'm training a neural network on 'easy' dataset with ~15k examples.
Network overfits pretty fast.
The thing I cannot understand that after 5th epoch validation loss is starting to worsen, while ...
1
vote
1answer
43 views
Loss for CNN decreases and settles but training accuracy does not improve
I am training a CNN with 2 conv layers 2 Relu and max pooling and 2 FC layers the last of which has only 2 units since it's a binary classification problem. The images are spatio-temporal continuous, ...
2
votes
0answers
89 views
How to use a cross entropy loss function for each letter/digit in a captcha?
I'm trying to develop a captcha solver using a simple fully-connected neural network in TensorFlow. All captchas have 5 digits/letters, each character can be a number 0-9 or a letter A-Z. They all ...
