Questions tagged [loss-function]
A function used to quantify the difference between observed data and predicted values according to a model. Minimization of loss functions is a way to estimate the parameters of the model.
391
questions
0
votes
0answers
18 views
Keras Beginner: How do I modify model.compile and model.fit as per custom loss and output?
I am a beginner experimenting with UNet deep learning model. I have implemented the basic model and was trying to incorporate weight map to the loss function to separate touching objects.
My reference ...
0
votes
0answers
6 views
larger batches decrease learning rate because of a technical artifact?
I'm training a neural network for a classification task and experimenting with different batch sizes. I'm using the negative log likelihood loss averaged over the ...
-1
votes
0answers
10 views
How to compute the loss with respect to learning rate and momentum (hyper-parameters) in TF with BERT? [closed]
I have a BERT model with TF 2.0 and I am trying to get the loss with respect to hyper parameters. I'm guessing that I would need to manually do some step function and manually update parameters using ...
0
votes
2answers
28 views
How to read the predicted label of a Neural Netowork with Cross Entropy Loss? Pytorch
I am using a neural network to predict the quality of the Red Wine dataset, available on UCI machine Learning, using Pytorch, and Cross Entropy Loss as loss function.
This is my code:
...
1
vote
0answers
15 views
Are there any reasons for using Huber over the Pseudo Huber Loss?
The Huber Loss is: $$ huber =
\begin{cases}
\frac{1}{2} t^2 & \quad\text{if}\quad |t|\le \beta \\
\beta |t| -\frac{\beta^2}{2} &\quad\text{else}
\end{cases} $$
The ...
0
votes
0answers
20 views
VAE KL-divergence with non-standard mean
I know I can make a VAE do generation with a mean of 0 and std-dev of 1.
I tested it with the following loss function:
...
1
vote
0answers
16 views
Which loss functions does h2o.gbm use by default?
the GBM implementation of the h2o package only allows the user to specify a loss function via the distribution argument, which defaults to ...
0
votes
0answers
14 views
Why is torch complaining about an empty tensor?
The structure_loss method is supposed to return a loss for ground truth vs predicted masks:
import numpy as np
import torch
import torch.nn.functional as F
import ...
0
votes
0answers
15 views
Best Loss function to use for Multiple Categories which have an implicit order
I am wondering what options I have for loss functions when the task at hand is Multi-Class Classification, where the classes themselves have an implicit order, ranging from least popular (class 0) to ...
-1
votes
1answer
25 views
Pytorch's CrossEntropyLoss? [closed]
Can anybody explain what's going on here? I thought I knew how cross entropy loss works.
I have tried with Negativeloglikelihood as well?
1
vote
0answers
23 views
Keras custom metric doesn't work as loss function [closed]
Referencing my previous question here.
I've managed to get my angular error metric working with tf.py_function;
...
0
votes
0answers
11 views
What is the right loss function for semantic segmentation or do I have to use all of them?
I'm doing my PhD research about image semantic segmentation and now I'm trying to understand what kind of loss function do I have to use with a CNN like U-Net.
I have found the paper "A survey of ...
1
vote
0answers
45 views
Can you estimate average precision from log loss?
I am doing my final thesis in the field of Deepfakes and their detection. The final outcome is to have a binary classifier which could predict which video was updated and which was not. In other words,...
0
votes
0answers
24 views
0
votes
0answers
17 views
Unbiased Predictions for all Distinct Training Subsets
Suppose I have a data set $\left(X_i \in \chi, y_i \in \zeta \right)$ where $X_i$ and $y_i$ correspond to instances and labels, and $\chi$ and $\zeta$ correspond to the space where $X_i$ and $y_i$ ...
0
votes
0answers
10 views
How can my loss be stable while the gradient keeps growing?
I have been working on an Offline/Batch Reinforcement Learning problem where I am using a BCQ-DDQN model as a Q-table. The model input is a state of 8 dimensions, and the output is a vector of Q-...
0
votes
2answers
29 views
Using Transaction Amount to Guide Learning in an Fraud Detection Machine Learning Model
I am currently using transaction amount as a feature in an XGBoost classification model designed to identify fraudulent transactions. Furthermore, transaction amount is bounded for this problem ...
0
votes
0answers
12 views
Prevent model from predicting the same examples for different inputs
I have a ANN model, that predicts a fixed length curve. The problem is, those curves are really similar to each other, for example these two:
To compare those curves, I use RMSD between their points....
0
votes
0answers
5 views
Encoding Data and huge loss during ANN training
I just started to learn on ANN and tried to experiment on my own on a Linear Regression. I got a dataset which had housing prices for a city. Tried going through this but my model gives me a huge loss....
0
votes
1answer
35 views
Why does my model give val_loss: nan
Here is my model. Why is it giving a val_loss of nan?
I am splitting the data in this way:
0
votes
0answers
14 views
Linear regression on car dehko dataset , validation loss lower than traiing
I was performing a simple regression on the car dehko dataset (Version 3, you can find it here), and I found that the validation loss is always lower than the training one.
Generally, I did some basic ...
1
vote
1answer
22 views
What quantile is used for the initial DummyRegressor for Gradient Boosting Regressor in scikit-learn?
According to the documentation of Scikit-Learn Gradient Boosting Regressor:
init: estimator or ‘zero’, default=None: An estimator object that is used to compute the initial predictions. init has to ...
0
votes
1answer
26 views
Oscilations in loss curve [closed]
I saw a similar question, but I think my problem is something different.
While training, the training loss and the validation loss move around one number, not decreasing significantly.
I have 122707 ...
0
votes
0answers
17 views
How to calculate loss?
Good afternoon!
I'm solving one task: train an LSTM to write an example like Shakespeare
I have torch tensor 'predictions' with dimensions (length_sequence, batch_size, num_of_symbols), num_of_symbols ...
1
vote
0answers
22 views
Custom Loss Function Equation
I am trying to reproduce a research paper, where it is a classification problem, and they have introduced a custom loss function that I am unable to understand.
Now I think I have to implement the ...
2
votes
1answer
181 views
Loss function for ReLu, ELU, SELU
Question
What is the loss function for each different activation function?
Background
The choice of the loss function of a neural network depends on the activation function. For sigmoid activation, ...
0
votes
1answer
28 views
Cause of periodic jumps in loss function
I might be missing something obvious as I am new to machine learning. I am training an SSD Inception V2 for detecting buildings from satellite images. I use the Tensorflow Object Detection API. I am ...
0
votes
0answers
48 views
PyTorch cross_entropy with 3D data (RNN/LSTM)
I am working on LSTMs and I want to compute cross_entropy loss, given X and y.
X.shape: (batch_size, time_steps, number_of_classes)
y.shape: (batch_size, time_steps)
y contains the ground truth ...
0
votes
0answers
22 views
How to reduce dimensionality of encoder decoder output?
I have an encoder decoder architecture where the output $ \bar{\bf{y}}_t $ is a sequence of integers of maximum length $n$. Each integer in the sequence is representative of a category so the ...
0
votes
2answers
137 views
Confused between optimizer and loss function
I always thought the SGD was a loss function then I read this on a notebook
...
3
votes
1answer
143 views
Which learning rate should I choose?
I'm training a segmentation model, Unet++, on 2d images and I am now trying to find the optimal learning rate.
The backbone of the model is Resnet34, I use Adam optimizer and the loss function is the ...
2
votes
3answers
69 views
Neural Network Loss Function - Mean Square Error: questions about what 'n' signifies
I'm very new to neural networks and have recently learnt about the loss functions used with neural networks.
This question is in regards to the mean square error metric, defined as (from the textbook ...
3
votes
2answers
41 views
no decrease loss and val_loss
I try to train a neural network for time series. I use some data from Covid, mainly the goal is knowing 14 days of number of people at hospital to predict the number at J+1.
I have use some early ...
0
votes
0answers
12 views
Loss function to compare non binary segmentation
I need to compare two images corresponding to landmark locations. I was thinking of something related to Dice loss. I cannot use dice loss since the image is not binary. The background is black but ...
0
votes
0answers
17 views
Loss function for 1d segment estimation
My problem is as follows:
I have a long segment [say, 100000 samples] as input.
Regardless of the method, I need to output a partition of this segment into k sub ...
0
votes
1answer
16 views
How to interpret this Plot of Model Loss from a BiLSTM model?
Hi everyone,
the above graph is produced by a BiLSTM model i just trained and tested. I can't seem to interpret it while it is very different from the references that i acquired by googling online.
...
1
vote
0answers
24 views
how weighted log loss works
I have seen this in a kaggle notebook. I understand we add some weight to classes. what I don't understand is how those weights are generated. below is the code. Can you explain why it's useful and ...
1
vote
2answers
120 views
Loss function in GradientBoostingRegressor
I was looking at the Scikit-Learn documentation for GradientBoostingRegressor.
Here it says that we can use 'ls' as a loss function which is least squares regression. But i am confused since least ...
0
votes
0answers
73 views
How do I implement my loss function in Keras/Tensorflow, when it seems to have different parameters to the default ones?
So, I'm a university student studying Data Science, and after my previous question about TensorFlow got literally zero answers on Stack Overflow, I figured I'd post this one here instead.
I need to ...
0
votes
0answers
55 views
How is BCELoss counted in PyTorch? [different result comparing to mathematical implementation]
I am trying to understand how Binary Cross Entropy is counted in PyTorch.
I've tried the same code from the PyTorch documentation here, but I get a different result comparing to mathematical ...
0
votes
0answers
77 views
Custom loss function with both min(y, p) and max(y,p)
I'm creating a neural network in tensorflow and need to minimize the following loss function:
$\frac{max(y,p)}{min(y,p)}$ Where $y$ represents the true value and $p$ the predicted value. Since the ...
1
vote
2answers
34 views
Will my validation loss eventually go down?
I'm currently training a binary classifier that takes in 2 inputs, and outputs which object it thinks is "better."
I have an absolutely massive dataset, about 2 trillion records, and I'm ...
0
votes
0answers
105 views
Binary cross entropy loss for one hot encoded 2 class problem
My aim is to predict whether a person is alive or dead. In the case there are two classes which can either be alive (1) or dead (0). The output could be only one class i.e 1 or 0 and not multi label ...
1
vote
0answers
49 views
Backpropagation Mathematics with Sigmoid Output Activation and Cross Entropy Loss
I am deriving a Weight update for a simple toy network with a Sigmoid Output Layer. I need some help double checking my math to make sure I did it correctly.
I am using Cross-Entropy Loss as my Loss ...
1
vote
1answer
345 views
Should Discriminator Loss increase or decrease?
This question is purely based on the theoretical aspect of GANs.
So, when training a GAN how should the discriminator loss look like?
Should the loss of discriminator increase (as the generator is ...
1
vote
1answer
60 views
How to implement a GridSearchCV custom scorer that is dependent on a training feature?
I would like to code a custom scoring function using the make_scorer function, where my custom_function(y_true, y_pred)...
3
votes
0answers
34 views
Train NN for balanced accuracy
I'm trying to get a NN (a CNN, to be precise) to predict labels on hugely imbalanced data, i.e. some classes are much bigger than others. I want to optimize the balanced accuracy. So I give the class ...
1
vote
0answers
17 views
Loss function in order to find the most certain samples
I have a classification problem where I try to predict if some measurement will go up or down in the next 10 seconds.
I have a few hundred of features and so far I simply trained a few classification ...
5
votes
2answers
70 views
How to use “tree boosting” with a data-driven loss function
We have a problem which has a data-driven (non-analytical) loss function. Our target contains whole numbers between 0 and 20 (the target is inherently discrete), although larger values are possible, ...
1
vote
1answer
54 views
Match between objective function and evaluation metric
Does the objective function for model fitting and the evaluation metric for model validation need to be identical throughout the hyperparameter search process?
For example, can a XGBoost model be ...
