Questions tagged [loss-function]

A function used to quantify the difference between observed data and predicted values according to a model. Minimization of loss functions is a way to estimate the parameters of the model.

Filter by
Sorted by
Tagged with
0
votes
0answers
18 views

Keras Beginner: How do I modify model.compile and model.fit as per custom loss and output?

I am a beginner experimenting with UNet deep learning model. I have implemented the basic model and was trying to incorporate weight map to the loss function to separate touching objects. My reference ...
0
votes
0answers
6 views

larger batches decrease learning rate because of a technical artifact?

I'm training a neural network for a classification task and experimenting with different batch sizes. I'm using the negative log likelihood loss averaged over the ...
-1
votes
0answers
10 views

How to compute the loss with respect to learning rate and momentum (hyper-parameters) in TF with BERT? [closed]

I have a BERT model with TF 2.0 and I am trying to get the loss with respect to hyper parameters. I'm guessing that I would need to manually do some step function and manually update parameters using ...
0
votes
2answers
28 views

How to read the predicted label of a Neural Netowork with Cross Entropy Loss? Pytorch

I am using a neural network to predict the quality of the Red Wine dataset, available on UCI machine Learning, using Pytorch, and Cross Entropy Loss as loss function. This is my code: ...
1
vote
0answers
15 views

Are there any reasons for using Huber over the Pseudo Huber Loss?

The Huber Loss is: $$ huber = \begin{cases} \frac{1}{2} t^2 & \quad\text{if}\quad |t|\le \beta \\ \beta |t| -\frac{\beta^2}{2} &\quad\text{else} \end{cases} $$ The ...
0
votes
0answers
20 views

VAE KL-divergence with non-standard mean

I know I can make a VAE do generation with a mean of 0 and std-dev of 1. I tested it with the following loss function: ...
1
vote
0answers
16 views

Which loss functions does h2o.gbm use by default?

the GBM implementation of the h2o package only allows the user to specify a loss function via the distribution argument, which defaults to ...
0
votes
0answers
14 views

Why is torch complaining about an empty tensor?

The structure_loss method is supposed to return a loss for ground truth vs predicted masks: import numpy as np import torch import torch.nn.functional as F import ...
0
votes
0answers
15 views

Best Loss function to use for Multiple Categories which have an implicit order

I am wondering what options I have for loss functions when the task at hand is Multi-Class Classification, where the classes themselves have an implicit order, ranging from least popular (class 0) to ...
-1
votes
1answer
25 views

Pytorch's CrossEntropyLoss? [closed]

Can anybody explain what's going on here? I thought I knew how cross entropy loss works. I have tried with Negativeloglikelihood as well?
1
vote
0answers
23 views

Keras custom metric doesn't work as loss function [closed]

Referencing my previous question here. I've managed to get my angular error metric working with tf.py_function; ...
0
votes
0answers
11 views

What is the right loss function for semantic segmentation or do I have to use all of them?

I'm doing my PhD research about image semantic segmentation and now I'm trying to understand what kind of loss function do I have to use with a CNN like U-Net. I have found the paper "A survey of ...
1
vote
0answers
45 views

Can you estimate average precision from log loss?

I am doing my final thesis in the field of Deepfakes and their detection. The final outcome is to have a binary classifier which could predict which video was updated and which was not. In other words,...
0
votes
0answers
24 views

VGG16 based model not learning to recognize emotions from videos

My model looks like this ...
0
votes
0answers
17 views

Unbiased Predictions for all Distinct Training Subsets

Suppose I have a data set $\left(X_i \in \chi, y_i \in \zeta \right)$ where $X_i$ and $y_i$ correspond to instances and labels, and $\chi$ and $\zeta$ correspond to the space where $X_i$ and $y_i$ ...
0
votes
0answers
10 views

How can my loss be stable while the gradient keeps growing?

I have been working on an Offline/Batch Reinforcement Learning problem where I am using a BCQ-DDQN model as a Q-table. The model input is a state of 8 dimensions, and the output is a vector of Q-...
0
votes
2answers
29 views

Using Transaction Amount to Guide Learning in an Fraud Detection Machine Learning Model

I am currently using transaction amount as a feature in an XGBoost classification model designed to identify fraudulent transactions. Furthermore, transaction amount is bounded for this problem ...
0
votes
0answers
12 views

Prevent model from predicting the same examples for different inputs

I have a ANN model, that predicts a fixed length curve. The problem is, those curves are really similar to each other, for example these two: To compare those curves, I use RMSD between their points....
0
votes
0answers
5 views

Encoding Data and huge loss during ANN training

I just started to learn on ANN and tried to experiment on my own on a Linear Regression. I got a dataset which had housing prices for a city. Tried going through this but my model gives me a huge loss....
0
votes
1answer
35 views

Why does my model give val_loss: nan

Here is my model. Why is it giving a val_loss of nan? I am splitting the data in this way:
0
votes
0answers
14 views

Linear regression on car dehko dataset , validation loss lower than traiing

I was performing a simple regression on the car dehko dataset (Version 3, you can find it here), and I found that the validation loss is always lower than the training one. Generally, I did some basic ...
1
vote
1answer
22 views

What quantile is used for the initial DummyRegressor for Gradient Boosting Regressor in scikit-learn?

According to the documentation of Scikit-Learn Gradient Boosting Regressor: init: estimator or ‘zero’, default=None: An estimator object that is used to compute the initial predictions. init has to ...
0
votes
1answer
26 views

Oscilations in loss curve [closed]

I saw a similar question, but I think my problem is something different. While training, the training loss and the validation loss move around one number, not decreasing significantly. I have 122707 ...
0
votes
0answers
17 views

How to calculate loss?

Good afternoon! I'm solving one task: train an LSTM to write an example like Shakespeare I have torch tensor 'predictions' with dimensions (length_sequence, batch_size, num_of_symbols), num_of_symbols ...
1
vote
0answers
22 views

Custom Loss Function Equation

I am trying to reproduce a research paper, where it is a classification problem, and they have introduced a custom loss function that I am unable to understand. Now I think I have to implement the ...
2
votes
1answer
181 views

Loss function for ReLu, ELU, SELU

Question What is the loss function for each different activation function? Background The choice of the loss function of a neural network depends on the activation function. For sigmoid activation, ...
0
votes
1answer
28 views

Cause of periodic jumps in loss function

I might be missing something obvious as I am new to machine learning. I am training an SSD Inception V2 for detecting buildings from satellite images. I use the Tensorflow Object Detection API. I am ...
0
votes
0answers
48 views

PyTorch cross_entropy with 3D data (RNN/LSTM)

I am working on LSTMs and I want to compute cross_entropy loss, given X and y. X.shape: (batch_size, time_steps, number_of_classes) y.shape: (batch_size, time_steps) y contains the ground truth ...
0
votes
0answers
22 views

How to reduce dimensionality of encoder decoder output?

I have an encoder decoder architecture where the output $ \bar{\bf{y}}_t $ is a sequence of integers of maximum length $n$. Each integer in the sequence is representative of a category so the ...
0
votes
2answers
137 views

Confused between optimizer and loss function

I always thought the SGD was a loss function then I read this on a notebook ...
3
votes
1answer
143 views

Which learning rate should I choose?

I'm training a segmentation model, Unet++, on 2d images and I am now trying to find the optimal learning rate. The backbone of the model is Resnet34, I use Adam optimizer and the loss function is the ...
2
votes
3answers
69 views

Neural Network Loss Function - Mean Square Error: questions about what 'n' signifies

I'm very new to neural networks and have recently learnt about the loss functions used with neural networks. This question is in regards to the mean square error metric, defined as (from the textbook ...
3
votes
2answers
41 views

no decrease loss and val_loss

I try to train a neural network for time series. I use some data from Covid, mainly the goal is knowing 14 days of number of people at hospital to predict the number at J+1. I have use some early ...
0
votes
0answers
12 views

Loss function to compare non binary segmentation

I need to compare two images corresponding to landmark locations. I was thinking of something related to Dice loss. I cannot use dice loss since the image is not binary. The background is black but ...
0
votes
0answers
17 views

Loss function for 1d segment estimation

My problem is as follows: I have a long segment [say, 100000 samples] as input. Regardless of the method, I need to output a partition of this segment into k sub ...
0
votes
1answer
16 views

How to interpret this Plot of Model Loss from a BiLSTM model?

Hi everyone, the above graph is produced by a BiLSTM model i just trained and tested. I can't seem to interpret it while it is very different from the references that i acquired by googling online. ...
1
vote
0answers
24 views

how weighted log loss works

I have seen this in a kaggle notebook. I understand we add some weight to classes. what I don't understand is how those weights are generated. below is the code. Can you explain why it's useful and ...
1
vote
2answers
120 views

Loss function in GradientBoostingRegressor

I was looking at the Scikit-Learn documentation for GradientBoostingRegressor. Here it says that we can use 'ls' as a loss function which is least squares regression. But i am confused since least ...
0
votes
0answers
73 views

How do I implement my loss function in Keras/Tensorflow, when it seems to have different parameters to the default ones?

So, I'm a university student studying Data Science, and after my previous question about TensorFlow got literally zero answers on Stack Overflow, I figured I'd post this one here instead. I need to ...
0
votes
0answers
55 views

How is BCELoss counted in PyTorch? [different result comparing to mathematical implementation]

I am trying to understand how Binary Cross Entropy is counted in PyTorch. I've tried the same code from the PyTorch documentation here, but I get a different result comparing to mathematical ...
0
votes
0answers
77 views

Custom loss function with both min(y, p) and max(y,p)

I'm creating a neural network in tensorflow and need to minimize the following loss function: $\frac{max(y,p)}{min(y,p)}$ Where $y$ represents the true value and $p$ the predicted value. Since the ...
1
vote
2answers
34 views

Will my validation loss eventually go down?

I'm currently training a binary classifier that takes in 2 inputs, and outputs which object it thinks is "better." I have an absolutely massive dataset, about 2 trillion records, and I'm ...
0
votes
0answers
105 views

Binary cross entropy loss for one hot encoded 2 class problem

My aim is to predict whether a person is alive or dead. In the case there are two classes which can either be alive (1) or dead (0). The output could be only one class i.e 1 or 0 and not multi label ...
1
vote
0answers
49 views

Backpropagation Mathematics with Sigmoid Output Activation and Cross Entropy Loss

I am deriving a Weight update for a simple toy network with a Sigmoid Output Layer. I need some help double checking my math to make sure I did it correctly. I am using Cross-Entropy Loss as my Loss ...
1
vote
1answer
345 views

Should Discriminator Loss increase or decrease?

This question is purely based on the theoretical aspect of GANs. So, when training a GAN how should the discriminator loss look like? Should the loss of discriminator increase (as the generator is ...
1
vote
1answer
60 views

How to implement a GridSearchCV custom scorer that is dependent on a training feature?

I would like to code a custom scoring function using the make_scorer function, where my custom_function(y_true, y_pred)...
3
votes
0answers
34 views

Train NN for balanced accuracy

I'm trying to get a NN (a CNN, to be precise) to predict labels on hugely imbalanced data, i.e. some classes are much bigger than others. I want to optimize the balanced accuracy. So I give the class ...
1
vote
0answers
17 views

Loss function in order to find the most certain samples

I have a classification problem where I try to predict if some measurement will go up or down in the next 10 seconds. I have a few hundred of features and so far I simply trained a few classification ...
5
votes
2answers
70 views

How to use “tree boosting” with a data-driven loss function

We have a problem which has a data-driven (non-analytical) loss function. Our target contains whole numbers between 0 and 20 (the target is inherently discrete), although larger values are possible, ...
1
vote
1answer
54 views

Match between objective function and evaluation metric

Does the objective function for model fitting and the evaluation metric for model validation need to be identical throughout the hyperparameter search process? For example, can a XGBoost model be ...

1
2 3 4 5
8