Questions tagged [loss-function]

A function used to quantify the difference between observed data and predicted values according to a model. Minimization of loss functions is a way to estimate the parameters of the model.

Filter by
Sorted by
Tagged with
1 vote
0 answers
33 views

Analysis of relationship between accuracy and total loss (or cost) during training with logistic loss function and threshold 0.5

I'm trying to understanding the relationship between training accuracy and training loss in classification tasks, specifically using logistic regression. When using logistic loss as the loss function ...
Tran Khanh's user avatar
0 votes
0 answers
24 views

Ordinal log-loss in a multiclass classification in XGBoost?

I have a multi-class problem that which classes are simultaneously mutually exclusive and have ordering. You can think of the classes as being some score: 0 (Low), 1 (Medium), 2 (High). What I would ...
deblue's user avatar
  • 111
0 votes
0 answers
22 views

Generator loss not decreasing while training GAN

I’ve been attempting to create a basic GAN to generate images using this database of flowers (https://www.robots.ox.ac.uk/~vgg/data/flowers/102/). I’ve spent a few days on this, and largely based my ...
Hozaifa Bhutta's user avatar
0 votes
0 answers
37 views

Doubts on a custom loss function for regression problems

From what I read, I know we don't use log loss or cross entropy for regression problems. However, the entire logic behind binary cross entropy(say) is to firstly squeeze the y_hat between 0 and 1 (...
the_he_man's user avatar
0 votes
0 answers
10 views

Meaning of mean squared error in multistep prediction

In multistep prediction with LSTM(keras), say we had this kind of result: target = [[1,2,3] ,[4,5,6] ] predictions = [[1.1,2.2,3.3] , [4.4,5.5,6.6]] When we choose mean_squared_error as the loss ...
the_he_man's user avatar
0 votes
0 answers
39 views

Custom Loss Function Returns Graph Execution Error: Can not squeeze dim[0], expected a dimension of 1, got 32

I have built a loss function which adds time and frequency weighted averages and variances to the MSE: ...
Harry Chittenden's user avatar
0 votes
0 answers
15 views

Minimize MAE loss for a target that is sum of two other targets

Working on a regression modelling task where my dataset have some feature columns, two more columns A, B and a target column T. The goal is to predict T, and minimize MAE, that is ...
Ngo Cuong's user avatar
0 votes
0 answers
49 views

Custom loss and metric functions including additional parameter in Keras

The following example is based on this approach. Similar to that approach, I am wanting to pass an additional parameter with y_true for my custom metric, as both will be used in the computation of ...
David's user avatar
  • 1
0 votes
0 answers
17 views

In the GAN objective function, why do we first do we first find the D(x) that maximizes the objective function and then maximise wrt the generator?

The GAN objective function is optimised like this: argmin(argmax(L(G,D))) where the argmax finds the D (Discriminator) that maximises L(G,D). Why is it not the other way around, i.e. argmax(argmin(L(G,...
thebasqueinterdisciplinarian's user avatar
0 votes
0 answers
16 views

Cost function looks like the real math which is responsible for actually working on out problem statement If I talk on a whole and on a surface level?

Looking at the cost function for say linear regression, apart from changing the weight or the parameters, the cost function does the real job, right? If it is correct, what does cost function do in ...
Gurjot Singh's user avatar
0 votes
0 answers
9 views

Search recall optimization - what appropriate loss function to use?

I am studying machine learning and wanted to work on a project of my own so that I have better chances after graduating college. I'm studying the application of ML to improve searches using a toy ...
user9343456's user avatar
0 votes
0 answers
27 views

Training loss is much higher than validation loss

I am trying to train a neural network with 2 hidden layers to perform a multi class classification of 3 different classes. There is a huge imbalance to the classes, with the distribution being around ...
joseph wong's user avatar
1 vote
0 answers
16 views

Avoid overfitting to noise by a noise penalty approach instead of early stopping?

I came across this article on deep learning for computational MRI and found an interesting sentence "However, early stopping has to be performed to not overfit to the noisy measurements." ...
Shihao ZENG's user avatar
0 votes
0 answers
37 views

Understanding the desired behavior of the loss function of Variational Autoencoders

So I understood that when training VAE, we need to weight the KL part of the loss with a weight less than 1 so that the reconstruction loss can get a chance to learn (avoiding the posterior collapse). ...
user1407562's user avatar
0 votes
0 answers
11 views

Loss function for classifcation rewarding closer guess?

The default loss function in multi class classification is cross_entropy, which treats all wrong guesses equally. If the distance between buckets are meaningful, for example, given the real bucket is ...
jerron's user avatar
  • 101
2 votes
1 answer
67 views

What is the benefit of the exponential function inside softmax?

I know that softmax is: $$ softmax(x) = \frac{e^{x_i}}{\sum_j^n e^{x_j}}$$ This is an $\mathbb{R}^n \implies \mathbb{R}^n$ function, and the elements of the output add up to 1. I understand that the ...
Victor2748's user avatar
0 votes
0 answers
20 views

Is there a canonical cross entropy from the confusion matrix?

In Wu, MT. Confusion matrix and minimum cross-entropy metrics based motion recognition system in the classroom. Sci Rep 12, 3095 (2022). https://doi.org/10.1038/s41598-022-07137-z the author uses a ...
arivero's user avatar
  • 101
2 votes
1 answer
86 views

Representation of Strictly Proper Scoring Rule for Multiclasss Classificaiton

I am working on a classification problem, using features $\mathbf{x}$ to predict a target variable $y \in \mathbb{N}_0$. By a strictly proper scoring rule I mean a loss function $\ell(y,\hat{y})$ for ...
user1337's user avatar
1 vote
0 answers
43 views

Replication of XGBoost's binary:logistic loss

I am trying to replicate XGBoost's logistic loss function as a first step before implementing my own custom loss functions. Following from here and looking at the original code in git repository, I ...
deblue's user avatar
  • 111
0 votes
0 answers
47 views

Train neural network to predict multiple distributions

I aim to train a neural network to predict 2 distributions (10 quantiles, i.e. deciles) at 5 time points. So my y is of shape: ...
A_Murphy's user avatar
0 votes
0 answers
48 views

Need feedback on idea for new regularization term

I've been working on creating a regularization term that ensures that correlated attributes are given similar weights in a linear model. This helps to avoid some of the inconsistency in the weights of ...
Brett L's user avatar
0 votes
0 answers
45 views

The cost function gets stuck at 120 epochs

I did a neural network in c++ to recognize handwritten digits using the MNIST dataset without any neural network pre-existing libraries. My network has 784 inputs neuron (the pixel of the image), 100 ...
kripi's user avatar
  • 1
0 votes
0 answers
54 views

XGBoost Architecture Diagram required

Good Day! My topic is general and theory related, about XGBoost working. I am searching XGBoost Architecture Diagram. I know it works on principles of Decision Trees, Bagging, Random Forest, Boosting, ...
P_Z's user avatar
  • 1
0 votes
0 answers
47 views

BCE loss stuck at 0.693 in the beginnng of training and then started to decrease, why?

I'm using a Transformer encoder with a binary cross entropy loss for CTR prediction. The training batch loss is at around 0.693 constantly for the beginning several thousand steps (batches). I'm using ...
CyberPlayerOne's user avatar
0 votes
1 answer
44 views

How to implement a custom loss with a non-mathematical operation (simulation) that backpropagates with PyTorch?

I am writing a Neural Network, which output is not used directly for the loss-function, but rather as the input for a simulation model. After the simulation ran, I am using the simulated_value and the ...
zenia's user avatar
  • 1
0 votes
1 answer
50 views

Transpose of a 3D tensor

I need to transpose a 3-dimensional tensor of the shape (batch_size, N, M) to (batch_size, M, N) in a custom loss function in Keras with tensorflow as the backend. I tried using the following function ...
Doc Jazzy's user avatar
0 votes
0 answers
186 views

How can I interpret the results of my loss functions?

I use yolov8 for object detection. The results for my training look like this: As you can see in general my validation losses are quite higher than my training losses. Here the comparison of box_loss ...
michaelgr22's user avatar
0 votes
1 answer
30 views

How to select the validation loss value in this model to be compared with other models?

I'm training an LSTM model. I'm confusing about the validation loss of the model. Which value better represents the validation loss of the model? Is it the last value I obtain in the floowing loop, or ...
Mouna Ahmen's user avatar
0 votes
0 answers
29 views

Which loss function should I use if multiple values are correct?

My task is to create a QA-model. I give it a context and a question that it should answer. The answer is usually one word, so a very simplified input would be e.g. Context: "Max eats a banana. ...
max245905's user avatar
0 votes
0 answers
28 views

How to implement a custom loss function acting differently on multiple instances with keras?

I want to reproduce the results in "Online Neural Networks for Change-Point Detection" Hushchyn et al., but I'm having trouble implementing their loss function with Keras. The algorithm ...
Andrea Giovanni De Marchi's user avatar
1 vote
1 answer
30 views

GAN Output Gradient Calculation

Loss function for discriminator, which needs to be maximized: -log(D(x)) + log(1-D(G(z))). Loss function for generator, which needs to be maximized: log(D(G(z))) What would the calculation of the loss ...
David's user avatar
  • 11
0 votes
0 answers
11 views

Learning target of Denoising Diffusion Probabilistic Model

I am trying to understand the learning target of DDPM. Trying to understand $D_{KL}(q(x_{1:T}|x_0)||p_\theta(x_{1:T}|x_0)$ in following line. $$ -\log p_\theta(x_0) <= -\log p_\theta(x_0) + D_{KL}(...
Mr.Wang from Next Door's user avatar
0 votes
1 answer
177 views

How complex can I make a classifier's loss function in Scikit-Learn?

I want to customize the loss vanilla loss function being used by scikit-learn classifiers like the Logistic Regression classifier, etc. For example, if the vanilla empirical risk minimization ...
Inferno Dynast's user avatar
1 vote
0 answers
30 views

What is the l2-norm of a scalar

What is the meaning of the l2-norm when dealing with scalar values? I'm assuming it would be the same thing as taking the absolute value. For context: I am trying to implement the clustering method ...
Droidenkiller's user avatar
1 vote
1 answer
313 views

PyTorch CrossEntropyLoss and Log_SoftMAx + NLLLoss give different results

As per PyTorch documentation CrossEntropyLoss() is a combination of LogSoftMax() and NLLLoss() function. However, calling CrossEntropyLoss() gives different results compared to calling LogSoftMax() ...
cbelwal's user avatar
  • 113
0 votes
0 answers
40 views

CTC loss Expected input_lengths to have value at most 144, but got value 174

Can someone help me with CTC loss. I wrote a conformer model for ASR and to train encoder I need CTC loss. But when I train model I got error "Expected input_lengths to have value at most 144, ...
EZpeezy's user avatar
0 votes
1 answer
47 views

What is this "F" subscript symbol that shows up in this loss function?

i was reading this https://arxiv.org/pdf/2303.14535v1.pdf paper when i came across this: What is this F? Initially i assumed this was a standard L2 distance but i'm not so sure anymore.
Alessandro Polidori's user avatar
0 votes
0 answers
32 views

Parametric loss function

I would like to train a network to predict two continuously-valued outputs, x and y, from 64 continuously-valued inputs. For each set of 64 input values, there isn't a single "correct" or &...
paperjam's user avatar
  • 101
1 vote
2 answers
83 views

Neural network not learning at all

I am training a MLP on a tabular dataset, the pendigits dataset. Problem is that training loss and accuracy are more or less stable, while validation and test loss and accuracy are completely constant....
CasellaJr's user avatar
  • 229
0 votes
0 answers
176 views

Weighting and loss function for multi-dimensional output on ECG neural network in Tensorflow

I am working on a DNN that is training on ecg data with a shape of [None,1,2500] and output shape of [None,12,19] where 19 is a ...
ekg-display's user avatar
0 votes
0 answers
46 views

Problem of constant shift in prediction for neural network regression model with gradient-domain loss function

I'm training a regression model using neural network which is trained on MSE of both output and spatial gradient of output. With some simplification, the model is: $$ y = f(\mathbf{x};\theta) $$ where ...
mjhong's user avatar
  • 1
1 vote
0 answers
51 views

NANs, Infinities, and very large losses with normalizing flows

I am new to normalizing flows and have been trying to use them with a high-dimensional dataset, and I have been running into very large numbers and errors with sampling that don't occur when I use a ...
quail's user avatar
  • 31
0 votes
1 answer
231 views

Why is cross entropy loss averaged and not used directly as a sum during model training(such as in neural networks)

Why is the cross entropy loss for all training examples(or the training examples in a batch) averaged over size of the training set(or batch size) ? Why is it not just summed and used ?
user17420392's user avatar
0 votes
0 answers
23 views

What is log likelihood/maximum likelihood loss?

Very naively, how might one conceptualize log likelihood/maximum likelihood? What situations is it used in? Why would someone prefer this sort of loss function compared to something like an L2 or MSE ...
quail's user avatar
  • 31
0 votes
0 answers
24 views

Weighting loss functions for multi task learning

I am training a multi-task neural network which is predicting a binary target variable, an 18-class target, and a 17-class target. I am calculating the cross-entropy loss for each task, then summing ...
sdg's user avatar
  • 1
0 votes
0 answers
98 views

How does cross-entropy loss change with the number of classes?

How does the value of the cross-entropy loss function vary with the number of classes being predicted? Formally, if the loss function is $$ L = - \sum_{x \in X} P^*(x) \log P(x) $$ where $P^*(\cdot)$ ...
sdg's user avatar
  • 1
0 votes
0 answers
7 views

Approaching multiple records for one observation; radiomics of 2D slices of a 3D object

Background I am trying to create a model that can predict Type 2 diabetes in a patient based on MRI scans of their thigh muscle. Previous literature has shown that fat deposition in the muscle of ...
Saminy Creed's user avatar
0 votes
1 answer
26 views

Which is the loss function used for validating a CF Recommender System?

I am developing (from scratch) a memory-based CF Recommender System based on movielens dataset. My CF RS uses a URM (User Rating Matrix) where r_ij contains the rating the user i gave to movie j (or ...
tail's user avatar
  • 117
1 vote
0 answers
377 views

The val_loss is nan, but loss is printing. Both train and validation losses are nan in model.evaluate(), and the acc improves during training

There is a 2-class classification problem, and my loss function is custom. The labels are categorical, and the final activation function is Softmax. During the training, the loss is printed, but the ...
Armin Amini's user avatar
0 votes
0 answers
21 views

Ground truth as a function of weights in Keras

I have a convolutional neural network that takes an image an outputs a value between -1 and 1. If the image is an array $I$, and the network transforms the array such that $\text{output} = f(I) \in [-...
Chandrahas's user avatar

1
2 3 4 5
11