Skip to main content
Share Your Experience: Take the 2024 Developer Survey

Questions tagged [loss-function]

A function used to quantify the difference between observed data and predicted values according to a model. Minimization of loss functions is a way to estimate the parameters of the model.

Filter by
Sorted by
Tagged with
0 votes
0 answers
10 views

Why does the TensorFlow docs use a different GAN generator loss?

As per the original paper that introduced GANs, the generator loss is given as: $$ L_{G} = L _{BCE}(\mathbf{\vec 0}, \mathbf{D}(\mathbf{G}(\mathbf{\vec z}))) = \log(1 - \mathbf{D}(\mathbf{G}(\mathbf{\...
Sagnik Taraphdar's user avatar
1 vote
1 answer
37 views

Mean Absolute Error from Scratch in NumPy

I recently tried implementing MAE from scratch in NumPy. The loss value and the slope seem to be equivalent to what Scikit-learn outputs, but for some reason the intercept value seems to converge to ...
vxnuaj's user avatar
  • 11
0 votes
0 answers
7 views

How to implement CP tensor completion with extra calculations?

I am new to tensor decomposition. I want to know from a practical point of view, how to use an already known tool/library to compute CP factorization for tensor completion. Specifically, I want to ...
jjjj's user avatar
  • 1
0 votes
0 answers
18 views

Autoencoders failing to recreate MNIST numbers

I have been having trouble trying to get a working (non-variational) autoencoder to reproduce images from the MNIST dataset. The two biggest issues is an averaging of the samples to yield a single ...
Mce Bab's user avatar
1 vote
1 answer
36 views

How to choose a loss function and how to calculate the loss for Text Generation in Generative AI?

For the classification problems, what loss functions can I choose ? For the translation problem how do I decide whether the translation is good and how to choose a loss function? And what about the ...
Qiulang's user avatar
  • 133
2 votes
1 answer
37 views

Custom loss function in python

I am trying to implement a custom loss function inspired by https://arxiv.org/pdf/2305.10464.pdf. That is: $ L(\mathbf{x}) = (1-y) \left\lVert \mathbf{x_{true} - \mathbf{x_{pred}}} \right\rVert^2 + y \...
Gst's user avatar
  • 21
1 vote
0 answers
40 views

Multiclass matrix loss function in scikit-learn / xgboost / lightgbm

I have data with 4 classes: $c_1, c_2, c_3, c_4$. I'd like to create a classifier which has different scaling for the loss function per class combination: $$ \begin{bmatrix} 0 & l \left( \hat{c}_{...
Avi T's user avatar
  • 56
0 votes
0 answers
10 views

Problem when merge two probability distribution [Pointer Network]

I'm trying to re-implement Pointer Gen Net from this paper. Ya but you don't really need to read the paper. To sum up briefly, I have a vector probability distribution over vocabulary called p_vocab. ...
jupyter's user avatar
  • 101
1 vote
1 answer
44 views

Does using different optimizer change the loss landscape

I plot the landscape using this code, and I notice the landscape shape has changed a lot. My understanding is that the optimizer does not change the loss landscape. But now I'm confused if its just ...
user836026's user avatar
0 votes
0 answers
14 views

How to combine a classificiation dataset with a pair-wise comparison dataset

Let's say I'm trying to train a neural network that predicts a single output [0.0, 1.0] value that correlates to photo realism which I can use either in a classification setting or for ranking. I have ...
ahbutfore's user avatar
  • 191
1 vote
1 answer
42 views

How to comment on goodness of loss functions?

I have two loss functions $\mathcal{L}_1$ and $\mathcal{L}_2$ to train my model. The model is predominantly a classification model. Both $\mathcal{L}_1$ and $\mathcal{L}_2$ takes are two variants of ...
Aleph's user avatar
  • 123
0 votes
0 answers
8 views

Add a custom loss in a public github repository on distillation

I’m trying to implement a custom loss in a public repository regarding knowledge distillation. The link to the repository is the following: " https://github.com/DefangChen/SimKD " The main ...
PiEmmeC's user avatar
0 votes
1 answer
36 views

My custom neural network is converging but keras model not

in most cases it is probably the other way round but... I have implemented a basic MLP neural network structure with backpropagation. My data is just a shifted quadratic function with 100 samples. I ...
tymsoncyferki's user avatar
1 vote
1 answer
58 views

Use of Gradient with respect to feature instead of model parameters

Generally, for any machine learning/deep learning system, we compute a loss, $L = l(x, \theta, y)$ which is a function of the input feature vector $x$ (after activation), model parameters $\theta$ (...
OlorinIstari's user avatar
0 votes
0 answers
18 views

Custom loss function for collinearity of 3 embeddings

I am trying to implement a loss function that takes as input 3 embeddings and output a value that is proportional to the collinearity of the embeddings. This is to shape the latent space of a ...
vlc's user avatar
  • 1
0 votes
0 answers
10 views

Cross-Entropy Loss for Prediction Distribution

I'm trying to adapt an example from the Neural Tangents Cookbook which uses a variant of Mean Squared Error (MSE) loss suited for Regression prediction distributions given by Mean and Variance as ...
Krailon's user avatar
0 votes
0 answers
11 views

How to split the data for speaker verification using AAM Softmax loss?

There are several models for speaker verificaiton task (wavlm-ecapa / xvector / ...). Some of those model where trained with AAM Softmax loss, which gets the number of labels as input. When training ...
user3668129's user avatar
1 vote
0 answers
20 views

How to force my model heads to learn different things?

I have an Seq2Seq model that has 2 generative LM heads. I want the two heads to focus on different features/styles while decoding. The approach that I was thinking of is adding a distance cost to the ...
Tathagato Roy's user avatar
0 votes
0 answers
76 views

Custom loss function for multi label classification in catboost?

I have a data frame which I want to use for multi class classification problem. There are total 6 classes (say a, b, c, d, e, f). I want to improve the precision for three classes (say a, b, c) i.e. ...
SUNITA GUPTA's user avatar
0 votes
0 answers
42 views

Setting derivative to 0 when minimizing expected loss over possible data samples

Suppose, given $x\in \mathbb{R}^d$, you want $\theta$, which is the solution of $$ \text{argmin}_{\theta} \mathbb{E}_a[L(\theta;x,a)] $$ where $a \sim \mathcal{N}(x,\Sigma)$ and $L$ measures a (convex)...
user9781778's user avatar
1 vote
0 answers
36 views

Analysis of relationship between accuracy and total loss (or cost) during training with logistic loss function and threshold 0.5

I'm trying to understanding the relationship between training accuracy and training loss in classification tasks, specifically using logistic regression. When using logistic loss as the loss function ...
Tran Khanh's user avatar
0 votes
1 answer
63 views

Is my model overfitting based on my accuracy/loss curves?

Do those results indicate that my model is overfitting?
Begnnier's user avatar
1 vote
0 answers
67 views

Ordinal log-loss in a multiclass classification in XGBoost? [closed]

I have a multi-class problem that which classes are simultaneously mutually exclusive and have ordering. You can think of the classes as being some score: 0 (Low), 1 (Medium), 2 (High). What I would ...
deblue's user avatar
  • 121
0 votes
0 answers
114 views

Generator loss not decreasing while training GAN

I’ve been attempting to create a basic GAN to generate images using this database of flowers (https://www.robots.ox.ac.uk/~vgg/data/flowers/102/). I’ve spent a few days on this, and largely based my ...
Hozaifa Bhutta's user avatar
0 votes
0 answers
50 views

Doubts on a custom loss function for regression problems

From what I read, I know we don't use log loss or cross entropy for regression problems. However, the entire logic behind binary cross entropy(say) is to firstly squeeze the y_hat between 0 and 1 (...
the_he_man's user avatar
0 votes
0 answers
15 views

Meaning of mean squared error in multistep prediction

In multistep prediction with LSTM(keras), say we had this kind of result: target = [[1,2,3] ,[4,5,6] ] predictions = [[1.1,2.2,3.3] , [4.4,5.5,6.6]] When we choose mean_squared_error as the loss ...
the_he_man's user avatar
0 votes
0 answers
111 views

Custom Loss Function Returns Graph Execution Error: Can not squeeze dim[0], expected a dimension of 1, got 32

I have built a loss function which adds time and frequency weighted averages and variances to the MSE: ...
Harry Chittenden's user avatar
0 votes
0 answers
19 views

Minimize MAE loss for a target that is sum of two other targets

Working on a regression modelling task where my dataset have some feature columns, two more columns A, B and a target column T. The goal is to predict T, and minimize MAE, that is ...
Ngo Cuong's user avatar
0 votes
0 answers
93 views

Custom loss and metric functions including additional parameter in Keras

The following example is based on this approach. Similar to that approach, I am wanting to pass an additional parameter with y_true for my custom metric, as both will be used in the computation of ...
David's user avatar
  • 1
0 votes
0 answers
21 views

In the GAN objective function, why do we first do we first find the D(x) that maximizes the objective function and then maximise wrt the generator?

The GAN objective function is optimised like this: argmin(argmax(L(G,D))) where the argmax finds the D (Discriminator) that maximises L(G,D). Why is it not the other way around, i.e. argmax(argmin(L(G,...
thebasqueinterdisciplinarian's user avatar
0 votes
0 answers
17 views

Cost function looks like the real math which is responsible for actually working on out problem statement If I talk on a whole and on a surface level?

Looking at the cost function for say linear regression, apart from changing the weight or the parameters, the cost function does the real job, right? If it is correct, what does cost function do in ...
Gurjot Singh's user avatar
0 votes
0 answers
13 views

Search recall optimization - what appropriate loss function to use?

I am studying machine learning and wanted to work on a project of my own so that I have better chances after graduating college. I'm studying the application of ML to improve searches using a toy ...
user9343456's user avatar
0 votes
0 answers
118 views

Training loss is much higher than validation loss

I am trying to train a neural network with 2 hidden layers to perform a multi class classification of 3 different classes. There is a huge imbalance to the classes, with the distribution being around ...
joseph wong's user avatar
1 vote
0 answers
17 views

Avoid overfitting to noise by a noise penalty approach instead of early stopping?

I came across this article on deep learning for computational MRI and found an interesting sentence "However, early stopping has to be performed to not overfit to the noisy measurements." ...
Shihao ZENG's user avatar
0 votes
0 answers
103 views

Understanding the desired behavior of the loss function of Variational Autoencoders

So I understood that when training VAE, we need to weight the KL part of the loss with a weight less than 1 so that the reconstruction loss can get a chance to learn (avoiding the posterior collapse). ...
user1407562's user avatar
0 votes
0 answers
14 views

Loss function for classifcation rewarding closer guess?

The default loss function in multi class classification is cross_entropy, which treats all wrong guesses equally. If the distance between buckets are meaningful, for example, given the real bucket is ...
jerron's user avatar
  • 101
2 votes
1 answer
138 views

What is the benefit of the exponential function inside softmax?

I know that softmax is: $$ softmax(x) = \frac{e^{x_i}}{\sum_j^n e^{x_j}}$$ This is an $\mathbb{R}^n \implies \mathbb{R}^n$ function, and the elements of the output add up to 1. I understand that the ...
Victor2748's user avatar
0 votes
0 answers
35 views

Is there a canonical cross entropy from the confusion matrix?

In Wu, MT. Confusion matrix and minimum cross-entropy metrics based motion recognition system in the classroom. Sci Rep 12, 3095 (2022). https://doi.org/10.1038/s41598-022-07137-z the author uses a ...
arivero's user avatar
  • 101
3 votes
1 answer
94 views

Representation of Strictly Proper Scoring Rule for Multiclasss Classificaiton

I am working on a classification problem, using features $\mathbf{x}$ to predict a target variable $y \in \mathbb{N}_0$. By a strictly proper scoring rule I mean a loss function $\ell(y,\hat{y})$ for ...
user1337's user avatar
  • 103
1 vote
0 answers
78 views

Replication of XGBoost's binary:logistic loss

I am trying to replicate XGBoost's logistic loss function as a first step before implementing my own custom loss functions. Following from here and looking at the original code in git repository, I ...
deblue's user avatar
  • 121
0 votes
0 answers
49 views

Train neural network to predict multiple distributions

I aim to train a neural network to predict 2 distributions (10 quantiles, i.e. deciles) at 5 time points. So my y is of shape: ...
A_Murphy's user avatar
0 votes
0 answers
48 views

Need feedback on idea for new regularization term

I've been working on creating a regularization term that ensures that correlated attributes are given similar weights in a linear model. This helps to avoid some of the inconsistency in the weights of ...
Brett L's user avatar
0 votes
0 answers
47 views

The cost function gets stuck at 120 epochs

I did a neural network in c++ to recognize handwritten digits using the MNIST dataset without any neural network pre-existing libraries. My network has 784 inputs neuron (the pixel of the image), 100 ...
kripi's user avatar
  • 1
0 votes
0 answers
69 views

XGBoost Architecture Diagram required

Good Day! My topic is general and theory related, about XGBoost working. I am searching XGBoost Architecture Diagram. I know it works on principles of Decision Trees, Bagging, Random Forest, Boosting, ...
P_Z's user avatar
  • 101
0 votes
0 answers
98 views

BCE loss stuck at 0.693 in the beginnng of training and then started to decrease, why?

I'm using a Transformer encoder with a binary cross entropy loss for CTR prediction. The training batch loss is at around 0.693 constantly for the beginning several thousand steps (batches). I'm using ...
CyberPlayerOne's user avatar
0 votes
1 answer
58 views

How to implement a custom loss with a non-mathematical operation (simulation) that backpropagates with PyTorch?

I am writing a Neural Network, which output is not used directly for the loss-function, but rather as the input for a simulation model. After the simulation ran, I am using the simulated_value and the ...
zenia's user avatar
  • 1
0 votes
1 answer
66 views

Transpose of a 3D tensor

I need to transpose a 3-dimensional tensor of the shape (batch_size, N, M) to (batch_size, M, N) in a custom loss function in Keras with tensorflow as the backend. I tried using the following function ...
Doc Jazzy's user avatar
0 votes
0 answers
466 views

How can I interpret the results of my loss functions?

I use yolov8 for object detection. The results for my training look like this: As you can see in general my validation losses are quite higher than my training losses. Here the comparison of box_loss ...
michaelgr22's user avatar
0 votes
1 answer
53 views

How to select the validation loss value in this model to be compared with other models?

I'm training an LSTM model. I'm confusing about the validation loss of the model. Which value better represents the validation loss of the model? Is it the last value I obtain in the floowing loop, or ...
Mouna Ahmen's user avatar
0 votes
0 answers
38 views

Which loss function should I use if multiple values are correct?

My task is to create a QA-model. I give it a context and a question that it should answer. The answer is usually one word, so a very simplified input would be e.g. Context: "Max eats a banana. ...
max245905's user avatar

1
2 3 4 5
11