Questions tagged [loss-function]
A function used to quantify the difference between observed data and predicted values according to a model. Minimization of loss functions is a way to estimate the parameters of the model.
471
questions
0
votes
0answers
5 views
Best loss function for baseN encoding LSTM model
everyone, I am trying to train an LSTM model for sequence prediction.
I have X and Y which are two numpy arrays. X is a list of integers (integer encoded strings) while to encode Y I used a BaseN ...
0
votes
0answers
9 views
How to combine the loss in a multitask neural network?
when we train a model to learn two tasks at the same time (Multitask learning), we get losses from both tasks in the neural network and then we combine them.
I've seen several works where they've done ...
0
votes
1answer
17 views
Awful predictions of RNN while MSE is very low
I have encountered a strange situation where the predictions of RNN are just awful despite the fact that NN has found a minimum of loss function at 0.002 for training and 0.0013-0.0015 for validation ...
0
votes
1answer
25 views
resnet50 based model classification loss increase
I'm trying to classify fonts in images into 7 classes.
I wanted to use a pre-trained ResNet50 for the task and use its features to my classification. So I've followed some guide and came up with the ...
0
votes
0answers
12 views
Custom Tensorflow loss function that disincentivizes all black pixels
I'm training a Tensorflow model that receives an image and segments the image into foreground and background. That is, if the input image is w x h x 3, then the ...
0
votes
0answers
14 views
Custom loss function for out of distribution detection using CNN in Tensorflow 2.0+
My question is in reference to the paper "Learning Confidence for Out-of-Distribution Detection in Neural Networks".
I need help in creating a custom loss function in TensorFlow 2.0+ as per ...
0
votes
0answers
10 views
Expectation of ELBO in Variational Autoencoder
I am working with VAEs. My input is x, which is a product of two variables $x_1$ and $x_2$.
The objective (ELBO) of VAE in terms of x is:
$E_{z\sim Q}[\log P(x|z)] - \mathcal{D}[Q(z|x)||P(z)]$.
I want ...
1
vote
0answers
26 views
Can an Imbalanced Datset be an oportunity for Transfer Learning with Neural Networks?
While solving classification tasks on imbalanced datasets with Neural Networks(NN) there are two general ways of handling imbalanced data:
A. Resample the data, either with over or undersampling ...
0
votes
1answer
33 views
Custom loss function for regression
I am trying to write a custom loss function for a machine learning regression task. What I want to accomplish is following:
Reward higher preds, higher targets
Punish higher preds, lower targets
...
1
vote
1answer
62 views
Naive Bayes loss function
Does Naive Bayes classifier require a loss function for Bernoulli classification? If yes, what loss function does Naive Bayes classification use? And how does it work?
0
votes
0answers
22 views
Deep learning - edge detection
I am trying to build a model, which would be used for edge detection of the iris.
For this purpose, I have built a U-net model, which successfully works for image segmentation tasks. Also, I have a ...
0
votes
0answers
9 views
ignoring instances or masking by zero in a multitask learning model
For a multitask learning model, I've seen that approaches usually mask the output that doesn't have a label with zeros. As an example, have a look here: How to Multi-task learning with missing labels ...
1
vote
0answers
18 views
Shannon Information Content related to Uncertainty?
I'm a data scientist student currently writing my master thesis which resolves around the Cross Entropy (CE) Loss Function for neural networks. From my understanding, the CE is based on the Entropy, ...
0
votes
2answers
29 views
Activation and Loss Function not chosen correctly when use Neural Network
I have three classes for my text dataset before.
These are my classes:
0 = Cat
1 = Not Both
2 = Dog
Then I use this code:
...
0
votes
0answers
16 views
Validation accuracy have an almost good accuracy but loss function is high too
Based on my project, I find a little problem there with the statement like this.
I want to make model with neural networks for text dataset. Then I use Pad Sequence for my text and Array for the ...
0
votes
0answers
34 views
What if training loss is negative
I have designed a custom loss function where I have to maximize the KL divergence. So I took KL(P||Q) and negated it and now my loss function is -KL(P||Q) and this loss function leads to negative ...
0
votes
0answers
25 views
Siamese netwroks - how to choose loss function?
I have read several articles about siamese netwroks, and I understand that there are 3 different types of loss functions:
...
0
votes
1answer
19 views
Early stopping based on average val_loss of last ten epoches and with some n partiences
I am training a DNN with CNN in Keras. Though, I can write an EarlyStopping criteria based on val_loss but due to minor oscillations in the val_loss, I want to monitor the average validation loss over ...
1
vote
1answer
23 views
Finding logistic loss/negative log likelihood - binary logistic regression classification
I am new to ML and data science and am struggling with a simple problem. In my problem, I am given a series of datapoints $X_i$ where $X_i = (x_{i1}, x_{i2})$ with each data point having a label $y_i$ ...
1
vote
1answer
28 views
How do I know if my model's result is good enough?
I have a dataset of different people with their insurance cost. I have trained a neural-network to predict the insurance cost (charges column) based on the other features (age,bmi, etc.). Here is how ...
1
vote
0answers
9 views
How to treat losses asymmetrically in Keras?
I am developing a Keras NN that predicts three categories: A, B and C. However, the impact of errors is not symmetrical. i.e. If the network predicts A when the label is C, that is much less desirable ...
1
vote
0answers
12 views
Is this a correct implementation of a meandice loss function
HarDNet was trained to perform semantic segmentation of images. Increase of a meandice implies that the model quality improved.
...
1
vote
1answer
29 views
Cost function - Log Loss query
What is the purpose of using "log" in the logistic regression cost function "log loss"?
1
vote
0answers
40 views
How to Fine Tune a BERT model for sentiment analysis to get the best f1 score
I am building a multi-class sentiment analysis BERT model that's optimized to give the best f1 score.
More specifically, I train each epoch by optimizing binary cross entropy per class, taking the ...
1
vote
2answers
54 views
Why is cross entropy based on Bernoulli or Multinoulli probability distribution?
When we use logistic regression, we use cross entropy as the loss function. However, based on my understanding and https://machinelearningmastery.com/cross-entropy-for-machine-learning/, cross entropy ...
0
votes
1answer
10 views
The formula of loss function uses '(i)' as power of expected and real variables. What does that mean?
In the formula below, could one understand $y^{(i)}$ as $y_i$ ? If not, what is the fundamental difference ?
$$ j(\theta_0, \theta_1) = \frac{1}{2m}\sum_{i=1}^m(h_{\theta}(x^{(i)})-y^{(i)})^2 $$
0
votes
0answers
18 views
Keras Custom loss Penalize more when actual and prediction are on opposite sides of Zero
I'm training a model to predict percentage change in prices. Both MSE and RMSE are giving me up to 99% accuracy but when I check how often both actual and prediction are pointing in the same direction ...
0
votes
0answers
26 views
Understanding data and regularization loss
I am reading & coding along Neural Networks from scratch in Python. In it there is a section about L1 / L2 regularization ...
0
votes
0answers
9 views
Multi-class classification but a single feature sometimes boils it down to a binary-classification
I have a three-class classification problem for a large dataset. Classes are 0, 1 and 2. There's a categorical variable in my feature vectors such that when a sample point has this variable positive, ...
3
votes
0answers
37 views
Why margin loss is used in Capsule Network instead of Cross Entropy loss?
I'm reading the Capsule Network paper proposed by Hinton. I'm not sure why the margin loss is used instead of the cross entropy loss. Any intuitive explaination for this?
0
votes
0answers
13 views
validation loss early increase (during warm-up)
Several questions have been asked about validation loss behavior during training of a DNN. It's clear to me that validation loss and accuracy are somehow correlated, but their curves can differ from ...
0
votes
0answers
9 views
Loss function for normal distribution regression problem
My project involves training an input of random uniformly distributed data using regression (this is my approach) to output random normally distributed data. The issue with formulating the problem is ...
0
votes
1answer
21 views
Loss function for a regression model in image segmentation task
I am training a model to segment an image to predict the degree of damage (ranging from 0: no damage, to 5: severe damage) for each pixel of an image. I have approached it this way:
...
0
votes
0answers
27 views
How to use Softmax and CrossEntropyLoss in a classification problem?
I am new in the deep learning field and I would like to go in to understand some concepts.
I started learn with pytorch and I have the next question:
I constructed my own dataset that contains images ...
1
vote
1answer
20 views
Understanding deconvolutional network loss function
In the paper (1), there is a description of a deconvolutional network.
The loss function (with only one layer) compares the colour channels of the orignal image with the colour channels of the ...
2
votes
1answer
89 views
Does gradient descent always find global minimum for specific regression type?
From my understanding, linear regression is used for predicting an output based on an input using a linear equation that is optimally fitted to some input data. We choose the best fitted linear ...
0
votes
0answers
12 views
selection of loss function to avoid overfitting by autoencoder in prediction a figure with a sharp rise
I have to select the loss function to avoid overfitting by autoencoder in prediction of this figure that has a sharp raise, I would like to find how to avoid overfitting by autoencoder in prediction a ...
1
vote
1answer
45 views
Loss in multi-class classification
I have a multi-class classification task. One of the standard approach in choosing loss function is to use a CrossEntropyLoss. It is a good option when classes are standonlone and not similar to each ...
1
vote
0answers
12 views
Deep autoencoder: validation loss doesn't change
I'm trying to understand autoencoders and reproduced some code from Keras documentation:
...
1
vote
0answers
38 views
Word embedding autoencoder
I'm trying to train a word embedding autoencoder, but it either doesn't train, or trains but doesn't make predictions. I know I'm doing something wrong, so any help is greatly appreciated.
Here is my ...
0
votes
0answers
11 views
Continuous Bag of Words loss function and training objective
CBOW from what I understand, obtains a probability distribution $P(w|c)$ for all words $w$ in the vocabulary, given context $c$. Th loss function is: $-logP(w|c)$, which means this would be maximised ...
0
votes
0answers
18 views
loss function for simultaneous prediction of multiple OHE vectors
Hi I am trying to figure out how to set up the loss function for a model where the outputs are 3 one hot encoded vectors which are predicted simultaneously. This is a fully connected feed-forward ...
1
vote
0answers
27 views
Lovasz Softmax loss explanation
I would like to use Lovasz softmax for foreground background semantic segmentation because of its ability to improve segmentation with Jaccard index according to paper.
I got the idea that its a ...
0
votes
1answer
47 views
Reducing Validation loss for Triplet Loss Embeddings
I'm trying to create a facial recognition detector using triplet loss followed by a kNN algorithm.
I have roughly 10000 input images with 3 different classes, input size is 80x80. Model structure uses ...
1
vote
1answer
20 views
Significance of Convex Loss Function with Nonlinear Models
When used in a linear model, a convex loss function guarantees a unique global minimum for the parameters, which can be found by local optimization methods.
However, when the model is nonlinear (e.g. ...
0
votes
1answer
16 views
why the accuracy result and the loss result of an ANN model is inconsistent?
I trained a model based on an ANN and the accuracy is 94.65% almost every time while the loss result is 12.06%. Now my question is shouldn't the loss of the model be (100-94 = 6%) or near it? Why it ...
1
vote
1answer
26 views
Why would the accuracy of a model change when the loss doesn't?
I've trained 8 models based on the same architecture (convolutional neural network), and each uses a different data augmentation method. The accuracy of the models fluctuates greatly while the loss ...
0
votes
0answers
23 views
Inverting a matrix using a convolutional neural network
Just for a fun exercise, I am trying to invert a matrix, say size 28x28 (or even 5x5) with a neural network. The way I approached this (quite naively) is as follows:
I built a fully convolutional ...
0
votes
0answers
66 views
Custom keras callbacks and changing weight (beta) of regularization term in variational autoencoder loss function
The variational autoencoder loss function is this: Loss = Loss_reconstruction + Beta * Loss_kld. I am trying to efficiently implement Kullback-Liebler Divergence Cyclic Annealing--that is changing the ...
1
vote
0answers
13 views
Unable to train neural network for prediction
My data consists of a time series of values $\pm1$ and I am trying to apply a RBF NN as a function approximator. Essentially, the NN will take as input one data sample and predict the next sample (...
