Questions tagged [loss-function]

A function used to quantify the difference between observed data and predicted values according to a model. Minimization of loss functions is a way to estimate the parameters of the model.

Filter by
Sorted by
Tagged with
0
votes
0answers
5 views

Best loss function for baseN encoding LSTM model

everyone, I am trying to train an LSTM model for sequence prediction. I have X and Y which are two numpy arrays. X is a list of integers (integer encoded strings) while to encode Y I used a BaseN ...
0
votes
0answers
9 views

How to combine the loss in a multitask neural network?

when we train a model to learn two tasks at the same time (Multitask learning), we get losses from both tasks in the neural network and then we combine them. I've seen several works where they've done ...
0
votes
1answer
17 views

Awful predictions of RNN while MSE is very low

I have encountered a strange situation where the predictions of RNN are just awful despite the fact that NN has found a minimum of loss function at 0.002 for training and 0.0013-0.0015 for validation ...
0
votes
1answer
25 views

resnet50 based model classification loss increase

I'm trying to classify fonts in images into 7 classes. I wanted to use a pre-trained ResNet50 for the task and use its features to my classification. So I've followed some guide and came up with the ...
0
votes
0answers
12 views

Custom Tensorflow loss function that disincentivizes all black pixels

I'm training a Tensorflow model that receives an image and segments the image into foreground and background. That is, if the input image is w x h x 3, then the ...
0
votes
0answers
14 views

Custom loss function for out of distribution detection using CNN in Tensorflow 2.0+

My question is in reference to the paper "Learning Confidence for Out-of-Distribution Detection in Neural Networks". I need help in creating a custom loss function in TensorFlow 2.0+ as per ...
0
votes
0answers
10 views

Expectation of ELBO in Variational Autoencoder

I am working with VAEs. My input is x, which is a product of two variables $x_1$ and $x_2$. The objective (ELBO) of VAE in terms of x is: $E_{z\sim Q}[\log P(x|z)] - \mathcal{D}[Q(z|x)||P(z)]$. I want ...
1
vote
0answers
26 views

Can an Imbalanced Datset be an oportunity for Transfer Learning with Neural Networks?

While solving classification tasks on imbalanced datasets with Neural Networks(NN) there are two general ways of handling imbalanced data: A. Resample the data, either with over or undersampling ...
0
votes
1answer
33 views

Custom loss function for regression

I am trying to write a custom loss function for a machine learning regression task. What I want to accomplish is following: Reward higher preds, higher targets Punish higher preds, lower targets ...
1
vote
1answer
62 views

Naive Bayes loss function

Does Naive Bayes classifier require a loss function for Bernoulli classification? If yes, what loss function does Naive Bayes classification use? And how does it work?
0
votes
0answers
22 views

Deep learning - edge detection

I am trying to build a model, which would be used for edge detection of the iris. For this purpose, I have built a U-net model, which successfully works for image segmentation tasks. Also, I have a ...
0
votes
0answers
9 views

ignoring instances or masking by zero in a multitask learning model

For a multitask learning model, I've seen that approaches usually mask the output that doesn't have a label with zeros. As an example, have a look here: How to Multi-task learning with missing labels ...
1
vote
0answers
18 views

Shannon Information Content related to Uncertainty?

I'm a data scientist student currently writing my master thesis which resolves around the Cross Entropy (CE) Loss Function for neural networks. From my understanding, the CE is based on the Entropy, ...
0
votes
2answers
29 views

Activation and Loss Function not chosen correctly when use Neural Network

I have three classes for my text dataset before. These are my classes: 0 = Cat 1 = Not Both 2 = Dog Then I use this code: ...
0
votes
0answers
16 views

Validation accuracy have an almost good accuracy but loss function is high too

Based on my project, I find a little problem there with the statement like this. I want to make model with neural networks for text dataset. Then I use Pad Sequence for my text and Array for the ...
0
votes
0answers
34 views

What if training loss is negative

I have designed a custom loss function where I have to maximize the KL divergence. So I took KL(P||Q) and negated it and now my loss function is -KL(P||Q) and this loss function leads to negative ...
0
votes
0answers
25 views

Siamese netwroks - how to choose loss function?

I have read several articles about siamese netwroks, and I understand that there are 3 different types of loss functions: ...
0
votes
1answer
19 views

Early stopping based on average val_loss of last ten epoches and with some n partiences

I am training a DNN with CNN in Keras. Though, I can write an EarlyStopping criteria based on val_loss but due to minor oscillations in the val_loss, I want to monitor the average validation loss over ...
1
vote
1answer
23 views

Finding logistic loss/negative log likelihood - binary logistic regression classification

I am new to ML and data science and am struggling with a simple problem. In my problem, I am given a series of datapoints $X_i$ where $X_i = (x_{i1}, x_{i2})$ with each data point having a label $y_i$ ...
1
vote
1answer
28 views

How do I know if my model's result is good enough?

I have a dataset of different people with their insurance cost. I have trained a neural-network to predict the insurance cost (charges column) based on the other features (age,bmi, etc.). Here is how ...
1
vote
0answers
9 views

How to treat losses asymmetrically in Keras?

I am developing a Keras NN that predicts three categories: A, B and C. However, the impact of errors is not symmetrical. i.e. If the network predicts A when the label is C, that is much less desirable ...
1
vote
0answers
12 views

Is this a correct implementation of a meandice loss function

HarDNet was trained to perform semantic segmentation of images. Increase of a meandice implies that the model quality improved. ...
1
vote
1answer
29 views

Cost function - Log Loss query

What is the purpose of using "log" in the logistic regression cost function "log loss"?
1
vote
0answers
40 views

How to Fine Tune a BERT model for sentiment analysis to get the best f1 score

I am building a multi-class sentiment analysis BERT model that's optimized to give the best f1 score. More specifically, I train each epoch by optimizing binary cross entropy per class, taking the ...
1
vote
2answers
54 views

Why is cross entropy based on Bernoulli or Multinoulli probability distribution?

When we use logistic regression, we use cross entropy as the loss function. However, based on my understanding and https://machinelearningmastery.com/cross-entropy-for-machine-learning/, cross entropy ...
0
votes
1answer
10 views

The formula of loss function uses '(i)' as power of expected and real variables. What does that mean?

In the formula below, could one understand $y^{(i)}$ as $y_i$ ? If not, what is the fundamental difference ? $$ j(\theta_0, \theta_1) = \frac{1}{2m}\sum_{i=1}^m(h_{\theta}(x^{(i)})-y^{(i)})^2 $$
0
votes
0answers
18 views

Keras Custom loss Penalize more when actual and prediction are on opposite sides of Zero

I'm training a model to predict percentage change in prices. Both MSE and RMSE are giving me up to 99% accuracy but when I check how often both actual and prediction are pointing in the same direction ...
0
votes
0answers
26 views

Understanding data and regularization loss

I am reading & coding along Neural Networks from scratch in Python. In it there is a section about L1 / L2 regularization ...
0
votes
0answers
9 views

Multi-class classification but a single feature sometimes boils it down to a binary-classification

I have a three-class classification problem for a large dataset. Classes are 0, 1 and 2. There's a categorical variable in my feature vectors such that when a sample point has this variable positive, ...
3
votes
0answers
37 views

Why margin loss is used in Capsule Network instead of Cross Entropy loss?

I'm reading the Capsule Network paper proposed by Hinton. I'm not sure why the margin loss is used instead of the cross entropy loss. Any intuitive explaination for this?
0
votes
0answers
13 views

validation loss early increase (during warm-up)

Several questions have been asked about validation loss behavior during training of a DNN. It's clear to me that validation loss and accuracy are somehow correlated, but their curves can differ from ...
0
votes
0answers
9 views

Loss function for normal distribution regression problem

My project involves training an input of random uniformly distributed data using regression (this is my approach) to output random normally distributed data. The issue with formulating the problem is ...
0
votes
1answer
21 views

Loss function for a regression model in image segmentation task

I am training a model to segment an image to predict the degree of damage (ranging from 0: no damage, to 5: severe damage) for each pixel of an image. I have approached it this way: ...
0
votes
0answers
27 views

How to use Softmax and CrossEntropyLoss in a classification problem?

I am new in the deep learning field and I would like to go in to understand some concepts. I started learn with pytorch and I have the next question: I constructed my own dataset that contains images ...
1
vote
1answer
20 views

Understanding deconvolutional network loss function

In the paper (1), there is a description of a deconvolutional network. The loss function (with only one layer) compares the colour channels of the orignal image with the colour channels of the ...
2
votes
1answer
89 views

Does gradient descent always find global minimum for specific regression type?

From my understanding, linear regression is used for predicting an output based on an input using a linear equation that is optimally fitted to some input data. We choose the best fitted linear ...
0
votes
0answers
12 views

selection of loss function to avoid overfitting by autoencoder in prediction a figure with a sharp rise

I have to select the loss function to avoid overfitting by autoencoder in prediction of this figure that has a sharp raise, I would like to find how to avoid overfitting by autoencoder in prediction a ...
1
vote
1answer
45 views

Loss in multi-class classification

I have a multi-class classification task. One of the standard approach in choosing loss function is to use a CrossEntropyLoss. It is a good option when classes are standonlone and not similar to each ...
1
vote
0answers
12 views

Deep autoencoder: validation loss doesn't change

I'm trying to understand autoencoders and reproduced some code from Keras documentation: ...
1
vote
0answers
38 views

Word embedding autoencoder

I'm trying to train a word embedding autoencoder, but it either doesn't train, or trains but doesn't make predictions. I know I'm doing something wrong, so any help is greatly appreciated. Here is my ...
0
votes
0answers
11 views

Continuous Bag of Words loss function and training objective

CBOW from what I understand, obtains a probability distribution $P(w|c)$ for all words $w$ in the vocabulary, given context $c$. Th loss function is: $-logP(w|c)$, which means this would be maximised ...
0
votes
0answers
18 views

loss function for simultaneous prediction of multiple OHE vectors

Hi I am trying to figure out how to set up the loss function for a model where the outputs are 3 one hot encoded vectors which are predicted simultaneously. This is a fully connected feed-forward ...
1
vote
0answers
27 views

Lovasz Softmax loss explanation

I would like to use Lovasz softmax for foreground background semantic segmentation because of its ability to improve segmentation with Jaccard index according to paper. I got the idea that its a ...
0
votes
1answer
47 views

Reducing Validation loss for Triplet Loss Embeddings

I'm trying to create a facial recognition detector using triplet loss followed by a kNN algorithm. I have roughly 10000 input images with 3 different classes, input size is 80x80. Model structure uses ...
1
vote
1answer
20 views

Significance of Convex Loss Function with Nonlinear Models

When used in a linear model, a convex loss function guarantees a unique global minimum for the parameters, which can be found by local optimization methods. However, when the model is nonlinear (e.g. ...
0
votes
1answer
16 views

why the accuracy result and the loss result of an ANN model is inconsistent?

I trained a model based on an ANN and the accuracy is 94.65% almost every time while the loss result is 12.06%. Now my question is shouldn't the loss of the model be (100-94 = 6%) or near it? Why it ...
1
vote
1answer
26 views

Why would the accuracy of a model change when the loss doesn't?

I've trained 8 models based on the same architecture (convolutional neural network), and each uses a different data augmentation method. The accuracy of the models fluctuates greatly while the loss ...
0
votes
0answers
23 views

Inverting a matrix using a convolutional neural network

Just for a fun exercise, I am trying to invert a matrix, say size 28x28 (or even 5x5) with a neural network. The way I approached this (quite naively) is as follows: I built a fully convolutional ...
0
votes
0answers
66 views

Custom keras callbacks and changing weight (beta) of regularization term in variational autoencoder loss function

The variational autoencoder loss function is this: Loss = Loss_reconstruction + Beta * Loss_kld. I am trying to efficiently implement Kullback-Liebler Divergence Cyclic Annealing--that is changing the ...
1
vote
0answers
13 views

Unable to train neural network for prediction

My data consists of a time series of values $\pm1$ and I am trying to apply a RBF NN as a function approximator. Essentially, the NN will take as input one data sample and predict the next sample (...

1
2 3 4 5
10