Questions tagged [loss-function]

A function used to quantify the difference between observed data and predicted values according to a model. Minimization of loss functions is a way to estimate the parameters of the model.

Filter by
Sorted by
Tagged with
0
votes
0answers
6 views

Validation accuracy have an almost good accuracy but loss function is high too

Based on my project, I find a little problem there with the statement like this. I want to make model with neural networks for text dataset. Then I use Pad Sequence for my text and Array for the ...
0
votes
0answers
27 views

What if training loss is negative

I have designed a custom loss function where I have to maximize the KL divergence. So I took KL(P||Q) and negated it and now my loss function is -KL(P||Q) and this loss function leads to negative ...
0
votes
0answers
18 views

Siamese netwroks - how to choose loss function?

I have read several articles about siamese netwroks, and I understand that there are 3 different types of loss functions: ...
0
votes
1answer
17 views

Early stopping based on average val_loss of last ten epoches and with some n partiences

I am training a DNN with CNN in Keras. Though, I can write an EarlyStopping criteria based on val_loss but due to minor oscillations in the val_loss, I want to monitor the average validation loss over ...
1
vote
1answer
22 views

Finding logistic loss/negative log likelihood - binary logistic regression classification

I am new to ML and data science and am struggling with a simple problem. In my problem, I am given a series of datapoints $X_i$ where $X_i = (x_{i1}, x_{i2})$ with each data point having a label $y_i$ ...
1
vote
1answer
28 views

How do I know if my model's result is good enough?

I have a dataset of different people with their insurance cost. I have trained a neural-network to predict the insurance cost (charges column) based on the other features (age,bmi, etc.). Here is how ...
1
vote
0answers
9 views

How to treat losses asymmetrically in Keras?

I am developing a Keras NN that predicts three categories: A, B and C. However, the impact of errors is not symmetrical. i.e. If the network predicts A when the label is C, that is much less desirable ...
1
vote
0answers
11 views

Is this a correct implementation of a meandice loss function

HarDNet was trained to perform semantic segmentation of images. Increase of a meandice implies that the model quality improved. ...
1
vote
1answer
27 views

Cost function - Log Loss query

What is the purpose of using "log" in the logistic regression cost function "log loss"?
1
vote
0answers
33 views

How to Fine Tune a BERT model for sentiment analysis to get the best f1 score

I am building a multi-class sentiment analysis BERT model that's optimized to give the best f1 score. More specifically, I train each epoch by optimizing binary cross entropy per class, taking the ...
1
vote
2answers
47 views

Why is cross entropy based on Bernoulli or Multinoulli probability distribution?

When we use logistic regression, we use cross entropy as the loss function. However, based on my understanding and https://machinelearningmastery.com/cross-entropy-for-machine-learning/, cross entropy ...
0
votes
1answer
10 views

The formula of loss function uses '(i)' as power of expected and real variables. What does that mean?

In the formula below, could one understand $y^{(i)}$ as $y_i$ ? If not, what is the fundamental difference ? $$ j(\theta_0, \theta_1) = \frac{1}{2m}\sum_{i=1}^m(h_{\theta}(x^{(i)})-y^{(i)})^2 $$
0
votes
0answers
12 views

Keras Custom loss Penalize more when actual and prediction are on opposite sides of Zero

I'm training a model to predict percentage change in prices. Both MSE and RMSE are giving me up to 99% accuracy but when I check how often both actual and prediction are pointing in the same direction ...
1
vote
0answers
25 views

Understanding data and regularization loss

I am reading & coding along Neural Networks from scratch in Python. In it there is a section about L1 / L2 regularization ...
1
vote
0answers
9 views

Multi-class classification but a single feature sometimes boils it down to a binary-classification

I have a three-class classification problem for a large dataset. Classes are 0, 1 and 2. There's a categorical variable in my feature vectors such that when a sample point has this variable positive, ...
3
votes
0answers
37 views

Why margin loss is used in Capsule Network instead of Cross Entropy loss?

I'm reading the Capsule Network paper proposed by Hinton. I'm not sure why the margin loss is used instead of the cross entropy loss. Any intuitive explaination for this?
0
votes
0answers
12 views

validation loss early increase (during warm-up)

Several questions have been asked about validation loss behavior during training of a DNN. It's clear to me that validation loss and accuracy are somehow correlated, but their curves can differ from ...
0
votes
0answers
6 views

Loss function for normal distribution regression problem

My project involves training an input of random uniformly distributed data using regression (this is my approach) to output random normally distributed data. The issue with formulating the problem is ...
0
votes
1answer
19 views

Loss function for a regression model in image segmentation task

I am training a model to segment an image to predict the degree of damage (ranging from 0: no damage, to 5: severe damage) for each pixel of an image. I have approached it this way: ...
0
votes
0answers
27 views

How to use Softmax and CrossEntropyLoss in a classification problem?

I am new in the deep learning field and I would like to go in to understand some concepts. I started learn with pytorch and I have the next question: I constructed my own dataset that contains images ...
1
vote
1answer
18 views

Understanding deconvolutional network loss function

In the paper (1), there is a description of a deconvolutional network. The loss function (with only one layer) compares the colour channels of the orignal image with the colour channels of the ...
2
votes
1answer
57 views

Does gradient descent always find global minimum for specific regression type?

From my understanding, linear regression is used for predicting an output based on an input using a linear equation that is optimally fitted to some input data. We choose the best fitted linear ...
0
votes
0answers
11 views

selection of loss function to avoid overfitting by autoencoder in prediction a figure with a sharp rise

I have to select the loss function to avoid overfitting by autoencoder in prediction of this figure that has a sharp raise, I would like to find how to avoid overfitting by autoencoder in prediction a ...
1
vote
1answer
42 views

Loss in multi-class classification

I have a multi-class classification task. One of the standard approach in choosing loss function is to use a CrossEntropyLoss. It is a good option when classes are standonlone and not similar to each ...
1
vote
0answers
12 views

Deep autoencoder: validation loss doesn't change

I'm trying to understand autoencoders and reproduced some code from Keras documentation: ...
1
vote
0answers
34 views

Word embedding autoencoder

I'm trying to train a word embedding autoencoder, but it either doesn't train, or trains but doesn't make predictions. I know I'm doing something wrong, so any help is greatly appreciated. Here is my ...
0
votes
0answers
9 views

Continuous Bag of Words loss function and training objective

CBOW from what I understand, obtains a probability distribution $P(w|c)$ for all words $w$ in the vocabulary, given context $c$. Th loss function is: $-logP(w|c)$, which means this would be maximised ...
0
votes
0answers
18 views

loss function for simultaneous prediction of multiple OHE vectors

Hi I am trying to figure out how to set up the loss function for a model where the outputs are 3 one hot encoded vectors which are predicted simultaneously. This is a fully connected feed-forward ...
1
vote
0answers
18 views

Lovasz Softmax loss explanation

I would like to use Lovasz softmax for foreground background semantic segmentation because of its ability to improve segmentation with Jaccard index according to paper. I got the idea that its a ...
0
votes
1answer
44 views

Reducing Validation loss for Triplet Loss Embeddings

I'm trying to create a facial recognition detector using triplet loss followed by a kNN algorithm. I have roughly 10000 input images with 3 different classes, input size is 80x80. Model structure uses ...
1
vote
1answer
17 views

Significance of Convex Loss Function with Nonlinear Models

When used in a linear model, a convex loss function guarantees a unique global minimum for the parameters, which can be found by local optimization methods. However, when the model is nonlinear (e.g. ...
0
votes
1answer
15 views

why the accuracy result and the loss result of an ANN model is inconsistent?

I trained a model based on an ANN and the accuracy is 94.65% almost every time while the loss result is 12.06%. Now my question is shouldn't the loss of the model be (100-94 = 6%) or near it? Why it ...
1
vote
1answer
24 views

Why would the accuracy of a model change when the loss doesn't?

I've trained 8 models based on the same architecture (convolutional neural network), and each uses a different data augmentation method. The accuracy of the models fluctuates greatly while the loss ...
0
votes
0answers
22 views

Inverting a matrix using a convolutional neural network

Just for a fun exercise, I am trying to invert a matrix, say size 28x28 (or even 5x5) with a neural network. The way I approached this (quite naively) is as follows: I built a fully convolutional ...
0
votes
0answers
38 views

Custom keras callbacks and changing weight (beta) of regularization term in variational autoencoder loss function

The variational autoencoder loss function is this: Loss = Loss_reconstruction + Beta * Loss_kld. I am trying to efficiently implement Kullback-Liebler Divergence Cyclic Annealing--that is changing the ...
1
vote
0answers
12 views

Unable to train neural network for prediction

My data consists of a time series of values $\pm1$ and I am trying to apply a RBF NN as a function approximator. Essentially, the NN will take as input one data sample and predict the next sample (...
0
votes
0answers
15 views

Optimizing the Loss Function For Another Metric

Suppose I have a machine learning model which is used to improve the profitability of a business. One of the components of the model is a loss function, say for measuring the success of a ...
0
votes
0answers
22 views

Negative loss values for adaptive loss in tensorflow

I have used adaptive loss implementation on a neural network, however after training a model long enough, I am getting negative loss values. Any help/suggestion would be highly appreciated! Please let ...
0
votes
1answer
119 views

Can someone explain the solution to the following problem?

Q) We want to learn a function f(x) of the form f(x) = ax + b which is parameterized by (a, b). Using squared error as the loss function, which of the following parameters would you use to model this ...
1
vote
1answer
45 views

Loss drops to NaN after a short time for a time series classification

here is my model code for a binary classification of a time series: ...
1
vote
0answers
39 views

Which Neural Network or Gradient Boosting framework is the simplest for Custom Loss Functions?

I need to implement a custom loss function. The function is relatively simple: $$-\sum \limits_{i=1}^m [O_{1,i} \cdot y_i-1] \ \cdot \ \operatorname{ReLu}(O_{1,i} \cdot \hat{y_i} - 1)$$ With $O$ being ...
3
votes
0answers
26 views

How to train a neural network where computing the loss requires multiple object values?

I want to train a function that given metadata about an image produces hyper-parameters for an algorithm which operates on the image. My understanding is (please forgive me I'm a novice here) a neural ...
0
votes
1answer
11 views

Problem in convergence of hebbian learning approach for Fuzzy Cognitive Map

I was trying to learn Fuzzy Cognitive Map by Active Hebbian Learning approach from here. What I have understand is that the model learns iteratively, at each step a new concept values enters and tune ...
0
votes
1answer
99 views

Keras custom loss function with weight function

My LSTM neural network predicts nominal values between -1 and 1. I would like to set up a custom loss function in Keras that assigns a weight function depending on the predicted sign. If the predicted ...
1
vote
0answers
97 views

Worse performance after Hyperparameter tuning

I first construct a base model (using default parameters) and obtain MAE. ...
0
votes
1answer
118 views

SparseCategoricalcrossEntropy(from_logits=True) internally apply softmax?

Regarding Tensorflow/Keras SparseCategoricalcrossEntropy. SparseCategoricalcrossEntropy(from_logits=True) expects the logits that has not been normalized by softmax....
1
vote
1answer
86 views

Difference in performance Sigmoid vs. Softmax

For the same Binary Image Classification task, if in the final layer I use 1 node with Sigmoid activation function and ...
2
votes
2answers
133 views

How to interpreter Binary Cross Entropy loss function?

I saw some examples of Autoencoders (on images) which use sigmoid as output layer and BinaryCrossentropy as loss function. The ...
0
votes
0answers
12 views

Why checkpoint loss is different?

I am training a Mask RCNN model in Keras. I used checkpoints to save weights so I can resume training with the last optimized values. However, the loss is different when I save the checkpoint and ...
0
votes
0answers
50 views

How to incorporate multi-task in CTR/recommendation model (deep & wide/ xDeepFM etc)?

I am building a rank algorithm for an e-commerce website that ranks the product based on likely hood of purchase and I have formulated this problem into a binary classification problem. Given each ...

1
2 3 4 5
10