Questions tagged [cross-entropy]

The tag has no usage guidance.

Filter by
Sorted by
Tagged with
0 votes
0 answers
10 views

Problem when merge two probability distribution [Pointer Network]

I'm trying to re-implement Pointer Gen Net from this paper. Ya but you don't really need to read the paper. To sum up briefly, I have a vector probability distribution over vocabulary called p_vocab. ...
jupyter's user avatar
  • 101
0 votes
0 answers
22 views

Binary classification penalizing only one class

Let's suppose that the task is the following binary classification: Does this image have a cat or a dog? These are the four possible predictions: It predicts cat and the image has cat. It predicts ...
S Energeiakos's user avatar
0 votes
0 answers
10 views

Cross-Entropy Loss for Prediction Distribution

I'm trying to adapt an example from the Neural Tangents Cookbook which uses a variant of Mean Squared Error (MSE) loss suited for Regression prediction distributions given by Mean and Variance as ...
Krailon's user avatar
2 votes
0 answers
49 views

Why is cross-entropy increasing with accuracy?

I'm making an implementation of the softmax regression and I'm struggling to understand the nature behind the problem of increasing value of Cross-Entropy: $H(y_i, p_i)=-\sum_{i=1}^C y_i log(p_i)$, ...
JoshJohnson's user avatar
1 vote
0 answers
36 views

Analysis of relationship between accuracy and total loss (or cost) during training with logistic loss function and threshold 0.5

I'm trying to understanding the relationship between training accuracy and training loss in classification tasks, specifically using logistic regression. When using logistic loss as the loss function ...
Tran Khanh's user avatar
0 votes
0 answers
49 views

Doubts on a custom loss function for regression problems

From what I read, I know we don't use log loss or cross entropy for regression problems. However, the entire logic behind binary cross entropy(say) is to firstly squeeze the y_hat between 0 and 1 (...
the_he_man's user avatar
0 votes
0 answers
12 views

tensorflow binary_crossentropy gives unexpected(possibly wrong) results

...
CouldntLoginToMyPreviousAcc's user avatar
0 votes
0 answers
13 views

Loss function for classifcation rewarding closer guess?

The default loss function in multi class classification is cross_entropy, which treats all wrong guesses equally. If the distance between buckets are meaningful, for example, given the real bucket is ...
jerron's user avatar
  • 101
0 votes
0 answers
31 views

Is there a canonical cross entropy from the confusion matrix?

In Wu, MT. Confusion matrix and minimum cross-entropy metrics based motion recognition system in the classroom. Sci Rep 12, 3095 (2022). https://doi.org/10.1038/s41598-022-07137-z the author uses a ...
arivero's user avatar
  • 101
0 votes
0 answers
49 views

Train neural network to predict multiple distributions

I aim to train a neural network to predict 2 distributions (10 quantiles, i.e. deciles) at 5 time points. So my y is of shape: ...
A_Murphy's user avatar
1 vote
1 answer
578 views

PyTorch CrossEntropyLoss and Log_SoftMAx + NLLLoss give different results

As per PyTorch documentation CrossEntropyLoss() is a combination of LogSoftMax() and NLLLoss() function. However, calling CrossEntropyLoss() gives different results compared to calling LogSoftMax() ...
cbelwal's user avatar
  • 113
0 votes
1 answer
907 views

Why is the calculated cross-entropy not zero?

...
Tsiolkovsky's user avatar
0 votes
1 answer
374 views

Why is cross entropy loss averaged and not used directly as a sum during model training(such as in neural networks)

Why is the cross entropy loss for all training examples(or the training examples in a batch) averaged over size of the training set(or batch size) ? Why is it not just summed and used ?
user17420392's user avatar
0 votes
0 answers
559 views

Getting Error: TypeError: cross_entropy_loss(): argument 'target' (position 2) must be Tensor, not tuple

I am working on a CNN multi-class classification of different concentrations (10uM, 30uM, etc.) I create my dataset to include the images as the features and the concentrations as labels. Note that ...
Zelreedy's user avatar
0 votes
0 answers
192 views

Can we use MSELoss and CrossEntropyLoss alongside?

Can we apply both MSELoss and CrossEntropyLoss in a single network to predict both classification and regression in Deep Learning? Suppose that we have 4 points(regression) and 5 classes(...
Mahdi Amrollahi's user avatar
0 votes
1 answer
351 views

How to understand a large result of torch.nn.NLLLoss() with correct predicts?

I'm learning the usage of torch.nn.NLLLoss() and torch.nn.LogSoftmax(), and I'm confused about the results of them. For example: ...
EvilRoach's user avatar
  • 101
0 votes
1 answer
56 views

Connecting timeseries quantities to CDF

In the following paper, ...
Omar Shehab's user avatar
1 vote
1 answer
2k views

Loss on whole sequences in Causal Language Model

I'd like to know, from an implementation point of view, when training a Causal Transformer such as GPT-2, if making prediction on whole sequence at once and computing the loss on the whole sequence is ...
Valentin Macé's user avatar
1 vote
0 answers
19 views

Loss function for classification problem

So I'm working on a classification problem, I used convolutional neural networks to classify grayscale ECG beat images of dimension 200x200 (I had around 4000 images for each class in training and I ...
imene's user avatar
  • 23
0 votes
1 answer
372 views

How to compute the Gini index, the entropy and the classification error from a decision tree?

How to find the Gini index, the entropy, and the classification error for each node of the tree in the figure below. Please help me to compute them.
Nezuko's user avatar
  • 21
0 votes
1 answer
2k views

neural network binary classification softmax logsofmax and loss function

I am building a binary classification where the class I want to predict is present only <2% of times. I am using pytorch The last layer could be logosftmax or <...
user2543622's user avatar
1 vote
0 answers
32 views

Shannon Information Content related to Uncertainty?

I'm a data scientist student currently writing my master thesis which resolves around the Cross Entropy (CE) Loss Function for neural networks. From my understanding, the CE is based on the Entropy, ...
xflashx's user avatar
  • 11
1 vote
2 answers
965 views

Why is cross entropy based on Bernoulli or Multinoulli probability distribution?

When we use logistic regression, we use cross entropy as the loss function. However, based on my understanding and https://machinelearningmastery.com/cross-entropy-for-machine-learning/, cross entropy ...
Feng Chen's user avatar
  • 207