Questions tagged [cross-entropy]
The cross-entropy tag has no usage guidance.
20
questions
0
votes
0
answers
10
views
Loss function for classifcation rewarding closer guess?
The default loss function in multi class classification is cross_entropy, which treats all wrong guesses equally. If the distance between buckets are meaningful, for example, given the real bucket is ...
- 101
0
votes
0
answers
14
views
Is there a canonical cross entropy from the confusion matrix?
In
Wu, MT. Confusion matrix and minimum cross-entropy metrics based motion recognition system in the classroom. Sci Rep 12, 3095 (2022). https://doi.org/10.1038/s41598-022-07137-z
the author uses a ...
- 101
0
votes
0
answers
40
views
Train neural network to predict multiple distributions
I aim to train a neural network to predict 2 distributions (10 quantiles, i.e. deciles) at 5 time points. So my y is of shape:
...
- 30
1
vote
1
answer
111
views
PyTorch CrossEntropyLoss and Log_SoftMAx + NLLLoss give different results
As per PyTorch documentation CrossEntropyLoss() is a combination of LogSoftMax() and NLLLoss() function. However, calling CrossEntropyLoss() gives different results compared to calling LogSoftMax() ...
- 113
0
votes
1
answer
342
views
0
votes
1
answer
142
views
Why is cross entropy loss averaged and not used directly as a sum during model training(such as in neural networks)
Why is the cross entropy loss for all training examples(or the training examples in a batch) averaged over size of the training set(or batch size) ?
Why is it not just summed and used ?
0
votes
0
answers
329
views
Getting Error: TypeError: cross_entropy_loss(): argument 'target' (position 2) must be Tensor, not tuple
I am working on a CNN multi-class classification of different concentrations (10uM, 30uM, etc.) I create my dataset to include the images as the features and the concentrations as labels. Note that ...
- 13
0
votes
0
answers
79
views
How does cross-entropy loss change with the number of classes?
How does the value of the cross-entropy loss function vary with the number of classes being predicted?
Formally, if the loss function is
$$
L = - \sum_{x \in X} P^*(x) \log P(x)
$$
where $P^*(\cdot)$ ...
- 1
0
votes
0
answers
13
views
How backward() is calculated in CrossEntropyLoss?
I have a simple Linear model and I need to calculate the loss for it. I applied two CrossEntropyLoss and NLLLoss but I want to ...
- 263
0
votes
0
answers
117
views
Can we use MSELoss and CrossEntropyLoss alongside?
Can we apply both MSELoss and CrossEntropyLoss in a single network to predict both classification and regression in Deep Learning? Suppose that we have 4 points(regression) and 5 classes(...
- 263
0
votes
1
answer
255
views
How to understand a large result of torch.nn.NLLLoss() with correct predicts?
I'm learning the usage of torch.nn.NLLLoss() and torch.nn.LogSoftmax(), and I'm confused about the results of them.
For example:
...
- 101
0
votes
0
answers
13
views
Role of Expectation and (1-y) in GAN Equation
I am reading about the GAN equation and I was wondering why Expectation was used twice? As of my (lack of) understanding, Expectation is a generalization of the mean because there is an infinite ...
user141509
0
votes
0
answers
17
views
Predicted images are quite good with loss=0.20 while are black with loss=0.02
I'm trying to train a U-net with VGG16 as a backbone in order to recognize 4 classes: sky, rocks, trees and background in a dataset of about 10000 images.
I'm using categorical crossentropy as a loss ...
- 111
0
votes
1
answer
51
views
1
vote
1
answer
998
views
Loss on whole sequences in Causal Language Model
I'd like to know, from an implementation point of view, when training a Causal Transformer such as GPT-2, if making prediction on whole sequence at once and computing the loss on the whole sequence is ...
- 137
1
vote
0
answers
17
views
Loss function for classification problem
So I'm working on a classification problem, I used convolutional neural networks to classify grayscale ECG beat images of dimension 200x200 (I had around 4000 images for each class in training and I ...
- 23
0
votes
1
answer
304
views
How to compute the Gini index, the entropy and the classification error from a decision tree?
How to find the Gini index, the entropy, and the classification error for each node of the tree in the figure below.
Please help me to compute them.
- 21
0
votes
1
answer
2k
views
neural network binary classification softmax logsofmax and loss function
I am building a binary classification where the class I want to predict is present only <2% of times. I am using pytorch
The last layer could be logosftmax or <...
- 147
1
vote
0
answers
28
views
Shannon Information Content related to Uncertainty?
I'm a data scientist student currently writing my master thesis which resolves around the Cross Entropy (CE) Loss Function for neural networks. From my understanding, the CE is based on the Entropy, ...
- 11
1
vote
2
answers
656
views
Why is cross entropy based on Bernoulli or Multinoulli probability distribution?
When we use logistic regression, we use cross entropy as the loss function. However, based on my understanding and https://machinelearningmastery.com/cross-entropy-for-machine-learning/, cross entropy ...
- 207