# Questions tagged [cross-entropy]

The tag has no usage guidance.

23 questions
Filter by
Sorted by
Tagged with
10 views

### Problem when merge two probability distribution [Pointer Network]

I'm trying to re-implement Pointer Gen Net from this paper. Ya but you don't really need to read the paper. To sum up briefly, I have a vector probability distribution over vocabulary called p_vocab. ...
• 101
22 views

### Binary classification penalizing only one class

Let's suppose that the task is the following binary classification: Does this image have a cat or a dog? These are the four possible predictions: It predicts cat and the image has cat. It predicts ...
10 views

### Cross-Entropy Loss for Prediction Distribution

I'm trying to adapt an example from the Neural Tangents Cookbook which uses a variant of Mean Squared Error (MSE) loss suited for Regression prediction distributions given by Mean and Variance as ...
49 views

### Why is cross-entropy increasing with accuracy?

I'm making an implementation of the softmax regression and I'm struggling to understand the nature behind the problem of increasing value of Cross-Entropy: $H(y_i, p_i)=-\sum_{i=1}^C y_i log(p_i)$, ...
• 121
1 vote
36 views

### Analysis of relationship between accuracy and total loss (or cost) during training with logistic loss function and threshold 0.5

I'm trying to understanding the relationship between training accuracy and training loss in classification tasks, specifically using logistic regression. When using logistic loss as the loss function ...
• 155
49 views

### Doubts on a custom loss function for regression problems

From what I read, I know we don't use log loss or cross entropy for regression problems. However, the entire logic behind binary cross entropy(say) is to firstly squeeze the y_hat between 0 and 1 (...
12 views

...
13 views

### Loss function for classifcation rewarding closer guess?

The default loss function in multi class classification is cross_entropy, which treats all wrong guesses equally. If the distance between buckets are meaningful, for example, given the real bucket is ...
• 101
31 views

### Is there a canonical cross entropy from the confusion matrix?

In Wu, MT. Confusion matrix and minimum cross-entropy metrics based motion recognition system in the classroom. Sci Rep 12, 3095 (2022). https://doi.org/10.1038/s41598-022-07137-z the author uses a ...
• 101
49 views

### Train neural network to predict multiple distributions

I aim to train a neural network to predict 2 distributions (10 quantiles, i.e. deciles) at 5 time points. So my y is of shape: ...
• 30
1 vote
578 views

### PyTorch CrossEntropyLoss and Log_SoftMAx + NLLLoss give different results

As per PyTorch documentation CrossEntropyLoss() is a combination of LogSoftMax() and NLLLoss() function. However, calling CrossEntropyLoss() gives different results compared to calling LogSoftMax() ...
• 113
907 views

...
374 views

### Why is cross entropy loss averaged and not used directly as a sum during model training(such as in neural networks)

Why is the cross entropy loss for all training examples(or the training examples in a batch) averaged over size of the training set(or batch size) ? Why is it not just summed and used ?
559 views

### Getting Error: TypeError: cross_entropy_loss(): argument 'target' (position 2) must be Tensor, not tuple

I am working on a CNN multi-class classification of different concentrations (10uM, 30uM, etc.) I create my dataset to include the images as the features and the concentrations as labels. Note that ...
• 23
192 views

### Can we use MSELoss and CrossEntropyLoss alongside?

Can we apply both MSELoss and CrossEntropyLoss in a single network to predict both classification and regression in Deep Learning? Suppose that we have 4 points(regression) and 5 classes(...
351 views

### How to understand a large result of torch.nn.NLLLoss() with correct predicts?

I'm learning the usage of torch.nn.NLLLoss() and torch.nn.LogSoftmax(), and I'm confused about the results of them. For example: ...
• 101
56 views

### Connecting timeseries quantities to CDF

In the following paper, ...
1 vote
2k views

### Loss on whole sequences in Causal Language Model

I'd like to know, from an implementation point of view, when training a Causal Transformer such as GPT-2, if making prediction on whole sequence at once and computing the loss on the whole sequence is ...
1 vote
19 views

### Loss function for classification problem

So I'm working on a classification problem, I used convolutional neural networks to classify grayscale ECG beat images of dimension 200x200 (I had around 4000 images for each class in training and I ...
• 23
372 views

### How to compute the Gini index, the entropy and the classification error from a decision tree?

How to find the Gini index, the entropy, and the classification error for each node of the tree in the figure below. Please help me to compute them.
• 21
2k views

### neural network binary classification softmax logsofmax and loss function

I am building a binary classification where the class I want to predict is present only <2% of times. I am using pytorch The last layer could be logosftmax or <...
• 147
1 vote
32 views

### Shannon Information Content related to Uncertainty?

I'm a data scientist student currently writing my master thesis which resolves around the Cross Entropy (CE) Loss Function for neural networks. From my understanding, the CE is based on the Entropy, ...
• 11
1 vote