Questions tagged [entropy]

The tag has no usage guidance.

Filter by
Sorted by
Tagged with
0 votes
0 answers

Is all entropy calculation an estimation in practice?

Self information is defined by the $-\log$ the probability. However, how do we know the probability distribution in the first place? (Same goes for Shannon Entropy) Does this mean that in practice all ...
Danny  Han's user avatar
  • 101
1 vote
0 answers

Model uncertainty quantification

I'm reading a paper about model uncertainty quantification. Specifically, it says epistemic uncertainty is a kind of uncertainty due to lack of knowledge about a particular region in the input space. ...
piero's user avatar
  • 61
0 votes
0 answers

How many parameters are necessary before stable diffusion "works"?

Stable diffusion is essentially removing entropy from noisy images. If you have $n$ latent dimensions, and add Gaussian noise independently to each dimension, then the added entropy is on the order of ...
programjames's user avatar
1 vote
1 answer

How to use labels to fit several thresholds in a simple decision rule?

I have a binary labelled dataset with numeric features. I want to create a "business rule" of the type y = x1 > t1 and x2 > t2 and x3 > t3. ...
hipoglucido's user avatar
  • 1,170
112 votes
9 answers

When should I use Gini Impurity as opposed to Information Gain (Entropy)?

Can someone practically explain the rationale behind Gini impurity vs Information gain (based on Entropy)? Which metric is better to use in different scenarios while using decision trees?
Krish Mahajan's user avatar