Questions tagged [entropy]
The entropy tag has no usage guidance.
5
questions
0
votes
0
answers
15
views
Is all entropy calculation an estimation in practice?
Self information is defined by the $-\log$ the probability. However, how do we know the probability distribution in the first place? (Same goes for Shannon Entropy)
Does this mean that in practice all ...
- 101
1
vote
0
answers
74
views
Model uncertainty quantification
I'm reading a paper about model uncertainty quantification. Specifically, it says epistemic uncertainty is a kind of uncertainty due to lack of knowledge about a particular region in the input space.
...
- 61
0
votes
0
answers
249
views
How many parameters are necessary before stable diffusion "works"?
Stable diffusion is essentially removing entropy from noisy images. If you have $n$ latent dimensions, and add Gaussian noise independently to each dimension, then the added entropy is on the order of ...
1
vote
1
answer
28
views
How to use labels to fit several thresholds in a simple decision rule?
I have a binary labelled dataset with numeric features. I want to create a "business rule" of the type y = x1 > t1 and x2 > t2 and x3 > t3. ...
- 1,170
112
votes
9
answers
137k
views
When should I use Gini Impurity as opposed to Information Gain (Entropy)?
Can someone practically explain the rationale behind Gini impurity vs Information gain (based on Entropy)?
Which metric is better to use in different scenarios while using decision trees?
- 1,221