1
$\begingroup$

The formula for Information given by a data of occurring with probability p is: I=-log2 p

This formula gives the bits if information needed to know the outcome of the event.

This formula captures the intuition that the information needed to know the outcome of an event with probability 1 is 0 as we already know that outcome of the event.

So shouldn't the formula give the information as 0 for the event with probability 0 as we know the outcome of the event.

So shouldn't the graph of the I vs p be a symmetric as the values of p close to 1 will give similar information as the events with p close to 0?

$\endgroup$
  • $\begingroup$ The expectation of this quantity, denoted the entropy, behaves as you expect. $\endgroup$ – Emre Feb 18 '17 at 17:02

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.