I am reading gail implementation code in openai baselines. they compute bernoulli entropy as one of the loss in adversary network loss function.
In their code, they implement bernoulli entropy as this:
def logsigmoid(a): '''Equivalent to tf.log(tf.sigmoid(a))''' return -tf.nn.softplus(-a) def logit_bernoulli_entropy(logits): ent = (1.-tf.nn.sigmoid(logits))*logits - logsigmoid(logits) return ent
also there is a reference of another openai implementation, it's the same code but I can't see any explanation to it.
I checked that the equation to compute bernoulli entropy is:
$ -p\log p - (1-p)\log(1-p)$
I think the later equation is the right way to compute bernoulli entropy, but the first one should be right too as it's written in openai's implementationn. I can't see any similarity, is there any relationship between these two expression?