2
$\begingroup$

In Distilling a Neural Network Into a Soft Decision Tree the authors mention

exponentially decaying running average of the actual probabilities with a time window

but provide no formula for it.

What does it mean speaking formal?

$\endgroup$
1
  • $\begingroup$ Interesting paper. Note that in this question you are referring to how the tree is built in order to help distile the knowledge in a NN $\endgroup$ Dec 19 '20 at 9:50
1
$\begingroup$

This is likely the Exponential Moving Average function. In simplest form it is

ma_new = alpha * new_sample + (1-alpha) * ma_old

Where alpha is the parameter that controls rate of decay, between 0.0-1.0.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.