3
$\begingroup$

I am reading Bishop's Mixture Density Network paper at: https://www.microsoft.com/en-us/research/wp-content/uploads/2016/02/bishop-ncrg-94-004.pdf

This is a good paper, but I am still confused about a few small details. I am wondering if anyone could give me some help:

  1. Basically the mixing coefficient alpha_i can be computed through a softmax function in eq (25) below. However, in eq (25), what's the upper alpha for each $(z_i)^{alpha}$? Is it a free parameter to be fitted?

  2. Similarly, in eq (26), what's the upper sigma in $(z_i)^{sigma}$? Is it a free parameter to be fitted as well? Thanks!

enter image description here

enter image description here

Improve this question
$\endgroup$
1
  • 1
    $\begingroup$ I edited the equations due to unreadability, please check them. $\endgroup$
    – Green Falcon
    Aug 6, 2018 at 21:37

1 Answer 1

Reset to default
2
$\begingroup$

Actually, the upper alpha and the upper sigma are not free parameters to be set, they are just used to represent the output activations corresponding to the mixture coefficients and the variances, respectively. They are used to distinguish derivatives with respect to the alpha and sigma. I say it from page 275 of the book “pattern recognition and machine learning” by Christopher Bishop”:

page 275 of the book “pattern recognition and machine learning” by Christopher Bishop”

Improve this answer
$\endgroup$
2
  • $\begingroup$ In your example, Does this mean that $(a_k)^{pi}$, $(a_{kl})^{mu}$, $(a_k)^{sigma}$ are just 3 different output variables? So does in my example, $(z_i)^{alpha}$ and $(z_i)^{sigma}$ are just two different output variables (one corresponds to the mixture coefficient, and the other one corresponds to sigma). The alpha and sigma are just variable index, am I understanding this correctly? $\endgroup$
    – Edamame
    Aug 6, 2018 at 23:16
  • 1
    $\begingroup$ @Edamame Yes, they are just variable index. $\endgroup$
    – pythinker
    Aug 7, 2018 at 10:32

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.