I'm trying to get grasp of Bayes regularization algorithm.
List of symbols 1st:
$F$ - objective function
$\gamma$ - regularization parameter
$M$ - number od neural network weights
$N$ - number of data tuples
$e$ - modeling error
$w$ - net weight
$D$ - data set input-output pair
$H$ - Hess matrix of objective function computed based on Jacoby's matrix
I get it that minimalising an objective function:
$$
F = \gamma\sum\limits _{j=1} ^Mw_j^2 + (1-\gamma)\sum\limits_{i=1}^Ne_i^2
$$
is equivalent to maximising likelihood $P(w|D,\gamma)$, which can be computed according to Bayes' theorem:
$$
P(w|D,\gamma) = \frac{P(D|w,\gamma)P(w|\gamma)}{P(D|\gamma)}
$$
Likelihood $P(w|\gamma)$ is assumed to be a gaussian one and can be computed as:
$$
P(w|\gamma)=\left(\frac{\gamma}{2N}\right)^{\frac{M}{2}}\cdot e^{-\frac{\gamma}{2}w^Tw}
$$
Likelihood $P(D|\gamma)$ can be computed too as:
$$
P(D|\gamma)=\left(\frac{\pi}{\gamma}\right)^{-\frac{N}{2}}\left(\frac{\pi}{1-\gamma}\right)^{-\frac{M}{2}}\frac{(2\pi)^\frac{M}{2}\cdot -e^{-F(w)}}{\sqrt{|H|}}
$$
And now my questions are: how is $P(D|w,\gamma)$ computed? Or maybe is there any special assumptions about it? And should I minimalise or maximise $P(w|D,\gamma)$ to minimalise objective function $F$.
I base on those articles:
Article1
Article2
If anyone is able to explain those thing to me, I'd be most grateful.
Thank you,
Max