I have a homework problem where the neural network below is given with its description. We have been asked "Can the functions that the above network computes also be computed by a network that contains only an input layer and an output layer that has a single node?" If we can, we're to provide the network, weights, and activation function. And if not, we have to explain why.

I am very unsure how to approach this problem as I am not that strong with neural networks. Any starting point would be very helpful.

Thanks in advance.

Image and Description of Neural Network from Problem


You are asked whether we can mimic the given neural network with a new network that contains only a input layer and an output layer, which in turn means whether $a_5$ can be represented as a linear combination of $x_1, x_2, x_3, x_4$. Since every action the neural net does is linear it can be represented as a smaller one without a hidden layer.

Let's calculate the weights

$$a_5 = (a_1*\theta_9 + a_2*\theta_10 + a_3*\theta_11 + a_4*\theta_12)*C $$ $$a_5 = ((x_1*\theta_1 + x_2*\theta_2)*\theta_9*C + (x_1*\theta_3 + x_2*\theta_4)*\theta_{10}*C + (x_3*\theta_5 + x_4*\theta_6)*\theta_{11}*C+ (x_3*\theta_7 + x_4*\theta_8)*C)*\theta_{12}*C $$ $$a_5 = C^2 * (x_1*(\theta_1*\theta_9+\theta_3*\theta_{10}) + x_2(\theta_2*\theta_{9}+\theta_4*\theta_{10}) + x_3(\theta_5*\theta_{11}+ \theta_7*\theta_{12}) + x_4(\theta_6*\theta_{11}+\theta_8*\theta_{12})) $$

So our weights are: $$<\theta_1*\theta_9+\theta_3*\theta_{10},\theta_5*\theta_{11}+ \theta_7*\theta_{12} , \theta_2*\theta_{9}+\theta_4*\theta_{10}, \theta_6*\theta_{11}+\theta_8*\theta_{12}> $$ and our activation function is multiplication by $C^2$

  • $\begingroup$ Thank you very much for your assistance, this helped me a lot! $\endgroup$ – user73533 May 2 '19 at 19:15

Yes, it is possible to represent the neural network with a single hidden layer. Computing the final result of $a_5$:

$a_5 = (\theta_9a_1+\theta_{10}a_2+\theta_{11}a_3+\theta_{12}a_4)*C$ (Equation 1)

$a_1 = (\theta_1x_1+\theta_2x_2)*C$ (Equation 2)

$a_2 = (\theta_3x_1+\theta_4x_2)*C$ (Equation 3)

$a_3 = (\theta_5x_3+\theta_6x_4)*C$ (Equation 4)

$a_4 = (\theta_7x_3+\theta_8x_4)*C$ (Equation 5)

Replacing Equations 2,3,4,5 into Equation 1:


$$a_5=(\theta_9\theta_1x_1+\theta_9\theta_2x_2+\theta_{10}\theta_3x_1+\theta_{10}\theta_4x_2+\theta_{11}\theta_5x_3+\theta_{11}\theta_6x_4+\theta_{12}\theta_7x_3+\theta_{12}\theta_8x_4)*C^2$$9 $$a_5 = (\theta_9\theta_1+\theta_{10}\theta_3)C^2x_1+(\theta_9\theta_2+\theta_{10}\theta_4)C^2x_2+(\theta_{11}\theta_5+\theta_{12}\theta_7)C^2x_3+(\theta_{11}\theta_6+\theta_{12}\theta_8)C^2x_4$$

Calling $F(\theta,x) = C*\sum\theta x$ the activation function (according to the definition), we reexpress the above equation with this definition:

$$a_5 = F([\theta_9\theta_1+\theta_{10}\theta_3,\theta_9\theta_2+\theta_{10}\theta_4,\theta_{11}\theta_5+\theta_{12}\theta_7,\theta_{11}\theta_6+\theta_{12}\theta_8],[x_1,x_2,x_3,x_4])*C$$

Which means we can define the result of the network with only one equation which depends on the activation function $F$.

The network is (I can't draw here, sorry):

$x_1$ $->$

$x_2$ $->$ O $->$ $Y$

$x_3$ $->$

$x_4$ $->$

The weight are:





Activation function:

$F(\theta,x) = C*\sum\theta x$, defined as C times the sum product of two vectors, $\theta$ and $x$


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy