I am wondering if each feed-forward neural network can be written as a composition of the layer functions.
So if $F(x)$ is the output of a neural network F, do always exists layer functions $f_1$,...,$f_n$ with $F(x)$ = ($f_1$ o .. o $f_n$)$(x)$? Here "o" denotes the composition symbol.
If the network consists only of layers where each layer is connected to exactly one previous layer, this is obvious. However, this is not always the case, e.g. if you have skip-connections.
So what about the "concat" layer and the "sum" layer?
So, what if $F(x)$ = $G_1(x) + G_2(x)$, or $F(x) = [H_1(x), H_2(x)]$, for some layer functions $G_1$,$G_2$,$H_1$,$H_2$? Can $F$ still be written as a composition?