# Differentiation of Learning Capabilities of Different Networks

I have a conceptual problem regarding the overall learning capability of a neural network differentiated by the different types of input that we can give to the network. Suppose that we have a feedforward neural network. This network has only one output. We can give several different kinds of inputs, $$I_k$$, to the network in which $$k$$ denotes the different kinds of inputs that we have. Just to be clear, by input I don't mean the nodes which come before the layer containing $$x_i$$ but rather a function which the network receives as an input. to be more clear, suppose that each $$x_i$$ is a function of $$I_k$$.

We know that in order for the network to learn, backpropagation algorithm should work efficiently. Suppose that we fix the output $$y=y_{0}$$. We calculate the possible values for $$x_i$$ which can produce the output $$y_0$$ using the input $$I_1$$ first and the using another type of input $$I_2$$. Now we increase the desired output which we use to construct possible $$x_i$$ values by $$\Delta y$$ so that $$y_1=y_0 + \Delta y$$ and do the same calculations again. The idea that I have in my mind is as follows:

In order for the packpropagation to work efficiently, any change for the desired output $$y_1 - y_0 = \Delta y$$ should be reflected to the previous layer. Assume that we have a large number of $$x_i$$ (not just 5). We find $$x_i^{(1)}$$ and $$x_i^{(2)}$$, corresponding to the output $$y_1$$ and $$y_2$$ respectively and substract their norms:

$$\Delta\bar{x}=|\bar{x}^{(2)}|-|\bar{x}^{(1)}|$$

in which $$\bar{x}$$ denotes the vector of $$x_i$$. Can we say that if for the input $$I_1$$ the value for $$\Delta\bar{x}$$ is bigger than that for $$I_2$$, then the network has a better learning capacity using the input $$I_1$$ than the $$I_2$$? Basically the idea is to find an overall reflection of the changes of the output in the previous layer and use it as a criteria to differentiate between the learning capabilities of the networks. Any comment is highly appreciated.

• If the picture you used doesn't describe your problem you should use a different picture. It sounds like you have y(x(I)) and you want to find dI/dy. Are you defining learning capacity as $$\Delta\bar{x}$$? Sep 26 at 13:09
• I have updated the question if you are interested. All I want is a criteria which can decide for what input I can train the network more efficiently. Sep 26 at 13:25