# Backpropagation with a different sized training set?

I'm trying to create a NN whose input is a (length m) array of 3d vectors $$\vec{x}_i = [x_{i,1},x_{i,2},x_{i,3}], \hspace{5mm}i=1:m$$

and whose output is a similarly sized array:

$$\vec{h}_{\theta,i} = [h_{\theta,i1},h_{\theta,i2},h_{\theta,i3}], \hspace{5mm}i=1:m$$

BUT, my only training data is not 3d vectors but rather the magnitude/norm of such vectors (with no knowledge of the vector components ($$\lambda's$$) themselves):

$$y_i= ||[\lambda_{i,1},\lambda_{i,2},\lambda_{i,3}]||, \hspace{5mm}i=1:m$$

So, my concept is to use the cost function:

$$J = \frac{1}{2m}\sum (||\vec{h}_{\theta,i}|| - ||y_i||)^2$$

Note, the difference between this and the more usual quadratic cost function ($$J = \frac{1}{2m}\sum (\vec{h}_{\theta,i} - \vec{y}_i)^2$$) which I would use if $$y_i$$ was the same shape as the output.

In the typical cost function case I've mentioned above the backpropagation algorithm always starts with computing the output layer error (which based on the simple derivative of that cost function is just $$\delta^L = a^L - y_i$$). From there you follow to the next layer error which depends on $$\delta^L$$.

My problem is what should my output layer error be? I've tried just using the derivative of $$J$$ with respect to $$a^L$$ using my new cost function (I won't write out that derivative here because it's a bit hairy), but I can't seem to simply slot that in as my output error and continue on with normal back propagation with any kind of sensible results?

If you can't suggest how I should approach my BP algorithm, then perhaps it's because you think my cost function is a non-starter? As a note: I have sort of proven the concept to myself using a linear model rather than an NN and training it with this dataset and linear derivatives rather than BP. The accuracy was low though given the nonlinearity of the data so I would like to use an NN approach.

• Thanks for your response. To answer your first question, no my aim is not simply to predict the magnitude, it is to predict the three components of the vector, but using magnitude data, which I am confident is possible given my success training a linear model like I alluded to in the post (i.e. where I used $\vec{h}_{i} = A\vec{x}_i + b$ and trained for A and b rather than an NN training for $\theta$). Could you explain to me in more detail how this should be possible using MSE? The basic MSE function should imply $y$ is the same shape as $h$, but here I'm using 3d $h$ and 1d $y$? Apr 10, 2020 at 20:29