I am addressing a problem of multivariate regression by using a CNN. In particular, I have a data set of artificial images which have been generated by a physical model which takes in input, suppose, 4 parameters.
Now, I am using a CNN in order to estimate these 4 parameters from images. Three of them are pretty well retrieved except one, whose predictions are quite bad. So, we can imagine to exclude it from the learning formalization and set up a regression with only 3 parameters, such that the last one will not 'perturb' the learning anymore.
Interestingly, when estimating only 3 parameters, the performance of the network is worse than in case of estimation of 4 parameters. So I was wondering if the fourth 'noisy' parameters has given, in the first case, some 'clues' to the network, even if it's an output parameter. We may say that, even if that parameter represents an output variable, its presence will modify the structure of the network, affecting the paths of the other 3 output neurons. Yet, I cannot provide any formal demonstration of that, and I don't know if that reasoning can make any sense. I was trying to interpret the results, which yet are counterintuitive.
Do you have any explanation of that phenomenon? Can that output variable, in your opinion, constitute a possible 'hint' for the network?