We are trying to replace an existing physical model (8 inputs/7 outputs) with artificial neural networks. The physics behind the existing model is mainly thermodynamics of humid air for air conditioning, with some turbomachinery involved, which yields most likely complex functions between inputs and outputs.
One approach was already done: single output neural networks (10 NN with same # hidden layers but different parameters like batch size, # epochs, optimizer, etc). Then some sort of stacking ensembled was used: every predictions were used as new inputs for a single NN to predict a final value.
The accuracy is pretty well, however there are some test datapoints where the absolute errors are high enough to be worried about the predicted value (this could be used for example for air conditioning control strategies, so a bad prediction would result in uncontrolled system).
In order to improve accuracy some colleagues were suggesting to keep it simpler and perform just a multi-output regression with a single neural network.
From the mathematical point of view, I have following questions:
1) would a single output NNs stacking ensembled outperformed the multi-output single NN? 2) is the way of stack ensembling using NN a good approach? I saw some different techniques like arithmetic averaging the inputs
Thanks for your time! Regards