basically I have this problem where I need to forecast the sales of some stores. such stores have multiple product lines of which I have the split data (say Y1,Y2,Y3 where Y1+Y2+Y3=Y). I have also some X that I will use to estimate Y (the stores aggregate sales). tabular data, no time series. Loss function is RMSE. I have 2 approaches possible:
- simply use aggregated sales to train -> single-output problem
- forecast all the product lines at the same time (in
pythonthis is easily doable via deep learning libraries) and then aggregate the split sales into the aggregated sales -> multi-output problem
in theory the second approach should be superior as it leverages a greater amount of information and therefore should be able to forecast more effectively the final result reducing the unexplained variance while the first problem should be way easier in terms of convergence (loss function) and should be orders of magnitudes faster to train.
In practice (my trials so far), the NN is taking forever to train and, as I am forced to increase the ephocs, I am a bit scared overfitting will be there.
given the information available, do you still think it's just a matter of tuning the NN for the multi-output thing or am I missing something major for which apparently the simpler approach works better ?