I'm using a CNN to model a problem that involves precise numerical values from a physical simulation.
After months of design/redesign and optimization, I've noticed that the majority of the "error" in this network is accumulated because of lower accuracy for smaller valued cases, whereas it seems to work fine when the values are numerically larger.
The input values are always normalized to $(0,1)$ when fed into the network.
The error is measured on the denormalized output values.
How can I make sure that the network learns these "small" values better?
I believe that the key problem lies in the fact that the network is "blinded" to the actual size of the I/O pairs because of the normalization process.
Is this an inherent problem with the normalization process, or is this a problem in how the network is designed?
The error is quantified using MAPE, in this case.