I'm attempting to use a neural network as a kind of interpolator for a high-dimensional function.
We're doing this to circumvent the need for a physical model that calculates this function exactly, but is expensive to run. We have a dataset of past runs of this physical model that we can now use as the basis for interpolation.
The input is a set of 28 parameter values.
The output we're trying to estimate consists of the shape of 432 periodic functions (i.e. value of 432 different functions along the circumference of a circle). We know that the output is uniquely determined by the 28 input values; there's no noise.
For example, here is one of these periodic functions. For this example, it's only sampled with 12 points (though in production we will be sampling it more finely, perhaps 36 points around a circle).
My current neural network Keras implementation is:
- Input layer: 28 units (one per input parameter)
- One hidden layer: fully connected, 1000 units
- Output layer: fully connected, 432 periodic functions × 12 samples = 5184 units
As illustrated above, this works okay (even before optimizing hyperparameters).
But I'm concerned that with these fully-connected layers, I'm treating the 5184 outputs as unrelated numbers, completely disjointed from one another. In reality, there's some spatial relationship between the 12 samples describing each of the 432 underlying functions in the output. But I'm just throwing this spatial information away.
Is there another kind of layer which could take advantage of this spatial relationship, and thus lend itself better to this kind of regression?
If I were going the other way round, i.e. from spatial information to distinct values, I would consider convolution layers (perhaps with a periodic boundary). But what about this reverse case?