I want to perform a regression with a Neural Network using (environmental) spatiotemporal data. They share a target variable and have the same dimensions (latitude, longitude and time) but they are expressed in different styles/data shapes and show different features that I want to use as predictors.
My assumption: it would improve the training if the networks "knows" that the dimensions are shared as the time and location are important for training.
My question is: how can I train a Neural Network with these separate inputs, while making sure the network "knows" that the dimensions belong together? Here are the two data shapes:
- Dataset 1 has the data on a regular grid/matrix that has the size "latitude x longitude x time x features", which means that the dimension information (latitude, longitude, time) is contained in the position within the grid. They are very similar to images used in e.g. image classification that have different features/channels (where the equivalent to my features are the RGB-channels), only that there's an additional time dimension. I'd like to keep this data structure so I'd be able to use techniques from image classification, like working with 2D/3D convolutional neural networks.
- Dataset 2 is a very sparse dataset with individual locations over time (point measurements), which is most efficiently displayed as a csv-style table, where the features and also the latitude, longitude and time dimensions are shown as individual columns. I'm reluctant to turn these sparse datapoints into a similar grid as the first dataset, as it would result in 99.9% NaNs and I'm not sure how the network would handle that, even with data imputation.
I'm aware that you can't "label" features for a neural network so that it "knows" where they belong together and that the process of scaling/normalization will further anonymize things. Is there any kind of trick I can employ to better connect the two separate inputs given that they share the same dimensions? And at the same time make clear that the features are not the same and it is just additional information/predictors? Is there a way I could approach this? Or is it simply not possible and I'd have to turn either of the two into the other data type?
So far the closest solution I've come across was to build a neural network with two separate inputs following this example (which is using tensorflow/keras in Python). Unless I misunderstand it, it feeds in a image/table batch per timestep, which seems to be a way to get around my problem if the only dimension were time, but I'm not sure how to account for longitude/latitude. There are also a few more posts like here but they just explain that you can use multiple inputs - they don't talk about how different inputs could work together/influence each other.