How do I implement convolution partially on my dataset?

Question

I'm training a neural network on the results of a CFD simulation (or rather, 300-ish simulations with different initial conditions). The dataset contains the values for temperature, density, velocity, etc. at equidistant points on a known line segment (along a single spatial dimension) at each tick of the simulation. I also have some "metadata" i.e. some simulation parameters in each instance of the simulation. (like the initial conditions, the environmental conditions, etc.).

From what I understand, this is similar to image data (except it's 1D and we have physical quantities instead of RGB values); perhaps, I should melt the dataset and even use time as a second "dimension" rather than a column/feature. And from what I understand, a convolutional architecture might be the best option for such spatial/spatiotemporal data. I'll provide a simplified example of my dataframe at the bottom of this post.

In either case, my dataframe will have some non-spatial columns which are not to be used in the convolution (I'd probably use some fully connected Dense neurons for them), and others which are to be used (I'd probably use some Conv1D or Conv2D neurons for them). How do I build this kind of architecture on TensorFlow? I don't think the Sequential API can have parallel branches with different behaviours, right?

A simplified example:

--------------------------------------------------------------------------
| Time | Initial conditions | T1 | T2 | T3 | V1 | V2 | V3 | D1 | D2 | D3 |
|------|--------------------|----|----|----|----|----|----|----|----|----|
| 1    | 100                | 20 | 21 | 22 | 3  | 5  | 6  | 10 | 10 | 11 |
| 2    | 100                | 19 | 21 | 21 | 4  | 6  | 7  | 10 | 9  | 10 |
| ...  | 100                | ...                                        |

x1 is the value of the quantity x at a known point 1. Imagine hundreds of such CSV files each having different initial conditions (these are also numerical, not categorical). I can load them and join them melt columns into rows or vice-versa, no problem, I know the syntax for that. But TL;DR my 2 questions are:

1. Should I melt the time column into separate columns for each timestep (assuming I have sufficient computational power to handle such a wide dataset)?
2. How do I convolve the temperature columns, and the velocity columns, and the density columns (and perhaps the timestep columns), while leaving the initial conditions as independent inputs?

Sorry. Kinda new to neural network coding, especially convolutional ones. If there are any helpful resources for a newbie, please send them my way! I've only worked with Dense layers and Sequential architectures before. And no heterogeneous layers.

Answer 1

If you are using Conv1D, you will need a data structure that can be mapped onto a 3D tensor (such as a 3D numpy array). If you have some non-spatiotemporal variables that you want to process using dense layers, you will need these to be in a 2D structure. Say you have n simulations, each with m time steps, then you need to have two data structures. One will be an $n\times 1$ structure that holds the initial conditions. The other will be an $n\times m\times 9$ structure (assuming just the 9 spatiotemporal variables shown in your example) that holds the spatiotemporal data.

You can't use the Sequential API when you have multiple inputs, so you'll need to use the Functional API. Here's a minimal example of using the functional API to create a model with two inputs:

import tensorflow as tf
import tensorflow.keras as keras
from IPython.display import Image 

n = 10 # Simulations
m = 5  # Time steps
v = 9  # Spatiotemporal variables
i = 1  # Initial conditions variables

# Build the spatiotemporal branch

input1 = keras.Input((m, v), name="Conv_Input")
x1 = keras.layers.Conv1D(filters=8, kernel_size=3)(input1)

# Build the "initial conditions" branch

input2 = keras.Input(i, name="Dense_Input")
x2 = keras.layers.Dense(units=8)(input2)

# Merge the two branches

x1 = keras.layers.Flatten()(x1)
full = keras.layers.Concatenate()((x1, x2))
full = keras.layers.Dense(units=1, activation='sigmoid')(full)

# Build the full model and display the model structure

model = keras.Model(inputs=(input1, input2), outputs=full)
keras.utils.plot_model(model, show_shapes=True, show_layer_names=True, to_file='model.png')
Image('model.png')

This code produces the following model: