Wrangling data for CNN

Question

I am using convolutional nets for a physics application. I am trying to figure out how to structure my raw data as an image for input into the CNN.

I have $N$ samples. Each sample consists of the following:

(1) Temperature values at discrete times and at discrete spatial locations. I.e., temperature is a function of time, $t$, and space, $(x,y,z)$.

(2) TBD, but it should consist of some other spatial and temporal data. I haven't figured this out yet. For this question and for simplicity, we could just limit it to working with (1) only.

The examples that I've looked at deals with images, and they're inherently 3D. e.g., 100 pixels wide by 50 pixels high by 3 RGB channels. So the input can be structured as a 3-D tensor.

In my case, temperature depends on 4 independent variables (3 for space and 1 for time). What would be the best way to wrangle it for use as input into a CNN? A 4-D tensor? Or would it be better to map the 4-D tensor to a lower-order tensor, say, 3-D or 2-D?

Also, somewhat of a side note, I am a little bit confused on how an input image relates to the number of nodes in the input layer of the NN. Say, we have an image that is 100 pixels wide by 50 pixels high with 3 channels. Does this mean there should be 100x50x3=15,000 nodes in the input layer?

I'm relatively new to this, so I don't really know what I'm doing. Any insight would be greatly appreciated.

-----------------Edit 1---------------

After some thinking, this is one format that I'm considering. This format represents my temperature values in an image-like matrix: $$ \begin{bmatrix} T(x_1,y_1,z_1,t_1) & \cdots & T(x_1,y_1,z_1,t_n) \\ \vdots & \ddots & \vdots \\ T(x_m,y_m,z_m,t_1) & \cdots & T(x_m,y_m,z_m,t_n) \\ \end{bmatrix} $$ where $m$ is the total number of locations and $n$ is the number of discrete time intervals. Each column is constant in time, but varying in space. Each row is constant in space, but varying in time. The matrix elements closest to each other (i.e., closer in space and time) have temperature values that are similar, and elements further from each other have temperature values that are less related.

In this format, the number of channels would be 1. And my input into the first convolution layer would be $m \times n \times 1$. Does this format fit the definition of a 2-D topological order? It seems to me that it does since this matrix abstraction is structured such that elements close to each other should be more similar than the elements further apart (see bolded statement above).

Another format I am considering is similar to the one suggested by Antonio in his answer. In this format, the number of channels would be equal to the number of discrete times, so $n$, and my "image" becomes a 1-D column (or I guess it could be a row vector too): $$ \underbrace{\begin{bmatrix} T(x_1,y_1,z_1,t_1) \\ \vdots \\ T(x_m,y_m,z_m,t_1) \\ \end{bmatrix}}_{\text{channel 1}}, \ \ \underbrace{\begin{bmatrix} T(x_1,y_1,z_1,t_2) \\ \vdots \\ T(x_m,y_m,z_m,t_2) \\ \end{bmatrix}}_{\text{channel 2}}, \ldots ,\underbrace{\begin{bmatrix} T(x_1,y_1,z_1,t_n) \\ \vdots \\ T(x_m,y_m,z_m,t_n) \\ \end{bmatrix}}_{\text{channel n}}, $$ So each sample input is now $m \times 1 \times n$, and for each channel we have a $m \times 1$ column vector.

Answer 1

When you use a 2D-convolutional layer, you assume that there is some 2D topological order in your data. E.g. images have 2D topological order: it's important that some pixel is at position (x, y) - if we move that pixel to some other position, the image would be wrong. The number of channels can be arbitrary in images (1 for grayscale, 3 for RGB), but we still consider this as a 2D topological order.

Check the Table 9.1. at chapter 9.7. Data Types (Deep Learning book by Ian Goodfellow and Yoshua Bengio and Aaron Courville).

So, since your data doesn't have 2D topological order, but 1D, you should use 1D-convolutional layer with multiple channels.

You could have 4 channels: 1 for time, and 3 other for space coordinates.