I am running a 1D CNN on tabular data. The rows are data that I have are not sequential, that is to say they are not part of a time series or ordered string, which is why I am not using an LSTM.

So when the filter looks at both row n and row n+1, does it learn from them as if they are spatially collocated or dependent upon each other? Is there a way to prevent this from happening?

Alternatively, is there a better way to look at 2 dimensional data (rows of features) as independent rows?

enter image description here


Yes, the filter will learn as if they are spatially co-located.

The main purpose of convolutions is to detect local features, where the notion of locality comes from the positions over which the filter is applied.

Some neural network building blocks that you could use are:

  • Position-wise dense layers: this applies a linear transformation to each of the elements. The result would be an element for each of the elements in the input. The dimensionality depends on the matrix used for the multiplication. This can be achieved also by applying a 1D convolution with a filter of length 1.

  • Max/avg. pooling over time: taking the maximum/average value over all elements in the input, separately for each input channel. This gives you a single element, as it collapses all the elements in one. A variation is to take the $k$ maximum elements.

Of course, you may want to exploit the relations between the elements in your input, but not in their original meaningless ordering: you could sort your inputs (from lowest to highest or vice versa). Then you could apply convolutional layers.

  • $\begingroup$ Thank you "Position-wise feed forward accepts a 3 -dimensional input with shape (batch size, sequence length, feature size). The position-wise FFN consists of two dense layers that applies to the last dimension. Since the same two dense layers are used for each position item in the sequence, we referred to it as position-wise. Indeed, it is equivalent to applying two 1×1 convolution layers." d2l.ai/chapter_attention-mechanisms/transformer.html $\endgroup$ – HashRocketSyntax Jun 22 '20 at 1:17

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.