# How to build in symmetry of inputs into a Deep Neural Network?

I have a Deep Neural Network that takes $$n$$ inputs $$X = [X_1, \ldots, X_n]^T$$ and gives $$n$$ ouputs $$Y = [Y_1, \ldots, Y_n]^T$$. Normally, I can just do a standard deep neural network with a few fully connected hidden layers. However, I want to build into the network the fact that there should be permutational symmetry. That is to say, if $$\pi(\cdot)$$ is a particular permutation, then $$\pi(X)$$ should output $$\pi(Y)$$. So each input should be treated symmetrically in some sense.

A simple example would be I get $$n$$ IID sensor readings, and I want to build an auto-encoder for these sensor readings.

Is there some way of building this symmetry into the neural network, perhaps through some parameter sharing or a special architecture?

• What you're talking about here is the neural net being equivariant under the permutation action, so maybe you could try adapting some techniques from group equivariant convolutional neural networks. Nov 21, 2019 at 3:24
• Not quite sure if my reasoning is correct. If you permute the inputs and this will permute the outputs, then you should be able to represent the input-to-output behavior by a single transfer function from one input to another output. Dec 23, 2021 at 9:42

## 1 Answer

This might not be what you're looking for, but you can use data augmentation to do this: from each sample, you can generate multiple samples by cloning it, generating a random permutation and permuting the input and the expected output (for an autoencoder, this should achieve the same goal). This doesn't guarantee complete permutation invariance but if you do it enough times for each sample it should get you pretty close.