How to build in symmetry of inputs into a Deep Neural Network?

Question

I have a Deep Neural Network that takes $n$ inputs $X = [X_1, \ldots, X_n]^T$ and gives $n$ ouputs $Y = [Y_1, \ldots, Y_n]^T$. Normally, I can just do a standard deep neural network with a few fully connected hidden layers. However, I want to build into the network the fact that there should be permutational symmetry. That is to say, if $\pi(\cdot)$ is a particular permutation, then $\pi(X)$ should output $\pi(Y)$. So each input should be treated symmetrically in some sense.

A simple example would be I get $n$ IID sensor readings, and I want to build an auto-encoder for these sensor readings.

Is there some way of building this symmetry into the neural network, perhaps through some parameter sharing or a special architecture?

Answer 1

This might not be what you're looking for, but you can use data augmentation to do this: from each sample, you can generate multiple samples by cloning it, generating a random permutation and permuting the input and the expected output (for an autoencoder, this should achieve the same goal). This doesn't guarantee complete permutation invariance but if you do it enough times for each sample it should get you pretty close.