0
$\begingroup$

I have a Deep Neural Network that takes $n$ inputs $X = [X_1, \ldots, X_n]^T$ and gives $n$ ouputs $Y = [Y_1, \ldots, Y_n]^T$. Normally, I can just do a standard deep neural network with a few fully connected hidden layers. However, I want to build into the network the fact that there should be permutational symmetry. That is to say, if $\pi(\cdot)$ is a particular permutation, then $\pi(X)$ should output $\pi(Y)$. So each input should be treated symmetrically in some sense.

A simple example would be I get $n$ IID sensor readings, and I want to build an auto-encoder for these sensor readings.

Is there some way of building this symmetry into the neural network, perhaps through some parameter sharing or a special architecture?

$\endgroup$
  • 1
    $\begingroup$ What you're talking about here is the neural net being equivariant under the permutation action, so maybe you could try adapting some techniques from group equivariant convolutional neural networks. $\endgroup$ – Alexander Gruber Nov 21 '19 at 3:24
0
$\begingroup$

This might not be what you're looking for, but you can use data augmentation to do this: from each sample, you can generate multiple samples by cloning it, generating a random permutation and permuting the input and the expected output (for an autoencoder, this should achieve the same goal). This doesn't guarantee complete permutation invariance but if you do it enough times for each sample it should get you pretty close.

| improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.