# Unordered Set Classification Problem

In my setup I have one feature which is a sparse list representing categories. For example, let's say that we have M categories in the interval [0 .. M - 1]. My examples look like:

• ex 1: [0, 2, 1]
• ex 2: [3, 0]
• ex 3: [1, 4, 5]

The order doesn't matter. So, let's suppose that we already know $$f: s \rightarrow L$$, where $$s$$ is the sequence and $$L$$ is the label, for example, lets consider just a toy function $$f$$ (in a real setting I don't know what $$f$$ looks like):

def toy_func(set):
return int(
0 in set
and M // 2 in set
)


So, I am trying to train a neural network, currently I have tried the following model:

class SimpleMultiCategoricalClassifier(torch.nn.Module):
def __init__(self, num_embeddings, embedding_dim):
super().__init__()

self.embeddings = nn.EmbeddingBag(
num_embeddings=num_embeddings,
embedding_dim=embedding_dim, mode="mean")

self.net = nn.Sequential(
nn.Linear(embedding_dim, 1),
nn.ReLU(),
nn.Sigmoid(),
)

def forward(self, input, offset):
x = self.embeddings(input, offset)
return self.net(x)


Basically I represent each "category" as an embedding, and then do pooling on the entire sequence to learn the function.

However my model is not learning anything. Is there a better way to frame the problem so I can learn the latent function?

Typical setup for this would be to one hot encode each unordered set, so we have $$M$$ boolean indicator variables. And then we might train on cosine distance, to predict each label.