# Multi-label classification using output quantization

## Problem statement

It's a fact that in order to train the network for multilabel-dataset, a one-hot-vector output is usually used.

Example:

dog [1 0 0]

cat [0 1 0]

rabbit [0 0 1]

Consequently, we're increasing size of the weight matrix as well as the required training time.

Question: Is there an approach that we can use one output and quantize it for different classes?

Example:

0 <= output < =1

dog [0.0 - 0.33]

cat [0.33 - 0.66]

rabbit [0.66 - 1]

# Explanation

1) You are introducing an ordering in the classes that doesn't exist. Basically, with your example, you are saying that: "dog is close to cat", "cat is close to rabbit", but "dog is far from rabbit", which is an extra learning feature your network needs to learn.

For example, if the network outputs 0.35, then it is class cat but it is also close to dog and far from rabbit.

2) Another problem, that some results are not possible to be represented.

For example, what if you changed the range to:

0 <= output < =1

cat [0.0 - 0.33]

rabbit [0.33 - 0.66]

dog [0.66 - 1]

How will you represent that the same example from before ("it is class cat but it is also close to dog and far from rabbit.")? It become impossible.

With the previous representation you could have:

output = [0.5, 0.0, 0.5] # cat, rabbit, dog


how would you map that to the new range where cat = [0.0-0.33] and dog = [0.66-1.0]? Can we pick the middle (0.5)? No, because that is class rabbit.

# Conclusion

By doing that, you reduce the number of weights in the final layers (good), however, you introduce dependencies which needs to be learned and some results are not representable or interpretable.

• Thanks so much for the detailed explanation! – Tarlan Ahad Jan 4 at 12:43

Perform one label encoding and then normalise the column.