2
$\begingroup$

Problem statement

It's a fact that in order to train the network for multilabel-dataset, a one-hot-vector output is usually used.

Example:

dog [1 0 0]

cat [0 1 0]

rabbit [0 0 1]

Consequently, we're increasing size of the weight matrix as well as the required training time.


Question: Is there an approach that we can use one output and quantize it for different classes?

Example:

0 <= output < =1

dog [0.0 - 0.33]

cat [0.33 - 0.66]

rabbit [0.66 - 1]

$\endgroup$
4
$\begingroup$

TL;DR

Bad idea, don't do that.

Explanation

1) You are introducing an ordering in the classes that doesn't exist. Basically, with your example, you are saying that: "dog is close to cat", "cat is close to rabbit", but "dog is far from rabbit", which is an extra learning feature your network needs to learn.

For example, if the network outputs 0.35, then it is class cat but it is also close to dog and far from rabbit.

2) Another problem, that some results are not possible to be represented.

For example, what if you changed the range to:

0 <= output < =1

cat [0.0 - 0.33]

rabbit [0.33 - 0.66]

dog [0.66 - 1]

How will you represent that the same example from before ("it is class cat but it is also close to dog and far from rabbit.")? It become impossible.

With the previous representation you could have:

output = [0.5, 0.0, 0.5] # cat, rabbit, dog

how would you map that to the new range where cat = [0.0-0.33] and dog = [0.66-1.0]? Can we pick the middle (0.5)? No, because that is class rabbit.

Conclusion

By doing that, you reduce the number of weights in the final layers (good), however, you introduce dependencies which needs to be learned and some results are not representable or interpretable.

| improve this answer | |
$\endgroup$
  • 2
    $\begingroup$ Thanks so much for the detailed explanation! $\endgroup$ – Tarlan Ahad Jan 4 at 12:43
0
$\begingroup$

Perform one label encoding and then normalise the column.

| improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.