I'm trying to predict next label in a pattern based on previous labels using recurrent neural network. In total I have 100 labels

Example of input pattern:

1) orange, apple, banana, lemon -> grape
2) apple, banana, pineapple, mango -> orange
3) lychee, orange, grapefruit, apple -> lemon

Although this is a bogus example but it explains the problem pretty well. My target variable is a member of the input sequence set.

What I want to do now is as there is no ordinal relation between my input pattern, I dont want to simply label encode the data as the model might implicitly learn from the ordinal nature of label encoding, so I want to go for one-hot encoding.

But I'm having a hard time understanding how to create input feature space for input of one hot encoding. Should I have pattern input as:[1,0,0,0..,0],[0,1,0,0..,0],[0,0,1,0..0],[0,0,0,1..,0] or should it be just one matrix with 1 in place of all the labels present in the data and 0 where they aren't present something like [1,1,1,1,0,0..,0]?

  • $\begingroup$ Depends whether or not the order of the input is of any importance. If e.g. orange, apple, ... -> ... is different than apple, orange, ... -> ... then you should do it like you initially said. If not then the latter approach should be fine but then why use an RNN if you don't have sequential data. However the description of your problem seems to me more like you're looking for association rules... $\endgroup$
    – Djib2011
    Apr 24, 2019 at 11:27
  • $\begingroup$ The sequence matters that's why RNN is being used. if I use the latter method what will be the input_shape considering I have 20k patterns and 1 feature per timestamp (the label itself) $\endgroup$ Apr 24, 2019 at 11:33
  • $\begingroup$ If sequence matters you need to use the first way so your input shape will be (batch_size, sequence_size, num_labels) $\endgroup$
    – Djib2011
    Apr 24, 2019 at 11:45
  • $\begingroup$ oh yes sorry, 1st should be used. thanks I'll try it! $\endgroup$ Apr 24, 2019 at 12:03
  • $\begingroup$ Related, I think $\endgroup$
    – Dave
    Nov 2, 2022 at 3:38

1 Answer 1


It can be complex to one-hot encode features for a neural network.

Often categories are feature hashed instead. Each category value is assigned a numeric value:

  • orange: 1
  • apple: 2
  • banana: 3
  • lemon: 4
  • grape: 5 …

The sequence then becomes:

1, 2, 3, 4 -> 5

A neural network is then able learn the sequence of numbers which represents the categories.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.