# One hot encoding as input to recurrent neural networks

I'm trying to predict next label in a pattern based on previous labels using recurrent neural network. In total I have 100 labels

Example of input pattern:

1) orange, apple, banana, lemon -> grape
2) apple, banana, pineapple, mango -> orange
3) lychee, orange, grapefruit, apple -> lemon


Although this is a bogus example but it explains the problem pretty well. My target variable is a member of the input sequence set.

What I want to do now is as there is no ordinal relation between my input pattern, I dont want to simply label encode the data as the model might implicitly learn from the ordinal nature of label encoding, so I want to go for one-hot encoding.

But I'm having a hard time understanding how to create input feature space for input of one hot encoding. Should I have pattern input as:[1,0,0,0..,0],[0,1,0,0..,0],[0,0,1,0..0],[0,0,0,1..,0] or should it be just one matrix with 1 in place of all the labels present in the data and 0 where they aren't present something like [1,1,1,1,0,0..,0]?

• Depends whether or not the order of the input is of any importance. If e.g. orange, apple, ... -> ... is different than apple, orange, ... -> ... then you should do it like you initially said. If not then the latter approach should be fine but then why use an RNN if you don't have sequential data. However the description of your problem seems to me more like you're looking for association rules... – Djib2011 Apr 24 '19 at 11:27
• The sequence matters that's why RNN is being used. if I use the latter method what will be the input_shape considering I have 20k patterns and 1 feature per timestamp (the label itself) – Ammar Ahmed Apr 24 '19 at 11:33
• If sequence matters you need to use the first way so your input shape will be (batch_size, sequence_size, num_labels) – Djib2011 Apr 24 '19 at 11:45
• oh yes sorry, 1st should be used. thanks I'll try it! – Ammar Ahmed Apr 24 '19 at 12:03