I’m looking for some pointers on efficiency. I have potentially 100,000s of binary variables that i wish to encode in each time step of an RNN for binary classification of the entire sequence, but I am torn between trying to encode every feature in each step (will be very, very sparse), or just use for a better word a “dodge” effect where if 3 out of the 100,000’s of variables = 1 (the vast majority at each time step will be 0) then these 3 occurrences just take up a separate time step each. For example if i have:

ID   Event_type (100,000’s of different types)
—    —————————-
1.   A
1.   BX
1.   ZZA
2.   YZAE
3.   C
3.   FXN

where for each ID, these events all appear in time = 1, I either

  1. Create a very sparse representation at each time step that includes many 0’s to indicate all the events not experienced at this time step
  2. Or do something like this:
     [3,[“C”]] ,



i.e. pad out the sequences and effectively turn this 1 time step into 3 time steps (ID A has the largest number, 3 events in this particular time step).

Has anyone had to deal with many features per time step before, where they are always sparsely populated?


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.