0
$\begingroup$

I’m looking for some pointers on efficiency. I have potentially 100,000s of binary variables that i wish to encode in each time step of an RNN for binary classification of the entire sequence, but I am torn between trying to encode every feature in each step (will be very, very sparse), or just use for a better word a “dodge” effect where if 3 out of the 100,000’s of variables = 1 (the vast majority at each time step will be 0) then these 3 occurrences just take up a separate time step each. For example if i have:

ID   Event_type (100,000’s of different types)
—    —————————-
1.   A
1.   BX
1.   ZZA
2.   YZAE
3.   C
3.   FXN

where for each ID, these events all appear in time = 1, I either

  1. Create a very sparse representation at each time step that includes many 0’s to indicate all the events not experienced at this time step
  2. Or do something like this:
[
 [[1,[“A”],
     [2,[“YZA”],
     [3,[“C”]] ,

 [[1,[“BX”],
     [2,[0],
     [3,[“FXN”]],

 [[1,[“ZZA”],
     [2,[0],
     [3,[0]]
]

i.e. pad out the sequences and effectively turn this 1 time step into 3 time steps (ID A has the largest number, 3 events in this particular time step).

Has anyone had to deal with many features per time step before, where they are always sparsely populated?

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.