I’m looking for some pointers on efficiency. I have potentially 100,000s of binary variables that i wish to encode in each time step of an RNN for binary classification of the entire sequence, but I am torn between trying to encode every feature in each step (will be very, very sparse), or just use for a better word a “dodge” effect where if 3 out of the 100,000’s of variables = 1 (the vast majority at each time step will be 0) then these 3 occurrences just take up a separate time step each. For example if i have:
ID Event_type (100,000’s of different types) — —————————- 1. A 1. BX 1. ZZA 2. YZAE 3. C 3. FXN
where for each ID, these events all appear in time = 1, I either
- Create a very sparse representation at each time step that includes many 0’s to indicate all the events not experienced at this time step
- Or do something like this:
[ [[1,[“A”], [2,[“YZA”], [3,[“C”]] , [[1,[“BX”], [2,, [3,[“FXN”]], [[1,[“ZZA”], [2,, [3,] ]
i.e. pad out the sequences and effectively turn this 1 time step into 3 time steps (ID A has the largest number, 3 events in this particular time step).
Has anyone had to deal with many features per time step before, where they are always sparsely populated?