I am toying around with a clustering and churn prediction framework, cluschurn which they deployed in production at Snap, Inc. In their research paper, paper_link, they use 14 days of user data and treat it as a time series. They do some transformations and they have a have a 3-dimensional dataset which is 12x14xm users; each user, m, has 12 daily features for a total of 14 days. This set gets fed into an LSTM so that each LSTM timestep has receives a 12-dimensional vector. Hopefully, I worded this right so it's understandable. They were able to improve their results by adding a embedding layer between the input and the LSTM layer.
We connect a fully connected feedforward neural network to the original daily activity vactors, which converts users' sparse activity features of each day into distributional activity embeddings...
I'm completely confused on what this looks like dimension-wise and am confused if there is a fully connected network before each LSTM timestep or is there one fully connected network before the feeding the LSTM? For example, is the 3-dimensional data set flattened and then fed into a fully connected network and then reshaped back into a 3-dimension dataset that has 14 timesteps?
I made these concepts to help clarify what I mean.
