0
$\begingroup$

I am toying around with a clustering and churn prediction framework, cluschurn which they deployed in production at Snap, Inc. In their research paper, paper_link, they use 14 days of user data and treat it as a time series. They do some transformations and they have a have a 3-dimensional dataset which is 12x14xm users; each user, m, has 12 daily features for a total of 14 days. This set gets fed into an LSTM so that each LSTM timestep has receives a 12-dimensional vector. Hopefully, I worded this right so it's understandable. They were able to improve their results by adding a embedding layer between the input and the LSTM layer.

We connect a fully connected feedforward neural network to the original daily activity vactors, which converts users' sparse activity features of each day into distributional activity embeddings...

enter image description here

I'm completely confused on what this looks like dimension-wise and am confused if there is a fully connected network before each LSTM timestep or is there one fully connected network before the feeding the LSTM? For example, is the 3-dimensional data set flattened and then fed into a fully connected network and then reshaped back into a 3-dimension dataset that has 14 timesteps?

I made these concepts to help clarify what I mean.

enter image description here

enter image description here

$\endgroup$
1
  • $\begingroup$ Please, consider marking the answer as correct if deemed so. Alternatively, please considering describing what the answer is lacking or why you think it is not correct, so that it can be improved. $\endgroup$
    – noe
    Dec 1 '20 at 9:47
0
$\begingroup$

The most straightforward approach to understand their proposal is to look at their source code, which is linked from the article: https://github.com/yangji9181/ClusChurn/blob/master/model.py#L35

It reveals that the "embedding" is obtained by means of a mere linear layer.

$\endgroup$
3
  • $\begingroup$ I did not understand this answer so I did not mark it as right. I put 2 pictures up in my question asking for clarification on what the architecture is supposed to look like because I don't fully understand it and needed help in a deeper understanding on how it works. I'm not saying your answer is wrong or anything like that, it's just that it didn't help me understand the problem I was having. I do know the answer after lots of digging around and I should probably post an answer to explain my findings. $\endgroup$
    – zipline86
    Dec 2 '20 at 17:44
  • $\begingroup$ Sorry that my answer was not clear. I think it would be great if you posted the answer that made it clear for you. $\endgroup$
    – noe
    Dec 2 '20 at 17:52
  • $\begingroup$ No worries, thank you for posting though because maybe others will be able to figure it out. Yes, I will post my answer when I get some free time. $\endgroup$
    – zipline86
    Dec 3 '20 at 10:37

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.