Recurrent Neural Networks Over Multiple Documents Over Time

Question

So in my head, I have an idea about what this architecture should look like, or at least behave, but I am having trouble implementing it. So let me describe the problem, and if anyone has an idea on how to actually implement it let me know. Or if I am over-thinking a solution.

I am trying to classify accounts into one of two groups, good and bad. I have multiple text documents per account. What I want to do is take the text documents for a single account and order them chronologically. Then use a recurrent neural network to essentially learn an embedding for each document (traditional text recurrent architecture should work fine), and then plug those embeddings into a document-level recurrent neural net, to predict when an account switches from good to bad. That way, as new documents come in per account I get updated "badness" scores for an account, and potentially flag an account above some threshold. I would like some sort of end-to-end architecture too. Like I said, not sure how to get what is my head into the computer.

Here's my question, is anyone aware of an implementation like that?

Answer 1

A simple implementation of such a network would be:

Input --> Embedding layer --> LSTM layer --> Softmax layer

This would allow the embedding layer to learn your document representation, and the lstm layer to learn states based on the observed sequences of those embedding instances, and finally the softmax layer to interpret these states at the account level as "good" or "bad".