1
$\begingroup$

Most of the examples I have found online for LSTMs refer to "random" text generation. One of the problems that I'm trying to solve is to generate a "summary" of many docs into 1 doc. For example:

  • News Article: Barack Obama life history (2K words)
  • Wikipedia: Barack Obama (10K words)
  • Barack Obama biography (10K words)

Will generate an article about Obama in X words.

Is LSTM the right approach?

$\endgroup$
1
$\begingroup$

Although you can generate text in this way - sampling from a RNN trained to predict next character or word - it will not be meaningful. At best it will be grammatically accurate (in terms of nouns, verbs, adjectives etc) but semantic nonsense. It would not be a summary, except by lucky accident.

To generate summaries using LSTM directly, you would need to train a network with real examples of summary inputs and outputs. This would be similar task to machine translation, but much harder due to variability in size of the inputs. It is unlikely that you will find enough training data to test the idea fully, and it is not clear that such a direct approach can yield acceptable results even with large amounts of training data.

In fact text summarisation is not a solved problem by any means. There are deep learning approaches, such as Google Brain team's effort using TensorFlow, which you could study to get some sample code and a sense of state-of-the-art. This approach uses an attention model to extract apparently informational content (i.e. content that would have low probability of appearing in some assumed generic document, thus is assumed to be interesting due to standing out). It is possible to use a trained LSTM to build such an attention-based model - the intuition is that the parts of the document that the already-trained LSTM is least able to predict are likely to contain noteworthy information.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.