Extractive text summarization, as a classification problem using deep networks

Question

I understand that this is broad, but I merely require a few pointers.

I wish to implement a text summarization system. I have already tried out an abstractive approach, now I am trying an extractive approach. The current extractive approaches out there(Textrank, Lexrank etc) are not giving optimal performance. It was suggested to me that I take it as an unsupervised learning task, using Autoencoders/RBM's or K means or any other unsupervised learning algorithm to classify sentences as important or not.

My questions are:

1. Is something like this feasible? (I know that nothing can be said for sure in data science unless tried out, but is it worth the shot?)
2. What kind of features should I feed into the classifier? I already know of word frequency distribution(TF-IDF), sentence position, co-occurance stats, but will these be enough? What additional features should I consider? Or should I consider directly feeding in word vectors?

I have done my bit of work on the same and have read several papers, but none of them offer clarity in terms of feature computation, but only explain network architecture.

Any form of help will be appreciated, thank you!

Answer 1

TL;DR:

Is something like this feasible? (I know that nothing can be said for sure in data science unless tried out, but is it worth the shot?)

Yes

What kind of features should I feed into the classifier? I already know of word frequency distribution(TF-IDF), sentence position, co-occurance stats, but will these be enough? What additional features should I consider? Or should I consider directly feeding in word vectors?

Feed in word vectors. Deep learning has it's way of figuring out good features like tf-idf,co-occurence etc.

Longer Version:

Input:

Convert all the words to word vectors with a word embedding algorithm like Word2Vec or Glove. This is usually a good idea since this allows you to represent words in a better semantic (dog, cat, cow,etc will be close) and syntactic (November,December,etc will be close) sense.

(Using tf-idf as a feature hasn't worked for me. I used it to supervise extractive summarization with a neural net.)

How to manage inputs:

This depends very much on how your data is. (Please let me know which dataset) I assume you have something like [sentence1, sentence2,...] as inputs and an output like [True, False, ...] which implies sent1 is in the summary and sent2 isn't.

If your data isn't in this form, you should convert it into this form.

You haven't specified whether it's Unsupervised or Supervised.

for Supervised:

Use an LSTM. Feed the LSTM the sentence word by word. Once the sentence has ended (marked by a token say or a full stop) make the LSTM make a prediction whether the sentence should be in the summary or not. Train it based on this.

This is kind of similar to how you would do sentiment analysis: feed in the sentence and then ask if the sentiment is positive or negative.

You can experiment with stacking LSTMs or changing hyper parameters to get better results.

for Unsupervied:

This is a bit difficult. You could convert the sentences to vectors (Sent2Vec) and hope that important sentences cluster together because they contain important words.

Or you could train an RNN to predict the next character and hope that one of the neurons learns to predict important words/sentences as done by OpenAI's unsupervised sentiment neuron which learnt to predict the sentiment.

The general "Deep" architecture followed these days is : Embed, encode, attend, predict

Check out the links I have posted (in blue), they might work for you.

Note: If possible, please mention your dataset and also how you managed to make abstractive summaries (considering that it is a lot harder than extractive)

Answer 2

I have tried markovify, a markov chain library in python for automatic text summarization for text documents. You can check that out for a simple unsupervised approach. For me it did not give satisfactory results as I was using abstracts from medical journals as my dataset, but if you try to use it on a different themed dataset, it may help you.