Currently making a bot that condenses news articles. I'm tagging sentences as important or not important using a simple BERT classifier. The results were... not great. I'm really interested in how I can improve the results using LSTM.
I'm now batching 5 sentences together, calculating their BERT encodings, and then using 2 LSTM Layers, one backwards one forwards, to predict if the sentence is important.
Unfortunately I'm now calculating 5 times the number of embeddings, and if it doesn't work, I can't seem to figure out how to feed a variable number of things into BERT using Tensorflow, to see if I can tweak some results.
Are there other methods to add surrounding sentences to this context?