I'm working on a project that makes use of Flair for stacked embeddings. I'm looking at the built in embeddings on this page. I noticed that the table shows
news-X as being "Trained with 1 billion word corpus". However when actually making use of the embeddings it seems you either use
I'm assuming this means both of these embeddings are trained on the same dataset and related. However one of the projects I'm looking at actually stacks both of these embeddings:
embedding_types: List[TokenEmbeddings] = [ FlairEmbeddings('news-forward'), FlairEmbeddings('news-backward'), # ... additional embeddings ]
I'm having trouble understanding what the forward and backwards means in this situation. Furthermore if they're both based on the same data what would be the benefit of stacking them?
The page linked above also says: "We recommend combining both forward and backward Flair embeddings" but doesn't explain exactly how the two different embeddings are generated from the training data.