Here's a blog post providing a view to the history of the process with several popular references in it. In particular, consider the Natural Language Processing section and its conclusion:
On NLP tasks that have a long enough history to graph, there seems to be no clear indication that deep learning performs above trendline.
Also, worth noting from the general conclusion of the article:
Deep learning provides discontinuous jumps relative to previous machine learning or AI performance trendlines in image recognition and speech recognition; it doesn’t in strategy games or natural language processing, and machine translation and arcade games are ambiguous (machine translation because metrics differ; arcade games because there is no pre-deep-learning comparison.)