I am very much a beginner in ML space. I am learning keras to get hands-on experience. I picked the classification of 20_newsgroups data for my task, I used glove.6B.50d.txt for embeddings.

I chose to train a RNN for the task cos it has the capability to learn time series data better. Trained with different combinations of LSTM layers and output dimensions before converging to one combination. Then I tried CNN model for the same task.

What I found was CNN model could learn faster and gave about 65% accuracy at the end of 15 epochs, where as the RNN model took 50 epochs to get the same validation accuracy even after trying several learning rates. But at the end both models were giving about 65-70% validation accuracy after training for some more duration. I stopped my exploration there.

My question is I assumed/expected RNN to perform better. According to my understanding of RNN, it builds up the memory of time series data. What is it I am missing here ?


A Convolutional Neural Network (CNN) learns the spatial relationships in the data that are associated with the target values. In text data, the spatial relationships are which words are near each other. A specific example is the "changing on the fly" is associated with the rec.sport.hockey category.

Recurrent Neural Network (RNN) learns single latent representation of the data over the sequential sequence. That representation does not have an advantage on the evaluation metric performance for this problem.

In other words, learning a couple of important words to categorize text is faster and has just about the same evaluation metric performance as learning to represent the entire body of text.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.