I am building a NLP application. My dataset has 0.6M datapoints each of 0.15M dimensions. My feature vector is highly sparse - mostly 0s, atmost 20 1s in any feature vector. I am using a 32 GB machine on AWS.
Owing to size, Cant load entire dataset in one go in the memory. I am looking for a python based neural net library that supports (if not all then atleast some) of the following [Owing to the dataset size and dimension, i defined below mentioned abilities]:
- Ability to train the net on small batches
- library being able to exploit underlying multicore architecture (if available)
- A way to directly feed the feature vectors in compressed representation (like scipy.sparse) [in my case
I have had a good look at the following libraries - PyBrain, Pylearn2, NeuroLab, FANN. To the best of my understanding, none of them has any of the desired abilities (Pybrain facilitates batch leaning. Even this takes close to 55 hrs).
Any suggestions ? (cant reduce dimentionality) Or in case there is a shortcoming in my understanding and the any of the above libraries have (some) above capabilities, can you pls point out me to some sample code implementing/exploiting the same ?