4
$\begingroup$

I am building a NLP application. My dataset has 0.6M datapoints each of 0.15M dimensions. My feature vector is highly sparse - mostly 0s, atmost 20 1s in any feature vector. I am using a 32 GB machine on AWS.

Owing to size, Cant load entire dataset in one go in the memory. I am looking for a python based neural net library that supports (if not all then atleast some) of the following [Owing to the dataset size and dimension, i defined below mentioned abilities]:

  1. Ability to train the net on small batches
  2. library being able to exploit underlying multicore architecture (if available)
  3. A way to directly feed the feature vectors in compressed representation (like scipy.sparse) [in my case

I have had a good look at the following libraries - PyBrain, Pylearn2, NeuroLab, FANN. To the best of my understanding, none of them has any of the desired abilities (Pybrain facilitates batch leaning. Even this takes close to 55 hrs).

Any suggestions ? (cant reduce dimentionality) Or in case there is a shortcoming in my understanding and the any of the above libraries have (some) above capabilities, can you pls point out me to some sample code implementing/exploiting the same ?

$\endgroup$
  • 1
    $\begingroup$ You can use Theano backend to implement your own neural net. It will take advantage of multiple CPUs or GPUs $\endgroup$ – Bertrand R Oct 16 '15 at 17:33
  • $\begingroup$ @Betrand: I agree, but in theano can I run learning in batches and directly feed compressed representation ? $\endgroup$ – Anuj Gupta Oct 17 '15 at 1:58
  • 2
    $\begingroup$ Since you have to implement your own network then you can do whatever you want about batches as long as the learning process isn't harmed. About compressed representation I don't think it can be handled, but if I were you, my main concern would be the dimension of the input space which seems to be way too huge to learn anything meaningful. $\endgroup$ – Bertrand R Oct 17 '15 at 11:11
1
$\begingroup$

I experimented with PyBrain and PyLearn2 I rather find Keras [http://keras.io/ ] as an excellent choice. It way ahead of PyBrain for sure. PyBrain development has been stopped

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.