6
$\begingroup$

I am building a NLP application. My dataset has 0.6M datapoints each of 0.15M dimensions. My feature vector is highly sparse - mostly 0s, atmost 20 1s in any feature vector. I am using a 32 GB machine on AWS.

Owing to size, Cant load entire dataset in one go in the memory. I am looking for a python based neural net library that supports (if not all then atleast some) of the following [Owing to the dataset size and dimension, i defined below mentioned abilities]:

  1. Ability to train the net on small batches
  2. library being able to exploit underlying multicore architecture (if available)
  3. A way to directly feed the feature vectors in compressed representation (like scipy.sparse) [in my case

I have had a good look at the following libraries - PyBrain, Pylearn2, NeuroLab, FANN. To the best of my understanding, none of them has any of the desired abilities (Pybrain facilitates batch leaning. Even this takes close to 55 hrs).

Any suggestions ? (cant reduce dimentionality) Or in case there is a shortcoming in my understanding and the any of the above libraries have (some) above capabilities, can you pls point out me to some sample code implementing/exploiting the same ?

Improve this question
$\endgroup$
3
  • 1
    $\begingroup$ You can use Theano backend to implement your own neural net. It will take advantage of multiple CPUs or GPUs $\endgroup$
    – Bertrand R
    Oct 16, 2015 at 17:33
  • $\begingroup$ @Betrand: I agree, but in theano can I run learning in batches and directly feed compressed representation ? $\endgroup$
    – Anuj Gupta
    Oct 17, 2015 at 1:58
  • 2
    $\begingroup$ Since you have to implement your own network then you can do whatever you want about batches as long as the learning process isn't harmed. About compressed representation I don't think it can be handled, but if I were you, my main concern would be the dimension of the input space which seems to be way too huge to learn anything meaningful. $\endgroup$
    – Bertrand R
    Oct 17, 2015 at 11:11

1 Answer 1

Reset to default
2
$\begingroup$

I experimented with PyBrain and PyLearn2 I rather find Keras [http://keras.io/ ] as an excellent choice. It way ahead of PyBrain for sure. PyBrain development has been stopped

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.