# Best python library for neural networks

I'm using Neural Networks to solve different Machine learning problems. I'm using Python and pybrain but this library is almost discontinued. Are there other good alternatives in Python?

Thanks

-
–  Air Jul 8 '14 at 20:14
I saw thanks, but it is outdated, and closed. –  marcodena Jul 9 '14 at 0:06
If you want to only use Restricted Boltzmann Machine, you can stick with scikit-learn as well. –  user70747 Sep 17 '14 at 11:42
And now there's a new contender - Scikit Neuralnetwork: Has anyone had experience with this yet? How does it compare with Pylearn2 or Theano? –  Rafael_Espericueta Jul 7 at 12:20
There is also Keras - github.com/fchollet/keras - which is relatively recent. The problems with tracking "best" by any measure, and keeping the Q&A valid over time is why this sort of question is usually off topic in other Stack Exchange networks. –  Neil Slater Jul 9 at 8:37

Pylearn2 is generally considered the library of choice for neural networks and deep learning in python. Its designed for easy scientific experimentation rather than ease of use, so the learning curve is rather steep, but if you take your time and follow the tutorials I think you'll be happy with the functionality it provides. Everything from standard Multilayer Perceptrons to Restricted Boltzmann Machines to Convolutional Nets to Autoencoders are provided. There's great GPU support and everything is built on top of Theano, so performance is typically quite good. The source for Pylearn2 is available on github.

Be aware that Pylearn2 has the opposite problem of pybrain at the moment -- rather than being abandoned, Pylearn2 is under active development and is subject to frequent changes.

UPDATE: the landscape has changed quite a bit since I answered this question in July '14, and some new players have entered the space. In particular I would recommend checking out:

They each have their strengths and weaknesses, so give them all a go and see which best suits your use case.

-

Pylearn is relies on Theano and as mentioned in other answer to use the library is really complicated, until you get the hold of it.

In the meantime I would suggest using Theanets. It aslo built on top of Theano, but is much more easier to work with. It might be true, that it doesn't have all the features of Pylearn, but for the basic work it's sufficient.

Also it's open source, so you can add custom networks on the fly, if you dare. :)

-

Lasagne (docs) is very nice, as it uses theano (→ you can use the GPU) and makes it simpler to use. The author of lasagne won the Kaggle Galaxy challenge, as far as I know. It is nice with nolearn. Here is an MNIST example network:

#!/usr/bin/env python

import lasagne
from lasagne import layers
from lasagne.updates import nesterov_momentum
from nolearn.lasagne import NeuralNet

import sys
import os
import gzip
import pickle
import numpy

PY2 = sys.version_info[0] == 2

if PY2:
from urllib import urlretrieve

else:
from urllib.request import urlretrieve

DATA_URL = 'http://deeplearning.net/data/mnist/mnist.pkl.gz'
DATA_FILENAME = 'mnist.pkl.gz'

"""Load data from url and store the result in filename."""
if not os.path.exists(filename):
urlretrieve(url, filename)

with gzip.open(filename, 'rb') as f:

"""Get data with labels, split into training, validation and test set."""
X_train, y_train = data[0]
X_valid, y_valid = data[1]
X_test, y_test = data[2]
y_train = numpy.asarray(y_train, dtype=numpy.int32)
y_valid = numpy.asarray(y_valid, dtype=numpy.int32)
y_test = numpy.asarray(y_test, dtype=numpy.int32)

return dict(
X_train=X_train,
y_train=y_train,
X_valid=X_valid,
y_valid=y_valid,
X_test=X_test,
y_test=y_test,
num_examples_train=X_train.shape[0],
num_examples_valid=X_valid.shape[0],
num_examples_test=X_test.shape[0],
input_dim=X_train.shape[1],
output_dim=10,
)

def nn_example(data):
net1 = NeuralNet(
layers=[('input', layers.InputLayer),
('hidden', layers.DenseLayer),
('output', layers.DenseLayer),
],
# layer parameters:
input_shape=(None, 28*28),
hidden_num_units=100,  # number of units in 'hidden' layer
output_nonlinearity=lasagne.nonlinearities.softmax,
output_num_units=10,  # 10 target values for the digits 0, 1, 2, ..., 9

# optimization method:
update=nesterov_momentum,
update_learning_rate=0.01,
update_momentum=0.9,

max_epochs=10,
verbose=1,
)

# Train the network
net1.fit(data['X_train'], data['y_train'])

# Try the network on new data
print("Feature vector (100-110): %s" % data['X_test'][0][100:110])
print("Label: %s" % str(data['y_test'][0]))
print("Predicted: %s" % str(net1.predict([data['X_test'][0]])))

def main():
print("Got %i testing datasets." % len(data['X_train']))
nn_example(data)

if __name__ == '__main__':
main()


Caffe is a C++ library, but has Python bindings. You can do most stuff by configuration files (prototxt). It has a lot of options and can also make use of the GPU.

-

Pylearn2 seems to be the library of choice, however I find their YAML configuration files off-putting.

Python itself was designed to be an easy language for prototyping, why would you not use it to define the network properties themselves? We have great editors with autocompletion that would make your life much easier and Python is not like C++ where you have to wait for long builds to finish before you can run your code.

YAML files on the other hand you have to edit using a standard text editor with no assistance whatsoever and this makes the learning curve even steeper.

I may be missing the big picture but I still don't understand what were they thinking, I don't think prototyping in code would be much slower. For that reason I'm considering Theanets or using Theano directly.

-
I'm was also a bit thrown by the YAML files at first, but have since come to love the clean separation between configuration and code. You can choose to use Pylearn2 without YAML files, although this option is not well documented. –  Madison May Jul 21 '14 at 20:52
In short, however, I wouldn't discard the library because of this simple design decision. –  Madison May Jul 21 '14 at 20:53
As madison may, mentioned its all about separating configuration and code. It would be fine if you were running one network and knew all the parameters, but you don't. by splitting config and code, you can run multiple networks - different hidden neurons etc, etc. and source control is straight forward ( how do you keep track of which configuration you have tried if you keep it in the code). –  seanv507 Sep 18 '14 at 23:57

From what I heard, Pylearn2 might be currently the library of choice for most people. This reminds me of a recent blog post a few month ago that lists all the different machine learning libraries with a short explanation

https://www.cbinsights.com/blog/python-tools-machine-learning

The section you might be interested in here would be "Deep Learning". About Pylearn2, he writes

PyLearn2

There is another library built on top of Theano, called PyLearn2 which brings modularity and configurability to Theano where you could create your neural network through different configuration files so that it would be easier to experiment different parameters. Arguably, it provides more modularity by separating the parameters and properties of neural network to the configuration file.

-

I like Blocks, which is also built on top of Theano. Way more approachable than PyLearn2, and more feature rich than Lasagne. Neatly written, too.

-