Data Science Stack Exchange is a question and answer site for Data science professionals, Machine Learning specialists, and those interested in learning more about the field. It's 100% free, no registration required.

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

I'm using Neural Networks to solve different Machine learning problems. I'm using Python and pybrain but this library is almost discontinued. Are there other good alternatives in Python?

Thanks

share|improve this question
    
See also stackoverflow.com/q/2276933/2359271 – Air Jul 8 '14 at 20:14
2  
And now there's a new contender - Scikit Neuralnetwork: Has anyone had experience with this yet? How does it compare with Pylearn2 or Theano? – Rafael_Espericueta Jul 7 '15 at 12:20
1  
@Emre: Scalable is different to high performance. It typically means you can solve larger problems by adding more resources of the same type you have already. Scalability still wins out, when you have 100 machines available, even if your software is 20 times slower on each of them . . . (although I'd rather pay the price for 5 machines and have benefits of both GPU and multi-machine scale). – Neil Slater Jul 13 '15 at 15:25
1  
So use multiple GPUs...nobody uses CPUs for serious work in neural networks. If you can get Google-level performance out of a good GPU or two, just what are you going to do with a thousand CPUs? – Emre Jul 13 '15 at 16:11
1  
I'm voting to close this question as off-topic because is become a poster example of why recommendations and "best" questions don't work in the format. The accepted answer is factually inaccurate after 12 months (PyLearn2 has in that time gone from "active development" to "accepting patches") – Neil Slater Nov 26 '15 at 17:33

12 Answers 12

up vote 52 down vote accepted

UPDATE: the landscape has changed quite a bit since I answered this question in July '14, and some new players have entered the space. In particular I would recommend checking out:

They each have their strengths and weaknesses, so give them all a go and see which best suits your use case. Although I would have recommended using pylearn2 a year ago, the community is no longer active so I would recommend looking elsewhere. My original response to the answer is included below, but is largely irrelevant at this point.


Pylearn2 is generally considered the library of choice for neural networks and deep learning in python. It's designed for easy scientific experimentation rather than ease of use, so the learning curve is rather steep, but if you take your time and follow the tutorials I think you'll be happy with the functionality it provides. Everything from standard Multilayer Perceptrons to Restricted Boltzmann Machines to Convolutional Nets to Autoencoders is provided. There's great GPU support and everything is built on top of Theano, so performance is typically quite good. The source for Pylearn2 is available on github.

Be aware that Pylearn2 has the opposite problem of pybrain at the moment -- rather than being abandoned, Pylearn2 is under active development and is subject to frequent changes.

share|improve this answer
    
Note that nolearn is a wrapper that makes other libraries easier to use and compatible with sklearn. It's not of itself a neural network library, but nonetheless recommended. At the time of writing it's mostly for Lasagne but there's some Caffe code and maybe others. – Mark Aug 4 '15 at 16:36
    
Not sure if it's the same as Marks comment about nolearn, but github.com/aigamedev/scikit-neuralnetwork is also some form of a wrapper for a bunch of this stuff. – onaclov2000 Sep 24 '15 at 20:07
2  

Tensor Flow (docs) by Google is another nice framework which has automatic differentiation. I've written down some quick thoughts about Google Tensor Flow on my blog, together with the MNIST example which they have in their tutorial.

Lasagne (docs) is very nice, as it uses theano (→ you can use the GPU) and makes it simpler to use. The author of lasagne won the Kaggle Galaxy challenge, as far as I know. It is nice with nolearn. Here is an MNIST example network:

#!/usr/bin/env python

import lasagne
from lasagne import layers
from lasagne.updates import nesterov_momentum
from nolearn.lasagne import NeuralNet

import sys
import os
import gzip
import pickle
import numpy


PY2 = sys.version_info[0] == 2

if PY2:
    from urllib import urlretrieve

    def pickle_load(f, encoding):
        return pickle.load(f)
else:
    from urllib.request import urlretrieve

    def pickle_load(f, encoding):
        return pickle.load(f, encoding=encoding)

DATA_URL = 'http://deeplearning.net/data/mnist/mnist.pkl.gz'
DATA_FILENAME = 'mnist.pkl.gz'


def _load_data(url=DATA_URL, filename=DATA_FILENAME):
    """Load data from `url` and store the result in `filename`."""
    if not os.path.exists(filename):
        print("Downloading MNIST dataset")
        urlretrieve(url, filename)

    with gzip.open(filename, 'rb') as f:
        return pickle_load(f, encoding='latin-1')


def load_data():
    """Get data with labels, split into training, validation and test set."""
    data = _load_data()
    X_train, y_train = data[0]
    X_valid, y_valid = data[1]
    X_test, y_test = data[2]
    y_train = numpy.asarray(y_train, dtype=numpy.int32)
    y_valid = numpy.asarray(y_valid, dtype=numpy.int32)
    y_test = numpy.asarray(y_test, dtype=numpy.int32)

    return dict(
        X_train=X_train,
        y_train=y_train,
        X_valid=X_valid,
        y_valid=y_valid,
        X_test=X_test,
        y_test=y_test,
        num_examples_train=X_train.shape[0],
        num_examples_valid=X_valid.shape[0],
        num_examples_test=X_test.shape[0],
        input_dim=X_train.shape[1],
        output_dim=10,
    )


def nn_example(data):
    net1 = NeuralNet(
        layers=[('input', layers.InputLayer),
                ('hidden', layers.DenseLayer),
                ('output', layers.DenseLayer),
                ],
        # layer parameters:
        input_shape=(None, 28*28),
        hidden_num_units=100,  # number of units in 'hidden' layer
        output_nonlinearity=lasagne.nonlinearities.softmax,
        output_num_units=10,  # 10 target values for the digits 0, 1, 2, ..., 9

        # optimization method:
        update=nesterov_momentum,
        update_learning_rate=0.01,
        update_momentum=0.9,

        max_epochs=10,
        verbose=1,
        )

    # Train the network
    net1.fit(data['X_train'], data['y_train'])

    # Try the network on new data
    print("Feature vector (100-110): %s" % data['X_test'][0][100:110])
    print("Label: %s" % str(data['y_test'][0]))
    print("Predicted: %s" % str(net1.predict([data['X_test'][0]])))


def main():
    data = load_data()
    print("Got %i testing datasets." % len(data['X_train']))
    nn_example(data)

if __name__ == '__main__':
    main()

Caffe is a C++ library, but has Python bindings. You can do most stuff by configuration files (prototxt). It has a lot of options and can also make use of the GPU.

share|improve this answer

Pylearn relies on Theano and as mentioned in the other answer to use the library is quite complicated, until you get the hold of it.

In the meantime I would suggest using Theanets. It also built on top of Theano, but is much more easier to work with. It might be true, that it doesn't have all the features of Pylearn, but for the basic work it's sufficient.

Also it's open source, so you can add custom networks on the fly, if you dare. :)

EDIT: Dec 2015. Recently I have started using Keras. It is a bit lower level than Theanets, but much more powerful. For basic tests the Theanets is appropriate. But if you want to do some research in field of ANN Keras is much more flexible. Plus the Keras can use Tensorflow as a backend.

share|improve this answer

TensorFlow (by Google, released on 2015-11-09) looks promising.

  • open source (Apache 2.0 License) (GitHub)
  • Python (backend in C++)
  • CPU/GPU
  • Auto-Differentiation
  • Portable (even works on mobile devices)

enter image description here

FYI:

share|improve this answer
2  
Aargh! You beat me to it :) Indeed, TensorFlow is super good. Has a better compile time than the big guns out there like Theano, Torch, etc – Dawny33 Nov 10 '15 at 6:31
    
@Dawny33 What do you mean with "compile time"? Theano, Torch and most of the time also TensorFlow are used in an interpreted fashion. – Martin Thoma Dec 22 '15 at 1:02
    
@moose In Theano at least, some code is compiled in C++ or CUDA. – Franck Dernoncourt Dec 22 '15 at 1:03
    
@FranckDernoncourt Ok, but does that time really matter? When you have the training running for about 20 minutes, isn't only a couple of seconds the time it needs to generate the CUDA code? (How can I measure that?) – Martin Thoma Dec 22 '15 at 1:05
2  
@moose In Theano generating the CUDA / C++ code takes around 30 seconds to one minute for a reasonably sized model. It makes debugging quite tedious. To measure CUDA / C++ code generation time, you can time before/after a Theano function is compiled. – Franck Dernoncourt Dec 22 '15 at 1:21

Pylearn2 seems to be the library of choice, however I find their YAML configuration files off-putting.

Python itself was designed to be an easy language for prototyping, why would you not use it to define the network properties themselves? We have great editors with autocompletion that would make your life much easier and Python is not like C++ where you have to wait for long builds to finish before you can run your code.

YAML files on the other hand you have to edit using a standard text editor with no assistance whatsoever and this makes the learning curve even steeper.

I may be missing the big picture but I still don't understand what were they thinking, I don't think prototyping in code would be much slower. For that reason I'm considering Theanets or using Theano directly.

share|improve this answer
1  
I'm was also a bit thrown by the YAML files at first, but have since come to love the clean separation between configuration and code. You can choose to use Pylearn2 without YAML files, although this option is not well documented. – Madison May Jul 21 '14 at 20:52
    
In short, however, I wouldn't discard the library because of this simple design decision. – Madison May Jul 21 '14 at 20:53
    
As madison may, mentioned its all about separating configuration and code. It would be fine if you were running one network and knew all the parameters, but you don't. by splitting config and code, you can run multiple networks - different hidden neurons etc, etc. and source control is straight forward ( how do you keep track of which configuration you have tried if you keep it in the code). – seanv507 Sep 18 '14 at 23:57

I like Blocks, which is also built on top of Theano. Way more approachable than PyLearn2, and more feature rich than Lasagne. Neatly written, too.

Updated Jan 2016:

At the time of writing, Keras has by far the most momentum. It is highly modular and can run on both Theano and Tensorflow, giving it great opportunities.

share|improve this answer
    
Yeah, I would currently recommend blocks today over pylearn2 if you're ok putting in a bit of time to understand Theano. – Madison May Aug 6 '15 at 19:50
    
Great library built by great people. – Madison May Aug 6 '15 at 19:50

From what I heard, Pylearn2 might be currently the library of choice for most people. This reminds me of a recent blog post a few month ago that lists all the different machine learning libraries with a short explanation

https://www.cbinsights.com/blog/python-tools-machine-learning

The section you might be interested in here would be "Deep Learning". About Pylearn2, he writes

PyLearn2

There is another library built on top of Theano, called PyLearn2 which brings modularity and configurability to Theano where you could create your neural network through different configuration files so that it would be easier to experiment different parameters. Arguably, it provides more modularity by separating the parameters and properties of neural network to the configuration file.

share|improve this answer

I wrote up this post detailing some of my personal favorites:

The Best Machine Learning Libraries in Python

Since over 30 different libraries are mentioned, I won't post them all here, but these are among the most popular:

(Sorry, can't link to the Github repos since my rep is still < 10...)

Edit: Added links to Github repos.

share|improve this answer
    
Can you comment the link under my comment here? I shall edit and add it to your answer – Dawny33 Nov 11 '15 at 4:42
    
Now that my answer was upvoted, I now have the required rep and was able to add the links. Thanks though! – Scott Robinson Nov 11 '15 at 15:04

Just to add some more resources. Recently there was a paper studying the differences between several packages of neural networks and deep neural networks.

Here you can find the information. It looks that Torch and TensorFlow are the winners.

Note: not all of them are in python. However, I posted it to open the discussion.

share|improve this answer

MXNet:

  • written in C++ but has an API in Python (and a few other programming languages such as R, Julia, and Go)
  • Scales up to multi GPUs and distributed setting with auto parallelism.
  • Automatic differentiation
  • Decent performances:

enter image description here

share|improve this answer

neon:

share|improve this answer

If you are interested in performance comparisons, Soumith Chintala maintains a set of convnet benchmarks that cover several of the already mentioned python frameworks (TensorFlow, chainer, neon, Theano):

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.