1
$\begingroup$

Problem

In Keras for Python, I have to use multiple lines of code for a simple XOR neural network (from https://gist.github.com/stewartpark/187895beb89f0a1b3a54):

from keras.models import Sequential
from keras.layers.core import Dense, Dropout, Activation
from keras.optimizers import SGD
import numpy as np 

X = np.array([[0,0],[0,1],[1,0],[1,1]])
y = np.array([[0],[1],[1],[0]])

model = Sequential()
model.add(Dense(8, input_dim=2))
model.add(Activation('tanh'))
model.add(Dense(1))
model.add(Activation('sigmoid'))

sgd = SGD(lr=0.1); model.compile(loss='binary_crossentropy', optimizer=sgd)

model.fit(X, y, show_accuracy=True, batch_size=1, nb_epoch=1000)
print(model.predict_proba(X))

Although the network above is quite small, the current implementation may become frustrating with deeper networks.

Question

Is there a built-in shortcut in Keras that quickly adds multiple layers? For example: [2,3,4,5] would be a network with 2 input neurons, 3 neurons in the first hidden layer, 4 neurons in the second hidden layer, and 5 output neurons. I use the term built-in because otherwise I can loop through the example array, and add a layer for each element.

$\endgroup$
1
$\begingroup$

There is no shortcut syntax that goes as far as accepting [2,3,4,5] as a param and create a model. However, it would be very simple for you to create this as a Python function yourself, provided in your case you already have made the decisions about activation functions, the types of layer etc. The need to make those decisions and have them available in the Keras API means that Keras itself does not offer such a short form build function.

You can make the model building slightly less verbose by using a list of layers when you instantiate the model, instead of adding them afterwards:

model = Sequential([
  Dense(8, input_dim=2),
  Activation('tanh'),
  Dense(1),
  Activation('sigmoid')
])

However, if you want to try variations of a model, where the only things you change are the number and size of hidden layers, then you can write a short Python function to encapsulate that requirement. Here's an example (I'm sure you already know how to do this, just included for completeness):

def build_model(hidden_layer_sizes):
  model = Sequential()

  model.add(Dense(hidden_layer_sizes[0], input_dim=2))
  model.add(Activation('tanh'))

  for layer_size in hidden_layer_sizes[1:]:
    model.add(Dense(layer_size))
    model.add(Activation('tanh'))

  model.add(Dense(1))
  model.add(Activation('sigmoid'))

  return model

If you take this approach, you may find you end up parametrising other choices such as input size (as you try some feature engineering), hidden layer activation function, whether to use a Dropout layer etc. Again, it is this need to define all the other choices in a typical network that lead to Keras' design. The best you can do is compress down the choices for your case, with a custom function.

I'd like to address this comment in your question:

Although the network above is quite small, the current implementation may become frustrating with deeper networks.

In practice, I have not found Keras' design difficult to use for deep networks. I typically write a separate build function, and parametrise a few things, such as input dimensions. However, I typically don't loop through a list of different layer sizes as the main param of the build function. Instead, I find the more verbose approach just fine, even when trying variations of network size/shape (I guess that might change if I wanted to grid search including layer sizes). I think that is because I find the function names very easy to read - even with a screen full of .add() functions, I can see quite quickly what the NN structure is.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.