I have read the docs here and I understand the general idea. I am able to visualize the weights of the intermediate layers. However, I'm having trouble visualize the activations. Here's what I have:
I trained my model and saved the weights in a file called
Thanks to this jupyter notebook, I got the values of the weights. First I defined my model:
def mlp_model(hid_dim=10): model = Sequential() model.add(Dense(units=hid_dim, input_dim=X.shape, activation='relu')) model.add(Dense(Y.shape, activation='softmax')) model.load_weights(weights_file) model.compile(loss='categorical_crossentropy', optimizer=optimizer, metrics=['accuracy']) return model model_created = mlp_model(hid_dim=15)
To get the weights, I did this:
W = model_created.layers.kernel.get_value(borrow=True) W = np.squeeze(W) print("W shape : ", W.shape) #(153, 15) W_out = model_created.layers.kernel.get_value(borrow=True) W_out = np.squeeze(W_out) print('W_out shape : ', W_out.shape) #(15, 8)
From there I could create Hinton diagrams using this. However, when I try to work with the activations:
get_first_output = theano.function([model_created.layers.input], [model_created.layers.output]) layer_out = get_first_output([X[0,:]])
I get this error:
TypeError: ('Bad input argument to theano function with name "mlp1_visualize_weights.py:131" at index 0 (0-based). \nBacktrace when that variable is created:\n\n File "mlp1_visualize_weights.py", line 213, in <module>\n mlp_repeat(X, Y, Xtest, Ytest, params_to_use, weights_file)\n File "mlp1_visualize_weights.py", line 125, in mlp_repeat\n model_created = mlp_model(hid_dim=hid_val, lr=lrate, reg_val=reg, momentum=moment, nest=nestval, optimizer=optim)\n File "mlp1_visualize_weights.py", line 105, in mlp_model\n model.add(Dense(units=hid_dim, input_dim=X.shape, kernel_initializer=\'he_uniform\', activation=\'relu\', W_regularizer=l2(reg_val), b_regularizer=l2(reg_val)))\n File "/mnt/data/caugusta/pkgs/anaconda2/lib/python2.7/site-packages/keras/models.py", line 426, in add\n dtype=layer.dtype, name=layer.name + \'_input\')\n File "/mnt/data/caugusta/pkgs/anaconda2/lib/python2.7/site-packages/keras/engine/topology.py", line 1392, in Input\n input_tensor=tensor)\n File "/mnt/data/caugusta/pkgs/anaconda2/lib/python2.7/site-packages/keras/engine/topology.py", line 1303, in __init__\n name=self.name)\n File "/mnt/data/caugusta/pkgs/anaconda2/lib/python2.7/site-packages/keras/backend/theano_backend.py", line 184, in placeholder\n x = T.TensorType(dtype, broadcast)(name)\n', 'TensorType(float32, matrix) cannot store accurately value [array([ 0. , 0.2037037 , 0.20138889, 0.21100917, 0.62962963,\n 0.6875 , 0.61206897, 0.44660194, 0.31168831, 0.17391304,\n 0. ...
I would like to look at just one input example, and find the activation and the weights from just that input example. Essentially I'm trying to figure out which features of the data each hidden unit is picking up.
Can anyone explain how to get the activations of intermediate layers in Keras?