0
$\begingroup$

I am trying to understand logistic regression for a multi-class example and I have the following code:

import numpy
import theano
import theano.tensor as T
rng = numpy.random

num_classes = 3
#N = number of examples
N = 100
#feats = number of input neurons
feats = 784
#training rate
tr_rate = 0.1 

D = (rng.randn(N, feats), rng.randint(size=N, low=0, high=num_classes))
training_steps = 1000

# Declare Theano symbolic variables
x = T.matrix("x")
y = T.ivector("y")

w = theano.shared(rng.randn(feats), name="w")
b = theano.shared(0.01, name="b")

# Construct Theano expression graph
sigma = T.nnet.softmax(T.dot(x,w) + b) 
prediction = T.argmax(sigma, axis=1)          # The class with highest probability

# Cross-entropy loss function
xent = -T.mean(T.log(sigma)[T.arange(y.shape[0]), y])

cost = xent.mean() + 0.01 * (w ** 2).sum()      # Regularisation 
gw, gb = T.grad(cost, [w, b])                   # Compute the gradient of the cost 

# Compile
train = theano.function(
          inputs=[x,y],
          outputs=[xent],
          updates=((w, w - tr_rate * gw), (b, b - tr_rate * gb)),
          allow_input_downcast=True)
predict = theano.function(inputs=[x], outputs=prediction)

# Train
for i in range(training_steps):
    train(D[0], D[1])      

However at this point the code gives me the following error:

ValueError                                Traceback (most recent call last)
<ipython-input-246-c4753ce8ccc7> in <module>()
      1 # Train
      2 for i in range(training_steps):
----> 3     train(D[0], D[1])

/Users/vzocca/gitProjects/Theano/theano/compile/function_module.pyc in __call__(self, *args, **kwargs)
    869                     node=self.fn.nodes[self.fn.position_of_error],
    870                     thunk=thunk,
--> 871                     storage_map=getattr(self.fn, 'storage_map', None))
    872             else:
    873                 # old-style linkers raise their own exceptions

/Users/vzocca/gitProjects/Theano/theano/gof/link.pyc in raise_with_op(node, thunk, exc_info, storage_map)
    312         # extra long error message in that case.
    313         pass
--> 314     reraise(exc_type, exc_value, exc_trace)
    315 
    316 

/Users/vzocca/gitProjects/Theano/theano/compile/function_module.pyc in __call__(self, *args, **kwargs)
    857         t0_fn = time.time()
    858         try:
--> 859             outputs = self.fn()
    860         except Exception:
    861             if hasattr(self.fn, 'position_of_error'):

ValueError: number of rows in x (1) does not match length of y (100)
Apply node that caused the error: CrossentropySoftmaxArgmax1HotWithBias(Elemwise{Add}[(0, 0)].0, Alloc.0, y)
Toposort index: 12
Inputs types: [TensorType(float64, row), TensorType(float64, vector), TensorType(int32, vector)]
Inputs shapes: [(1, 100), (100,), (100,)]
Inputs strides: [(800, 8), (8,), (4,)]
Inputs values: ['not shown', 'not shown', 'not shown']
Outputs clients: [[Sum{acc_dtype=float64}(CrossentropySoftmaxArgmax1HotWithBias.0)], [], []]

Backtrace when the node is created:
  File "<ipython-input-244-834f04b2fb36>", line 16, in <module>
    xent = -T.mean(T.log(sigma)[T.arange(y.shape[0]), y])

HINT: Use the Theano flag 'exception_verbosity=high' for a debugprint and storage map footprint of this apply node.

Can anyone help understand what is wrong? Thank you!

$\endgroup$
1
$\begingroup$

I found my own answer. I had defined

w = theano.shared(rng.randn(feats), name="w")

and that was wrong. The correct definition is:

w = theano.shared(rng.randn(feats, num_classes), name="w")

since the weights link 'feats'-number of input neuron to 'num_classes'-number of output neurons.

| improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.