# Cannot fig out error in my gradient function implementation in python

Im trying to implement following gradient descent function in Python for logistic regression:

$$∇θ(−logL)=−X^T 􏰀(y−e^{Xθ}􏰁)$$

This is my python implementation:

def gradient(X, y, theta):
dtheta = -(np.dot(X.T,y - np.exp(X * theta)))
return dtheta


X is a dataframe of size: (2458, 31), y is a dataframe of size: (2458, 1) theta is dataframe of size: (2458,1)

when i pass values to my gradient descent function, it returns a dtheta parameter with size (31,31) due to which i cannot update my theta to pass it to cost function, i cannot fig out where im going wrong. any help will be appreciated.

Error i keep getting is: ValueError: operands could not be broadcast together with shapes (2458,1) (31,31)

and this is how im implementing the algorithm:

theta = np.random.uniform(low=-0.1,high=0.1, size=(2458,1))
# Iterate and update theta by using the gradient of the negative log-likelihood
max_iter = 100
learning_rate = 1e-3
for i in range(max_iter):

# Update theta

theta = (theta - learning_rate) * dtheta

# Calculate the value of the log-likelihood
cost = negative_loglikelihood(X,y,theta)

# Print iteration
print("Iteration %d, cost function %.3f" % (i+1,cost))

• Can you share the source of the above gradient descent formula? – Yash Jakhotiya Nov 4 '19 at 5:22
• This is part of an assignment that i received, in which i have a negative log likelihood function whose gradient function is as above. – Gaurang Swarge Nov 4 '19 at 5:26

Check your theta dimensions.
Most likely, your X dimensions indicate you have 2458 training samples per iteration with each having 31 features. Hence, your theta should be a matrix of shape (31, 1).
With X having shape (2458, 31) and if theta has shape (31, 1), X*theta will have dimensions (2458, 1), same as y and as expected. Now, y-theta has the same dimensions as that of y or theta. And so does exp(y-theta)
X_T has shape (31, 2458) and hence, d_theta = - X_T*exp(y-theta) will have shape (31, 1), same as our initial assumed theta shape and now, you can subtract d_theta from theta.