Cannot fig out error in my gradient function implementation in python

Question

Im trying to implement following gradient descent function in Python for logistic regression:

$∇θ(−logL)=−X^T 􏰀(y−e^{Xθ}􏰁)$

This is my python implementation:

def gradient(X, y, theta):
    dtheta = -(np.dot(X.T,y - np.exp(X * theta)))
    return dtheta

X is a dataframe of size: (2458, 31), y is a dataframe of size: (2458, 1) theta is dataframe of size: (2458,1)

when i pass values to my gradient descent function, it returns a dtheta parameter with size (31,31) due to which i cannot update my theta to pass it to cost function, i cannot fig out where im going wrong. any help will be appreciated.

Error i keep getting is: ValueError: operands could not be broadcast together with shapes (2458,1) (31,31)

and this is how im implementing the algorithm:

theta = np.random.uniform(low=-0.1,high=0.1, size=(2458,1))
# Iterate and update theta by using the gradient of the negative log-likelihood
max_iter = 100
learning_rate = 1e-3
for i in range(max_iter):
    # Calculate the gradient
    dtheta = gradient(X,y,theta)

    # Update theta

    theta = (theta - learning_rate) * dtheta

    # Calculate the value of the log-likelihood
    cost = negative_loglikelihood(X,y,theta)

    # Print iteration
    print("Iteration %d, cost function %.3f" % (i+1,cost))

Answer 1

Check your theta dimensions.

Most likely, your X dimensions indicate you have 2458 training samples per iteration with each having 31 features. Hence, your theta should be a matrix of shape (31, 1).

With X having shape (2458, 31) and if theta has shape (31, 1), X*theta will have dimensions (2458, 1), same as y and as expected. Now, y-theta has the same dimensions as that of y or theta. And so does exp(y-theta)

X_T has shape (31, 2458) and hence, d_theta = - X_T*exp(y-theta) will have shape (31, 1), same as our initial assumed theta shape and now, you can subtract d_theta from theta.