My question is the same as here: How to force weights to be non-negative in Linear regression

Except that I can only use Numpy (I cannot use Scipy or Scikit Learn). Indeed, I am running my Python script on a server which doesn't include these modules.

Is there any solution ?


The sklearn implementation of Lasso that can force non-negative weights (as in this answer) is based on the coordinate descent algorithm. You can reimplement it, using for example coordinate-wise Newton method. For simplicity, I did not inclide intercept into the model:

import numpy as np
# generate the data
n = 1000
X = np.random.normal(size=(n, 5))
y = np.dot(X, [0,0,1,2,3]) + np.random.normal(size=n, scale=3)
# initial solution (with some negative weights):
beta = np.dot(np.linalg.inv(np.dot(X.transpose(),X)), np.dot(X.transpose(), y))
# clip the solution from below with zero
prev_beta = beta.copy()
beta = np.maximum(beta, 1)
# improve the solution by restricted coordinate-wise Newton descent
hessian = np.dot(X.transpose(), X)
while not (prev_beta == beta).all():
    prev_beta = beta.copy()
    for i in range(len(beta)):
        grad = np.dot(np.dot(X,beta)-y, X)
        beta[i] = np.maximum(0, beta[i] - grad[i] / hessian[i,i])

This code will output initial and final beta's:

[-0.01404546 -0.02633036  1.06028543  1.99696564  2.93511618]
[ 0.          0.          1.05919989  1.99673774  2.93442334]

You can see that OLS beta differ from your optimal beta not only in the first two coefficients (that have been negative), but the rest of coefficients were also adjusted.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.