3
$\begingroup$

I am using a linear regression using scikit-learn in python. However, I would like to force the weights to add to $1$. Is there a way to do this? I know that I need to add a constraint but am not able to figure out how. My regression looks something like this $Y= a_0 + bX_1 + (1-b)X_2$

Thanks in advance

$\endgroup$
7
  • 1
    $\begingroup$ Linear Regression estimator has a coef_ attribute and an intercept_ attribute. coef_ contains the estimated weights, whereas the intercept_ contains the bias(es). You can directly modify their values by adding a 1. But I'm more interested in knowing that why would you do such a nasty thing in the first place? $\endgroup$ Feb 22 '19 at 19:51
  • $\begingroup$ Welcome to the site! Are you able to share your use case? Asking weights to add up to a certain number doesn't really make a whole lot of sense and it basically makes this no longer be a data science effort. What exactly are you trying to achieve here? $\endgroup$ Feb 22 '19 at 19:57
  • 1
    $\begingroup$ @SyedAliHamza he doesn't want to add a 1 he wants the sum of the weights to add up to 1. So, it's a slight correction to how you understood the question, but an equally nasty request :-) $\endgroup$ Feb 22 '19 at 19:58
  • $\begingroup$ Hi @I_Play_With_Data, I am working with some interest rate data. In interest rate shock scenarios, the shock for the dependant variable would be inconsistent if the weights do not add to 1. $\endgroup$
    – Shreyans
    Feb 22 '19 at 20:03
  • $\begingroup$ Interesting. I don't know a lot about that particular domain so maybe someone else can give you better guidance. But, my gut tells me that maybe regression wouldn't be the best approach for this. You're implying that your dependent results have a limited/restricted range, and that's OK, it may be that another model would be better suited for this. Is there a reason why you picked regression? Is it for transparency purposes? $\endgroup$ Feb 22 '19 at 20:06
3
$\begingroup$

Suppose the problem formulation is

$$\min_{a,b} \sum_{i=1}^n(y_i -a_0-\sum_{j=1}^pb_jx_{ij} )^2$$

subject to $$\sum_{j=1}^p b_j=1$$ $$b \ge 0$$

then this is an instance of a quadratic programming problem.

There are solvers such as Gurobi (free for academia) and CVXOPT (freeware) that can manage a quadratic programming problem. An example of CVXOPT code to solve a QP can be found here.

Note that the objective function can be written as

\begin{align}&\|Y-a_0e-Xb\|^2\\&=\left\|Y-\begin{bmatrix}e & X \end{bmatrix}\begin{bmatrix}a_0 \\ b\end{bmatrix}\right\|^2\\ &=\begin{bmatrix}a_0 & b \end{bmatrix}\begin{bmatrix} e^T\\X^T\end{bmatrix}\begin{bmatrix} e &X\end{bmatrix}\begin{bmatrix}a_0 \\ b \end{bmatrix}-2Y^T\begin{bmatrix} e &X\end{bmatrix}\begin{bmatrix}a_0 \\ b \end{bmatrix}+\|Y\|^2\end{align}

Also, you might like to check with people in your field what is the common package that they use.

$\endgroup$
1
  • 2
    $\begingroup$ great answer! I would also like to add a C++/python solver that I have found useful for optimization problems esa.github.io/pagmo2 $\endgroup$
    – pcko1
    Feb 23 '19 at 16:09

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.