Given a value $y$ and some values $x_1, ..., x_n$. How do I find weights $w_1, ... w_n$ so that the $error = y - w_1*x_1 + ... w_n*x_n$ is minimal, where the weights have to sum up to 1 or a different value?
$\begingroup$
$\endgroup$
0
Add a comment
|
2 Answers
$\begingroup$
$\endgroup$
4
The name of the model you are looking for is: Constrained Estimation of Ordinary Least Squares, this model allows you to introduce linear constrains to your estimation by just modifying the matrix expression of the OLS estimation.
The resulting model should satisfy the expression
$$Q^T\beta = c$$
It is solved with the equation: $$\hat{\beta^c} = \hat{\beta}-(X^TX)^{-1}Q(Q^T(X^TX)^{-1}Q)^{-1}(Q^T\hat{\beta}-c)$$
For your problem, $Q=[1,1,1]$, $c=1$, $X$ and $Y$ are the matrixes for your data.
answered Jun 5, 2019 at 13:24
-
$\begingroup$ Tanks for your help. Do you have a source where I can read a little more? $\endgroup$– RobinCommented Jun 5, 2019 at 13:51
-
$\begingroup$ abenkhalifa.files.wordpress.com/2016/12/…. Section 1.4 $\endgroup$ Commented Jun 5, 2019 at 14:17
-
$\begingroup$ the link is broken, do you have an alternate link? $\endgroup$– aadil095Commented Dec 12, 2021 at 6:43
-
$\begingroup$ Sorry to hear that: en.wikipedia.org/wiki/… $\endgroup$ Commented Dec 28, 2021 at 13:35
$\begingroup$
$\endgroup$
This problem is called constrained optimization, where your constraint is that sum of the weights is x (with for example x=1).
It would probably be insightful, and not too complex, to write your own implementation of gradient descent, that respects this constraint. To do this, you would scale the weight vector (w) after each update so that it sums to x.
Another possibility is to use an existing implementation of constrained optimization, such as in scipy. I think that the constraint you are asking for is as follows:
from scipy.optimize import LinearConstraint
linear_constraint = LinearConstraint([[1] * n], [x], [x])
Where n is the number of weights, and x is the length that you want to constrain w to.
