How to solve Ax = b for A [closed]

Question

Given two know vector x, and b (dimension 3*1 for example), what are the ways to approximate the matrix A (dimension 3*3) so that the equality Ax=b is as close as possible (something like least square), knowing that very likely the system does not have an answer.

Answer 1

Here you actually do not have a system of linear equations that needs to be seen at a whole and solved together. Here you have 3 independent equations, each of them with infinite valid answers. So:

$\begin{bmatrix} a_{1} & a_{2} & a_{3}\\ a_{4} & a_{5} & a_{6}\\ a_{7} & a_{8} & a_{9} \end{bmatrix}\times \begin{bmatrix} x_{1}\\ x_{2} \\ x_{3} \end{bmatrix} = \begin{bmatrix} b_{1}\\ b_{2} \\ b_{3} \end{bmatrix}$

equals solving

$a_{1}x_{1} + a_{2}x_{2} + a_{3}x_{3} = b_{1}$

and

$a_{4}x_{1} + a_{5}x_{2} + a_{6}x_{3} = b_{2}$

and

$a_{7}x_{1} + a_{8}x_{2} + a_{9}x_{3} = b_{3}$

independently.

Each of them in for example in $\mathbb{R}$ has infinite right answers. For instance in $a_{1}x_{1} + a_{2}x_{2} + a_{3}x_{3} = b_{1}$ you just need to randomly choose two of $a_{i}$s and simply get the last one. For example for known $x$ vector $\begin{bmatrix} 1\\ 2 \\ 3 \end{bmatrix}$, $b$ vector $\begin{bmatrix} 4\\ 5 \\ 6 \end{bmatrix}$ and $a_{1}=1$ and $a_{2}=2$, we get $a_{3}$:

$5 + 3\times a_{3} = 4$

so

$a_{3}=-\frac{1}{3}$!

Same procedure applies to other two equations as well. Randomly choose two $a_{i}$s for each equation and get the last one.

PS: I feel either something is missing in your question or I did not understand well. In either cases, please drop a comment here and I will update the answer.

Answer 2

As Kashra said, your "system" has an infinite number of valid solutions. However, there is one "canonical" solution, that might make more sense than others, depending what you are after.

A matrix is actually a way of writing down a linear operator. A linear operator transforms one vector into another, so when you say

$$ A \cdot x = b $$

you are basically saying that $A$ performs a transformation on $x$, so that it becomes $b$. It is somewhat easier to visualise if we talk about 2D vectors (i.e. vectors in a plane) and $2 \times 2$ matrices. Transforming $x$ into $b$ means rotating and scaling it by a suitable angle and factor.

A rotation in 2D is given by the matrix

$$ \begin{bmatrix} \cos \varphi & -\sin \varphi \\ \sin \varphi & \cos \varphi \end{bmatrix} $$

and scaling is simply multiplication by a scalar, say, $\lambda$. So your task of solving for $A$ reduces to finding $\varphi$ and $\lambda$.

Now, we know that dot product is

$$ a \cdot b = \left\lVert a \right\rVert \cdot \left\lVert b \right\rVert \cdot \cos \varphi $$

from which you can derive $\varphi$. Scaling is even easier: to scale $a$ to be as long as $b$ you just need to multiply it by

$$ \lambda = \frac{\left\lVert b \right\rVert}{\left\lVert a \right\rVert}. $$

For your 3D case it is a little bit more complicated, but the principle remains the same. You'll need to rotate along two axes, but scaling remains the same.

Answer 3

There are infinitely many solutions except in corner cases like x = 0 or something. In your case here, you could simply find a solution with $A = b x^+$ where $x^+$ is the Moore-Penrose pseudoinverse. In R that would be something like A = b %*% ginv(x), where ginv is from the MASS library.

Answer 4

Regarding your comment

The system will likely not have an answer and should be approximated. Do you see a way to do this with a method like Least square`

Yes you can do that. The Linear Regression is done using this method.

If the b is not in the column space of A, to get an approximated solution, the vector b is projected onto the column space of A. Now with the new b' a solution of the system is found. This can also work if b is in the column space of A since its projection will be itself