How to solve Ax = b for A [closed]

Given two know vector x, and b (dimension 3*1 for example), what are the ways to approximate the matrix A (dimension 3*3) so that the equality Ax=b is as close as possible (something like least square), knowing that very likely the system does not have an answer.

Here you actually do not have a system of linear equations that needs to be seen at a whole and solved together. Here you have 3 independent equations, each of them with infinite valid answers. So:

$$\begin{bmatrix} a_{1} & a_{2} & a_{3}\\ a_{4} & a_{5} & a_{6}\\ a_{7} & a_{8} & a_{9} \end{bmatrix}\times \begin{bmatrix} x_{1}\\ x_{2} \\ x_{3} \end{bmatrix} = \begin{bmatrix} b_{1}\\ b_{2} \\ b_{3} \end{bmatrix}$$

equals solving

$$a_{1}x_{1} + a_{2}x_{2} + a_{3}x_{3} = b_{1}$$

and

$$a_{4}x_{1} + a_{5}x_{2} + a_{6}x_{3} = b_{2}$$

and

$$a_{7}x_{1} + a_{8}x_{2} + a_{9}x_{3} = b_{3}$$

independently.

Each of them in for example in $$\mathbb{R}$$ has infinite right answers. For instance in $$a_{1}x_{1} + a_{2}x_{2} + a_{3}x_{3} = b_{1}$$ you just need to randomly choose two of $$a_{i}$$s and simply get the last one. For example for known $$x$$ vector $$\begin{bmatrix} 1\\ 2 \\ 3 \end{bmatrix}$$, $$b$$ vector $$\begin{bmatrix} 4\\ 5 \\ 6 \end{bmatrix}$$ and $$a_{1}=1$$ and $$a_{2}=2$$, we get $$a_{3}$$:

$$5 + 3\times a_{3} = 4$$

so

$$a_{3}=-\frac{1}{3}$$!

Same procedure applies to other two equations as well. Randomly choose two $$a_{i}$$s for each equation and get the last one.

PS: I feel either something is missing in your question or I did not understand well. In either cases, please drop a comment here and I will update the answer.

• Thanks, Kasra. The system will likely not have an answer and should be approximated. Do you see a way to do this with a method like Least square ? – silkAdmin May 26 '20 at 11:08
• I still feel I am misunderstood. You again used the word "system". This is certainly not a system if A could be anything. If rows of A are related to eachother, then it is a system. But now, as I said in my answer, multiplying first row of A to x creates the first element of b regardless of rest of elements in A! I assume there are info about A that you did not include. For instance if rows of A are same variables, then you are basically finding the intersection of 3 planes. – Kasra Manshaei May 26 '20 at 11:32
• As you mentioned there is no answer for this and according to my answer there is actually infnitely many, I feel like there is such constraint (like my previous comment) on A. In case of my previous comment, 3 planes might not have an intersection so the system has no answer and should be approaximated. I might be wrong but I think some info about A is missing in your question. – Kasra Manshaei May 26 '20 at 11:37

As Kashra said, your "system" has an infinite number of valid solutions. However, there is one "canonical" solution, that might make more sense than others, depending what you are after.

A matrix is actually a way of writing down a linear operator. A linear operator transforms one vector into another, so when you say

$$A \cdot x = b$$

you are basically saying that $$A$$ performs a transformation on $$x$$, so that it becomes $$b$$. It is somewhat easier to visualise if we talk about 2D vectors (i.e. vectors in a plane) and $$2 \times 2$$ matrices. Transforming $$x$$ into $$b$$ means rotating and scaling it by a suitable angle and factor.

A rotation in 2D is given by the matrix

$$\begin{bmatrix} \cos \varphi & -\sin \varphi \\ \sin \varphi & \cos \varphi \end{bmatrix}$$

and scaling is simply multiplication by a scalar, say, $$\lambda$$. So your task of solving for $$A$$ reduces to finding $$\varphi$$ and $$\lambda$$.

Now, we know that dot product is

$$a \cdot b = \left\lVert a \right\rVert \cdot \left\lVert b \right\rVert \cdot \cos \varphi$$

from which you can derive $$\varphi$$. Scaling is even easier: to scale $$a$$ to be as long as $$b$$ you just need to multiply it by

$$\lambda = \frac{\left\lVert b \right\rVert}{\left\lVert a \right\rVert}.$$

For your 3D case it is a little bit more complicated, but the principle remains the same. You'll need to rotate along two axes, but scaling remains the same.

There are infinitely many solutions except in corner cases like x = 0 or something. In your case here, you could simply find a solution with $$A = b x^+$$ where $$x^+$$ is the Moore-Penrose pseudoinverse. In R that would be something like A = b %*% ginv(x), where ginv is from the MASS library.

The system will likely not have an answer and should be approximated. Do you see a way to do this with a method like Least square
Yes you can do that. The Linear Regression is done using this method.
If the b is not in the column space of A, to get an approximated solution, the vector b is projected onto the column space of A. Now with the new b' a solution of the system is found. This can also work if b is in the column space of A` since its projection will be itself