1
$\begingroup$

I don't know if this is a good question. I would like to use machine learning to approximate a matrix. For example, a very simple one as

\begin{align} A=\begin{pmatrix} 1 &0 \\ 0& 2 \end{pmatrix}. \end{align}

I can easily generate many input-output pairs by left-multiplying the input with $A$. The input are random vectors of dimension $2\times 1$. I use the following neural network to do the job

input layer : 2 nodes;

hidden layer : 1 node;

output layer : 2 nodes.

A very simple Matlab code is written as follows, but it doesn't work. The testing results simply blow up, showing NaN.

m=1000;                % number of input-output pairs      
alp=0.001;             % learning rate

inp=rand(2,m);         % random input
oup=[1 0;0 2]*inp;     % constructing output

W1=rand(1,2);          % initialise the weights
W2=rand(2,1);

for epoch=1:1000       % running for 1000 epochs
    for i=1:m          % do the forward propagation and backpropagation
       a1 = inp(:,i);
       z2 = W1*a1;    a2 = actf(z2);
       a3 = actf(W2*a2);

       delta2 = a3 - oup(:,i);
       delta1 = actfd(z2).*(W2'*delta2);   

       W2 = W2 + alp*delta2*a2';
       W1 = W1 + alp*delta1*a1'; 
    end
end

actf here is the activation function $f(x)=\frac{1}{1+e^{-x}}$ and actfd is its derivative $f(x)=x(1-x)$. I see no problem of overfitting since in $A$ matrix there are 4 entries and the NN also has 4 weights in total. But the code seems to be wrong.

I'm wondering if people here know how to solve this problem (approximating a matrix using ML)? Did I do anything wrong in the code above?

Thanks in advance for your help. Any comments will be appreciated.

$\endgroup$

closed as unclear what you're asking by Sean Owen Nov 25 '18 at 0:13

Please clarify your specific problem or add additional details to highlight exactly what you need. As it's currently written, it’s hard to tell exactly what you're asking. See the How to Ask page for help clarifying this question. If this question can be reworded to fit the rules in the help center, please edit the question.

  • $\begingroup$ Does it work? what are you asking for help with? $\endgroup$ – Sean Owen Nov 25 '18 at 0:13
  • $\begingroup$ edited accordingly. $\endgroup$ – jengmge Nov 25 '18 at 0:48