I started a course in Deep Learning. I'm trying to make an example in order to explain to myself how the weights are found mathematically. If what I wrote below is nonsense I'll be glad to hear an explanation. Thanks.
So, for a given image we do WX+b. We get some vector Y and then we compare it to a desired label vector L according to . I'm assuming that we calculate D with "Cosine Similarity". For simplicity S(Y)==Y. So what we're trying to do is to calculate so it will be one. Let’s say we have image X of the letter “a” and two labels (“a”, “b”). Then . We want to calculate W and b for which we will get such vector that when we’ll insert it into we’ll get zero. We convert X to a vector. Since we have 2 labels and size of the X is 9, the W and b are the following: . So, we get: . This gives us the following system of equations: . So, now we need to solve the following .
If what I wrote above is not nonsense, I don't quite understand where finding minimum is applied?