0
$\begingroup$

I started a course in Deep Learning. I'm trying to make an example in order to explain to myself how the weights are found mathematically. If what I wrote below is nonsense I'll be glad to hear an explanation. Thanks.

So, for a given image we do WX+b. We get some vector Y and then we compare it to a desired label vector L according to enter image description here. I'm assuming that we calculate D with "Cosine Similarity". For simplicity S(Y)==Y. So what we're trying to do is to calculate enter image description here so it will be one. Let’s say we have image X of the letter “a” enter image description here and two labels (“a”, “b”). Then enter image description here . We want to calculate W and b for which we will get such vector enter image description here that when we’ll insert it into enter image description here we’ll get zero. We convert X to a vectorenter image description here. Since we have 2 labels and size of the X is 9, the W and b are the following: enter image description here. So, we get: enter image description here. This gives us the following system of equations: enter image description here. So, now we need to solve the following enter image description here.

If what I wrote above is not nonsense, I don't quite understand where finding minimum is applied?

Improve this question
$\endgroup$
0

2 Answers 2

Reset to default
1
$\begingroup$

This is interesting !?! The way you think about it is as solving a linear system equation, which is not exactly the case. Imagine if you have thousands of data points , and you try to find the set of weights that solves ( satisfies) these points, what if an exact solution is not exist. What if there is no set of weights that exactly solves all these points. It becomes an optimization problem. What is the set of weight that minimizes the error. In this case the error is the difference between the actual output and the output you get by using the current weights. In fact, in current machine learning technology, it is difficult even to construct such a linear system. We use optimization techniques to solve this and find the optimal set of weights

Improve this answer
$\endgroup$
0
$\begingroup$

So I think one important point you need to note, we try to minimize the error and not the weights. I didn't get exactly the number of nodes in each layer you have from your example, but I will give a generic understanding.

So from your example, to obtain cos(θ) you use L which is the result but this value is unknown. You need to first train on the weights and then calculate resultant L. Then check the difference between this resultant L and the actual L. Please separate the knowns and unknowns or input and output for a better understanding.

I think you should check this to actually get a deep understanding. A

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.