I have a problem which i have attached as an image.

Problem is in image attached enter image description here

what I understand

error function is given by: $e(y, \hat y)=0$ if $y \cdot a(x-b) \ge 1$ or $e(y, \hat y) = 1-y\cdot a\cdot (x-b)$ if $y a(x-b) < 1$.

Gradient descent at current $t$ is (1,3).

Gradient Descent($E_{in}(a,b)$) as per definition should be equal to partial derivative of the equation of $E_{in}(a,b)$ wrt $a$ and $b$. ($w$ is equivalent to $[a, b]$, according to me)

My doubt

Please note that when i say sum over N points, i mean to use that greek symbol used for summing of N points

  1. I am not sure how would I calculate the partial derivative in this case. When we say to fix '$a$' and vary '$b$', does it mean to find differentiation only wrt '$b$'? Which would mean that gradient($E_{in}$)= $-1 / N (\sum_{i=1}^N y_i a)$. But this removes dependence on $x$ and that's why I doubt my approach.

  2. The final equation for whom derivative needs to be done is: $1/N \sum_{i=1}^N 1-ya(x-b)$ for N points misclassified. But as per the dataset, no point is misclassified as the error function for each point with the (a,b)=(1,3) is equal to 0.

    For point 1, where x=1.2 and y = -1, as $y \cdot a \cdot (x-b) = (-1)\cdot (1)\cdot (1.2-3)=+1.8$. This means that $e(y_1, h(x_1)) = 0$.

So what should be the answer to this question?

I hope my doubt in the question is cleare to audience. But in any case please let me know if there is something that I am unable to explain.


1 Answer 1


It appears to me that your gradient calculation is off:

$$ \frac{\partial E_{in}(a,b)}{\partial{b}} = \frac{1}{|D|} \sum_i \frac{\partial e(y_i,h_{a,b}(x_i))}{\partial b} $$

Where (with some lenient notation) :

$$ \frac{\partial e(y_i,h_{a,b}(x_i))}{\partial b} = \frac{\partial e(y,s)}{\partial{s}} * \frac{\partial h_{a,b}(x_i)}{\partial{b}} $$

We can show that :

$$ \frac{\partial e(y,s)}{\partial{s}} = -y $$ if $$1-y.s >0$$ and, otherwise: $$ \frac{\partial e(y,s)}{\partial{s}} = 0 $$

This give you an expected dependency on x, as the value change the criterion.

And :

$$ \frac{\partial h_{a,b}(x_i)}{\partial{b}} = -a $$

So you have :

$$ \frac{\partial E_{in}(a,b)}{\partial{b}} = \frac{1}{|D|} \sum_i \mathbb{1}_{y.x<1}*y*a $$


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.