0
$\begingroup$

This slide here defines a hyperplane by using a normal vector $w$.

enter image description here

And claims that:

"The hyperplane passes through origin"

Is it a general default parameter? w = [-1,1]?

I see a lot of figures with different directions.

enter image description here

Where $w$ and $b$ play in the opposite direction.

$\endgroup$
1
$\begingroup$

This question is for general hyperplane.

The normal direction is the direction that is perpendicular (also known as normal) to the plane.

Take a piece of paper (treat it as a plane), hold a pencil perpendicular to the paper. The pencil is parallel to the normal direction.

There is no such thing as the default direction as you see, a piece of paper that is oriented in a different manner would have a different normal direction.

| improve this answer | |
$\endgroup$
1
$\begingroup$

The perceptron can be described by the normal vector $\boldsymbol{w}$, a bias $b$ and the signum function $\text{sgn}$ as an activation function.

The output of the perceptron for the input $\boldsymbol{x}$ is given by

$$y = \text{sgn}(\boldsymbol{w}^T\boldsymbol{x}+b).$$

By defining the output $y=1$ for the positive class and $y=-1$ for the negative class. The optimization will lead to a normal vector pointing towards the positive class. The bias will shift the hyperplane out of the origin. Normally, the weight vector $\boldsymbol{w}$ is initialized with normally distributed random noise. In the lecture notes, some arbitrary simple weight vector was used for the example.

Why does the weight vector point into the direction of the positive class?

To answer this question we will look at an artificial example in the following picture. enter image description here The blue line at the $y$-axis is the hyperplane/decision boundary. The circles with the plus sign are a positive class and the circles with the cross are the negative class. The red arrow is the normal vector of the hyperplane. In this example it has the form $\boldsymbol{w}=[\mp 1,0]^T$. If we assign $+1$ to the positive class the dot product of $\boldsymbol{w}^T\boldsymbol{x}$ will be positive if the weight vector $\boldsymbol{w}$ points into the direction of the positive class. The dot product will be negative for the negative class which should have the output value $-1$. In our example the bias $b=0$. Hence, the argument of the $\text{sgn}$ function will have the correct sign to produce the correct class labels. If we invert the class labels the weight vector will flip its direction in order to point towards the negative class.

| improve this answer | |
$\endgroup$
0
$\begingroup$

You need to distinguish between the parameter space and the feature space. In the context of the perceptron you often see hyperplanes in the parameter space (i.e. $w$ on the axis) whereas in typical classifications you often see the decision boundary depicted in terms of feature values ($x$, $x_1$, etc.).

Without looking at the sources, it seems like the first image shows parameter space and the second the feature space. So they are not really comparable in any direct, meaningful way.

In feature space, a hyperplane can point in any direction depending on the features. In the parameter space for a perceptron the hyperplane always goes through the origin. The second image just depicts something different.

| improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.