Questions tagged [perceptron]
Perceptron is a basic linear classifier that outputs binary labels.
87
questions
2
votes
1
answer
363
views
Interpreting MLP output
I just wrote an MLP in Python. After having trained it, I pass in some test data to see the result, and I get an array of decimal numbers at the output, rather than the desired binary output.
For ...
- 53
3
votes
1
answer
96
views
Coding MLP: good practices?
I recently finished coding my own MLP neural network in Python. To make my code easier to read, I separated the MLP, into classes; the network class, the layers class and the neuron class, where the ...
- 53
5
votes
4
answers
262
views
Is the prediction algorithm absolutely the same for all linear classifiers?
Is the prediction algorithm absolutely the same for all linear classifiers and linear regression algorithms?
As known, any linear classifier can be described as: ...
- 649
1
vote
1
answer
426
views
Confusion on Delta Rule and Error
I'm currently reading Mitchell's book for Machine Learning, and he just started gradient descent. There's one part that's really confusing me.
At one point, he gives this equation for the error of a ...
- 189
2
votes
1
answer
609
views
Perceptron Primitive Boolean Functions
Thanks for reading.
I'm currently reading Tom Mitchell's Machine Learning (I'm a beginner into ML), and I'm on chapter 4 about perceptrons. I'm really confused about this paragraph:
I understand the ...
- 189
0
votes
1
answer
231
views
Why can't the XOR linear inseparability problem be solved with one perceptron - like this?
Consider a perceptron where $w_0=1$ and $w_1=1$:
Now, say we use an activation function
$f(x)=1,~for~x=1$$~~~~~~~~~~~~~0, otherwise$
The output is then summarised as:
$x_0~~~~~x_1~~~~~w_0*x_0 + w_1*...
- 160
1
vote
1
answer
175
views
Effect of adding extra unrelated features to linear perceptron
Suppose that we are training a linear regressor (perceptron). Adding extra features that are not related to the target (e.g. randomly generated values) before training will typically ____ our training ...
- 141
1
vote
1
answer
6k
views
sklearn.GridSearchCV predict method not providing the best estimate and accuracy score
I was playing around with the credit default dataset in UCI
("https://archive.ics.uci.edu/ml/machine-learning-databases/00350/default%20of%20credit%20card%20clients.xls")
These are the steps i have ...
- 143
0
votes
1
answer
193
views
Why my perceptron doesn't train well and produces bad results on test data?
I am a newbie in Machine learning and I am writing a small code for Perceptron. This is the first time I am writing code in Python.
I have four training data points (X). As they are used for ...
- 185
3
votes
2
answers
1k
views
Why perceptron does not converge on data not linearly separable
This is how I understand the perceptron algorithm.
The perceptron loss function is the hinge loss $\ell(w,x,y) = \max(0, -yw\cdot x)$.
Suppose the data set is $D = \{(x_1,y_1),\dots,(x_n,y_n)\}$ with ...
- 133
1
vote
2
answers
1k
views
Learning rate of perception
I don't understand the following statement:
The choice of learning rate m does not matter (for Perceptron) because
it just changes the scaling of w (weights). The site with this statement that ...
- 49
1
vote
1
answer
345
views
Machine Learning Perceptron Algorithm
I'm studying Machine Learning using Sebastian Raschka's book.
Wonder if someone could please help me to confirm if I have the following steps correct if I apply Perceptron Algorithm to Iris dataset ...
- 139
10
votes
3
answers
16k
views
What is the difference between Perceptron and ADALINE?
What is the difference between Perceptron and ADALINE?
- 1,099
5
votes
1
answer
380
views
Whether add bias or not in a perceptron
In some places, perceptron is described as having added bias, while in some places, bias is not added.
Which one is right for you?
- 1,099
1
vote
1
answer
489
views
How does combining two linear perceptrons create non-linear boundaries?
I don't understand the equation that you get from combining the two linear perceptrons is non-linear?
The video starts with two linear perceptrons with the equations:
$$e1 = 5x_1 -2x_2 - 8 = 0 \...
- 111
1
vote
0
answers
1k
views
Optimizing an averaged perceptron algorithm using numpy and scipy instead of dictionaries
So I'm trying to write an averaged perceptron algorithm (page 48 here for the equation) in python. Instead of storing the historical weights, I simply accumulate the weights and then multiply ...
- 101
1
vote
1
answer
636
views
Trouble with accuracy of multiclass perceptron
I have built a multiclass perceptron, but it has low accuracy (around 80%). I think I'm missing something. One possibility is that I should add a bias, but I'm not sure how to incorporate that.
The ...
- 141
1
vote
1
answer
970
views
Predict method of the perceptron algorithm
Can someone explain to me how the predict method of the perceptron algorithm works?
...
- 113
5
votes
1
answer
2k
views
Are single input single output neural networks possible?
This might be a weird question but I'm trying to have a deep understanding of how neural networks work theoretically.
I was doing some tests with my perceptron and I decided to test it on a single ...
- 151
1
vote
1
answer
20
views
Weights and bias' relative to preprocessed X
I am currently using sklearn scale to preprocess my X data before being put into a perceptron - mean/stddev so as to prevent the ...
- 113
0
votes
1
answer
4k
views
number of neurons for mnist dataset using mlp?
I am trying to find out what is optimum number of neurons that can be used in MNIST dataset(60,000 training and 10,000 testing data). I build a single hidden layer model using keras,with relu ...
- 463
1
vote
1
answer
1k
views
Perceptron learning rate irrelevant in convergence [duplicate]
Via this MIT document i studied the single layer Perceptron convergence proof (= maximum number of steps).
In the convergence proof inside this document , the learning rate is implicitly defined as 1 ...
- 695
3
votes
1
answer
3k
views
Learning rate in the Perceptron Proof and Convergence
Every perceptron convergence proof i've looked at implicitly uses a learning rate = 1.
However, the book I'm using ("Machine learning with Python") suggests to use a small learning rate for ...
- 695
3
votes
2
answers
640
views
Normalizing the final weights vector in the upper bound on the Perceptron's convergence
The convergence of the "simple" perceptron says that:
$$k\leqslant \left ( \frac{R\left \| \bar{\theta} \right \|}{\gamma } \right )^{2}$$
where $k$ is the number of iterations (in which the weights ...
- 695
3
votes
1
answer
1k
views
Perceptron - Convergence proof
I studied the perceptron algorithm and I'm trying to prove the convergence by myself. However, I'm wrong somewhere and I am not able to find the error.
Assumption:
We assume that there is some $\...
- 161
3
votes
2
answers
2k
views
Perceptron - Which step function to choose
I'm studying Perceptron algorithm. Some books use this step function
1 if x>=0 else -1
where x is a dot product between the weights w and a sample x.
Other ...
- 451
16
votes
2
answers
15k
views
Strange behavior with Adam optimizer when training for too long
I'm trying to train a single perceptron (1000 input units, 1 output, no hidden layers) on 64 randomly generated data points. I'm using Pytorch using the Adam optimizer:
...
- 263
4
votes
1
answer
2k
views
How to implement gradient descent for a tanh() activation function for a single layer perceptron?
I am required to implement a simple perceptron based neural network for an image classification task, with a binary output and a single layer, however I am having difficulties. I have a few problems:
...
- 43
3
votes
1
answer
13k
views
Is the percepetron algorithm's convergence dependent on the linearity of the data?
Does the fact that I have linearly separable data or not impact the convergence of the perceptron algorithm?
Is it always gonna converge if the data is linearly separable and not if it is not ? Is ...
- 273
0
votes
1
answer
97
views
An optimization problem involving perceptron
What I do not understand is that is there only one input for each weights or there is a vector of inputs from 1 to n?
I also dont get the notation for inputs and outputs? Is that a notation or a ...
- 11
2
votes
2
answers
1k
views
How should the hyper parameters be defined for the other algorithms defined for sklearn_crfsuite.CRF
In the example given for sklearn_crfsuite, the parameter space that needs to be passed to a cross_validating class like RandomizedSearchCV is defined as below.
...
1
vote
0
answers
264
views
Why is a perceptron initialized with a random line?
I'm setting up a single perceptron for doing linear classification.
Why is the perceptron initialized with random weights and a random bias instead of just having all of the weights set to zero ...
- 111
1
vote
1
answer
207
views
Criterion for Firing a perceptron
The criterion for firing a perceptron is as follows
Why is it that when the function $w \cdot x + b = 0$ the output is zero as well. Why couldn't it have been set to 1?
If one were to simulate the ...
- 33
4
votes
1
answer
1k
views
Perceptron weight vector update
I read about the Rosenblatt Perceptron Learning Algorithm. Often there is an explicit note:
It is important to note that all weights in the weight vector are being updated simultaneously
But why ...
- 143
3
votes
2
answers
480
views
Is there a relationship between LDA, linear SVMs and Perceptron?
LDA (linear discriminant analysis), SVMs with a linear kernel, and perceptrons are linear classifiers. Is there any other relationship between them, e.g.:
Every decision boundary that can be found by ...
- 18.8k
9
votes
5
answers
16k
views
Perceptron learning rate
Today I've seen many Perceptron implementations with learning rates. According to Wikipedia:
there is no need for a learning rate in the perceptron algorithm.
This is because multiplying the ...
- 323
4
votes
1
answer
909
views
How to deal with a sparse matrix when using a perceptron based recommender system?
I'm constrained to use a perceptron based method. I have a user-item matrix filled with rating data on scale of 1 to 5 like this, with around 50% of the matrix with no data:
...
- 258