Questions tagged [perceptron]
Perceptron is a basic linear classifier that outputs binary labels.
87
questions
16
votes
2
answers
15k
views
Strange behavior with Adam optimizer when training for too long
I'm trying to train a single perceptron (1000 input units, 1 output, no hidden layers) on 64 randomly generated data points. I'm using Pytorch using the Adam optimizer:
...
- 263
10
votes
3
answers
16k
views
What is the difference between Perceptron and ADALINE?
What is the difference between Perceptron and ADALINE?
- 1,099
9
votes
4
answers
13k
views
Why must a CNN have a fixed input size?
Right now I'm studying Convolutional Neural Networks.
Why must a CNN have a fixed input size?
I know that it is possible to overcome this problem (with fully convolutional neural networks etc...), and ...
9
votes
5
answers
16k
views
Perceptron learning rate
Today I've seen many Perceptron implementations with learning rates. According to Wikipedia:
there is no need for a learning rate in the perceptron algorithm.
This is because multiplying the ...
- 323
6
votes
1
answer
6k
views
difference between empirical risk minimization and structural risk minimization?
I understand the meaning of empirical risk minimization as separate topic and was reading about structural risk minimization, it is hard for me to understand the difference between these two.
I read ...
- 316
5
votes
4
answers
264
views
Is the prediction algorithm absolutely the same for all linear classifiers?
Is the prediction algorithm absolutely the same for all linear classifiers and linear regression algorithms?
As known, any linear classifier can be described as: ...
- 649
5
votes
1
answer
384
views
Whether add bias or not in a perceptron
In some places, perceptron is described as having added bias, while in some places, bias is not added.
Which one is right for you?
- 1,099
5
votes
1
answer
2k
views
Are single input single output neural networks possible?
This might be a weird question but I'm trying to have a deep understanding of how neural networks work theoretically.
I was doing some tests with my perceptron and I decided to test it on a single ...
- 151
4
votes
1
answer
458
views
What's wrong with my implementation of the Adaline algorithm?
I'm working through the textbook called Learning From Data and one of the problems from the first chapter has the reader implement the Adaline algorithm from scratch and I chose to do so using Python. ...
- 51
4
votes
1
answer
423
views
Ambiguity in Perceptron loss function (C. Bishop vs F. Rosenblatt)
Bishop's Perceptron loss
On one hand, it is stated in equation 4.54 of Chris Bishop's book (pattern recognition and machine learning) that the loss function of perceptron algorithm is given by:
$${...
- 1,237
4
votes
1
answer
1k
views
Perceptron weight vector update
I read about the Rosenblatt Perceptron Learning Algorithm. Often there is an explicit note:
It is important to note that all weights in the weight vector are being updated simultaneously
But why ...
- 143
4
votes
1
answer
2k
views
How to implement gradient descent for a tanh() activation function for a single layer perceptron?
I am required to implement a simple perceptron based neural network for an image classification task, with a binary output and a single layer, however I am having difficulties. I have a few problems:
...
- 43
4
votes
1
answer
909
views
How to deal with a sparse matrix when using a perceptron based recommender system?
I'm constrained to use a perceptron based method. I have a user-item matrix filled with rating data on scale of 1 to 5 like this, with around 50% of the matrix with no data:
...
- 258
4
votes
2
answers
1k
views
How do we define a linearly separable problem?
When we talk about Perceptrons, we say that they are limited for approximating functions that are linearly separable, while Neural Networks that use non-linear transformations are not.
I am having ...
- 716
3
votes
1
answer
13k
views
Is the percepetron algorithm's convergence dependent on the linearity of the data?
Does the fact that I have linearly separable data or not impact the convergence of the perceptron algorithm?
Is it always gonna converge if the data is linearly separable and not if it is not ? Is ...
- 273
3
votes
1
answer
378
views
Understanding computations of Perceptron and Multi-Layer Perceptrons on Geometric level
I am currently watching amazing Deep Learning lecture series from Carnegie Melllon University, but I am having little bit of trouble understanding how Perceptrons and MLP are making their decisions on ...
- 716
3
votes
2
answers
641
views
Normalizing the final weights vector in the upper bound on the Perceptron's convergence
The convergence of the "simple" perceptron says that:
$$k\leqslant \left ( \frac{R\left \| \bar{\theta} \right \|}{\gamma } \right )^{2}$$
where $k$ is the number of iterations (in which the weights ...
- 695
3
votes
2
answers
481
views
Is there a relationship between LDA, linear SVMs and Perceptron?
LDA (linear discriminant analysis), SVMs with a linear kernel, and perceptrons are linear classifiers. Is there any other relationship between them, e.g.:
Every decision boundary that can be found by ...
- 18.8k
3
votes
1
answer
96
views
Coding MLP: good practices?
I recently finished coding my own MLP neural network in Python. To make my code easier to read, I separated the MLP, into classes; the network class, the layers class and the neuron class, where the ...
- 53
3
votes
1
answer
3k
views
Learning rate in the Perceptron Proof and Convergence
Every perceptron convergence proof i've looked at implicitly uses a learning rate = 1.
However, the book I'm using ("Machine learning with Python") suggests to use a small learning rate for ...
- 695
3
votes
1
answer
72
views
What does n means in neural network neuron output?
I've found this equation that explains the output of a neuron in a MLP network:
$y(n) = f(\mathbf{w}^T \mathbf{x}(n) + b)$
I can understand the general context, but since i have no background with ...
- 395
3
votes
2
answers
318
views
Geometric interpretation of MLP output
I am really interested in the geometric interpretation of perceptron outputs, mainly as a way to better understand what the network is really doing, but I can't seem to find much information on this ...
3
votes
2
answers
323
views
Perceptron algorithm
I am trying to build a simple linear classifier. I have two classes A and B each with two features [x, y] and hence a 2d dataset. Now, I need to find the equation of the line that separates the two ...
- 131
3
votes
2
answers
1k
views
Why perceptron does not converge on data not linearly separable
This is how I understand the perceptron algorithm.
The perceptron loss function is the hinge loss $\ell(w,x,y) = \max(0, -yw\cdot x)$.
Suppose the data set is $D = \{(x_1,y_1),\dots,(x_n,y_n)\}$ with ...
- 133
3
votes
1
answer
1k
views
Perceptron - Convergence proof
I studied the perceptron algorithm and I'm trying to prove the convergence by myself. However, I'm wrong somewhere and I am not able to find the error.
Assumption:
We assume that there is some $\...
- 161
3
votes
2
answers
2k
views
Perceptron - Which step function to choose
I'm studying Perceptron algorithm. Some books use this step function
1 if x>=0 else -1
where x is a dot product between the weights w and a sample x.
Other ...
- 451
2
votes
1
answer
256
views
Conceptual questions on MLP and Perceptrons
I am facing some confusion regarding the terminologies assocaiated to classification and regression problems esp. using the MLP and Perceptron models.
These are the following:
1) When the data is ...
- 521
2
votes
2
answers
1k
views
How should the hyper parameters be defined for the other algorithms defined for sklearn_crfsuite.CRF
In the example given for sklearn_crfsuite, the parameter space that needs to be passed to a cross_validating class like RandomizedSearchCV is defined as below.
...
2
votes
1
answer
843
views
How do I include the Bias term in the Pegasos algorithm?
I have been asked to implement the Pegasos algorithm as below. It is similar to the Peceptron algorithm but includes eta and lambda terms. However, there is no bias term below and I don't know how ...
2
votes
1
answer
407
views
Threshold logic unit (McCulloch-Pitts 1943) vs. Perceptron (Rosenblatt 1958)
I have searched various sources to find out what distinguishes the McCulloch-Pitts neuron from the perceptron invented by Rosenblatt. In most sources only one of these elements is considered, in ...
- 21
2
votes
1
answer
208
views
SciKit Learn: Multilayer perceptron early stopping, restore best weights
In the SciKit documentation of the MLP classifier, there is the early_stopping flag which allows to stop the learning if there is not any improvement in several ...
- 121
2
votes
3
answers
206
views
About different structures of neural network
https://www.mathworks.com/help/deeplearning/ref/fitnet.html is the tutorial that I am following to understand fitting data to a function. I have few doubts regarding structure and terminologies which ...
- 521
2
votes
1
answer
365
views
Interpreting MLP output
I just wrote an MLP in Python. After having trained it, I pass in some test data to see the result, and I get an array of decimal numbers at the output, rather than the desired binary output.
For ...
- 53
2
votes
1
answer
609
views
Perceptron Primitive Boolean Functions
Thanks for reading.
I'm currently reading Tom Mitchell's Machine Learning (I'm a beginner into ML), and I'm on chapter 4 about perceptrons. I'm really confused about this paragraph:
I understand the ...
- 189
2
votes
0
answers
307
views
How does permutation of training data improve convergence time when training a perceptron or neural network model? [duplicate]
I'm currently studying some basic concepts regarding Deep Learning and Neural Networks with this material.
When discussing the training algorithm for a perceptron, the author states that looping ...
2
votes
1
answer
66
views
Version of Perceptron
If we change the $ywx<0$ condition (for performing update) to $ywx<1$ like in SVM (but without adding regularization to maximize the margin), is there any difference from the basic perceptron (...
- 51
1
vote
1
answer
492
views
How does combining two linear perceptrons create non-linear boundaries?
I don't understand the equation that you get from combining the two linear perceptrons is non-linear?
The video starts with two linear perceptrons with the equations:
$$e1 = 5x_1 -2x_2 - 8 = 0 \...
- 111
1
vote
1
answer
50
views
Multilayer Perceptron: What is the value used to update the weights in the hidden layers?
As i understand for the output layer the error rate is used with the mean squared error function to update the weights.
For the hidden layers as well? Does that make sense?
- 113
1
vote
1
answer
3k
views
How to find bias for perceptron algorithm?
My question is very basic. I am starting with ML and am working on the perceptron algorithm. I successfully computed the weights for this input data:
...
- 195
1
vote
2
answers
1k
views
Learning rate of perception
I don't understand the following statement:
The choice of learning rate m does not matter (for Perceptron) because
it just changes the scaling of w (weights). The site with this statement that ...
- 49
1
vote
1
answer
84
views
Visualizing a Perceptron
I wanted to visualize how a perceptron learns, so I made a class that performs gradient descent. To show the decision, I plot a plane showing where positive examples and negative examples are, ...
- 113
1
vote
1
answer
34
views
What issue is there, when training this network with gradient descent? [closed]
Suppose we have the following fully connected network made of perceptrons with a sign function as the activation unit, what issue arises, when trying to train this network with gradient descent?
- 131
1
vote
3
answers
61
views
Why there is only one type of artificial neuron?
I find it strange that so many deep learning tricks and improvements have been invented in the past decade but I never heard about someone trying out different models of the artificial neuron other ...
- 195
1
vote
2
answers
35
views
Should a bias neuron be connected to previous neurons?
I'm trying to add a bias neuron to my neural network that uses the backpropagation algorithm. I'm trying to figure out how I should go about this, should I treat the bias neuron as a regular neuron? ...
- 131
1
vote
1
answer
427
views
Confusion on Delta Rule and Error
I'm currently reading Mitchell's book for Machine Learning, and he just started gradient descent. There's one part that's really confusing me.
At one point, he gives this equation for the error of a ...
- 189
1
vote
1
answer
6k
views
sklearn.GridSearchCV predict method not providing the best estimate and accuracy score
I was playing around with the credit default dataset in UCI
("https://archive.ics.uci.edu/ml/machine-learning-databases/00350/default%20of%20credit%20card%20clients.xls")
These are the steps i have ...
- 143
1
vote
1
answer
638
views
Trouble with accuracy of multiclass perceptron
I have built a multiclass perceptron, but it has low accuracy (around 80%). I think I'm missing something. One possibility is that I should add a bias, but I'm not sure how to incorporate that.
The ...
- 141
1
vote
1
answer
1k
views
Perceptron learning rate irrelevant in convergence [duplicate]
Via this MIT document i studied the single layer Perceptron convergence proof (= maximum number of steps).
In the convergence proof inside this document , the learning rate is implicitly defined as 1 ...
- 695
1
vote
1
answer
233
views
What is the hypothesis space used by this AND gate Perceptron?
Per this post
The hypothesis space used by a machine learning system is the set of
all hypotheses that might possibly be returned by it.
Per this post, the Perceptron algorithm makes prediction
...
- 157
1
vote
1
answer
161
views
neural networks error function: is global minimum desirable?
In "Elements of statistical learning" page 395 the authors state that, relative to R(θ), the regression/classification error function in a neural network such as a multi layer perceptron:
Typically ...
- 35