Skip to main content

Questions tagged [perceptron]

Perceptron is a basic linear classifier that outputs binary labels.

Filter by
Sorted by
Tagged with
16 votes
2 answers
16k views

Strange behavior with Adam optimizer when training for too long

I'm trying to train a single perceptron (1000 input units, 1 output, no hidden layers) on 64 randomly generated data points. I'm using Pytorch using the Adam optimizer: ...
Bai Li's user avatar
  • 263
10 votes
4 answers
14k views

Why must a CNN have a fixed input size?

Right now I'm studying Convolutional Neural Networks. Why must a CNN have a fixed input size? I know that it is possible to overcome this problem (with fully convolutional neural networks etc...), and ...
Mattia Surricchio's user avatar
10 votes
3 answers
16k views

What is the difference between Perceptron and ADALINE?

What is the difference between Perceptron and ADALINE?
Developer's user avatar
  • 1,099
9 votes
5 answers
16k views

Perceptron learning rate

Today I've seen many Perceptron implementations with learning rates. According to Wikipedia: there is no need for a learning rate in the perceptron algorithm. This is because multiplying the ...
bkoodaa's user avatar
  • 323
7 votes
1 answer
6k views

difference between empirical risk minimization and structural risk minimization?

I understand the meaning of empirical risk minimization as separate topic and was reading about structural risk minimization, it is hard for me to understand the difference between these two. I read ...
A.B's user avatar
  • 336
5 votes
4 answers
276 views

Is the prediction algorithm absolutely the same for all linear classifiers?

Is the prediction algorithm absolutely the same for all linear classifiers and linear regression algorithms? As known, any linear classifier can be described as: ...
Alex's user avatar
  • 649
5 votes
1 answer
401 views

Whether add bias or not in a perceptron

In some places, perceptron is described as having added bias, while in some places, bias is not added. Which one is right for you?
Developer's user avatar
  • 1,099
5 votes
1 answer
2k views

Are single input single output neural networks possible?

This might be a weird question but I'm trying to have a deep understanding of how neural networks work theoretically. I was doing some tests with my perceptron and I decided to test it on a single ...
MaTHStudent's user avatar
4 votes
1 answer
521 views

What's wrong with my implementation of the Adaline algorithm?

I'm working through the textbook called Learning From Data and one of the problems from the first chapter has the reader implement the Adaline algorithm from scratch and I chose to do so using Python. ...
lamyvista's user avatar
4 votes
1 answer
434 views

Ambiguity in Perceptron loss function (C. Bishop vs F. Rosenblatt)

Bishop's Perceptron loss On one hand, it is stated in equation 4.54 of Chris Bishop's book (pattern recognition and machine learning) that the loss function of perceptron algorithm is given by: $${...
pythinker's user avatar
  • 1,267
4 votes
1 answer
1k views

Perceptron weight vector update

I read about the Rosenblatt Perceptron Learning Algorithm. Often there is an explicit note: It is important to note that all weights in the weight vector are being updated simultaneously But why ...
wake-0's user avatar
  • 143
4 votes
1 answer
3k views

How to implement gradient descent for a tanh() activation function for a single layer perceptron?

I am required to implement a simple perceptron based neural network for an image classification task, with a binary output and a single layer, however I am having difficulties. I have a few problems: ...
Waylander's user avatar
4 votes
1 answer
910 views

How to deal with a sparse matrix when using a perceptron based recommender system?

I'm constrained to use a perceptron based method. I have a user-item matrix filled with rating data on scale of 1 to 5 like this, with around 50% of the matrix with no data: ...
UD1989's user avatar
  • 258
4 votes
2 answers
1k views

How do we define a linearly separable problem?

When we talk about Perceptrons, we say that they are limited for approximating functions that are linearly separable, while Neural Networks that use non-linear transformations are not. I am having ...
Stefan Radonjic's user avatar
3 votes
1 answer
13k views

Is the percepetron algorithm's convergence dependent on the linearity of the data?

Does the fact that I have linearly separable data or not impact the convergence of the perceptron algorithm? Is it always gonna converge if the data is linearly separable and not if it is not ? Is ...
astudentofmaths's user avatar
3 votes
1 answer
450 views

Understanding computations of Perceptron and Multi-Layer Perceptrons on Geometric level

I am currently watching amazing Deep Learning lecture series from Carnegie Melllon University, but I am having little bit of trouble understanding how Perceptrons and MLP are making their decisions on ...
Stefan Radonjic's user avatar
3 votes
2 answers
661 views

Normalizing the final weights vector in the upper bound on the Perceptron's convergence

The convergence of the "simple" perceptron says that: $$k\leqslant \left ( \frac{R\left \| \bar{\theta} \right \|}{\gamma } \right )^{2}$$ where $k$ is the number of iterations (in which the weights ...
Qwerto's user avatar
  • 705
3 votes
2 answers
519 views

Is there a relationship between LDA, linear SVMs and Perceptron?

LDA (linear discriminant analysis), SVMs with a linear kernel, and perceptrons are linear classifiers. Is there any other relationship between them, e.g.: Every decision boundary that can be found by ...
Martin Thoma's user avatar
3 votes
1 answer
108 views

Coding MLP: good practices?

I recently finished coding my own MLP neural network in Python. To make my code easier to read, I separated the MLP, into classes; the network class, the layers class and the neuron class, where the ...
Liam F-A's user avatar
3 votes
1 answer
3k views

Learning rate in the Perceptron Proof and Convergence

Every perceptron convergence proof i've looked at implicitly uses a learning rate = 1. However, the book I'm using ("Machine learning with Python") suggests to use a small learning rate for ...
Qwerto's user avatar
  • 705
3 votes
1 answer
76 views

What does n means in neural network neuron output?

I've found this equation that explains the output of a neuron in a MLP network: $y(n) = f(\mathbf{w}^T \mathbf{x}(n) + b)$ I can understand the general context, but since i have no background with ...
heresthebuzz's user avatar
3 votes
2 answers
368 views

Geometric interpretation of MLP output

I am really interested in the geometric interpretation of perceptron outputs, mainly as a way to better understand what the network is really doing, but I can't seem to find much information on this ...
Juan González's user avatar
3 votes
2 answers
340 views

Perceptron algorithm

I am trying to build a simple linear classifier. I have two classes A and B each with two features [x, y] and hence a 2d dataset. Now, I need to find the equation of the line that separates the two ...
ukh's user avatar
  • 131
3 votes
2 answers
1k views

Why perceptron does not converge on data not linearly separable

This is how I understand the perceptron algorithm. The perceptron loss function is the hinge loss $\ell(w,x,y) = \max(0, -yw\cdot x)$. Suppose the data set is $D = \{(x_1,y_1),\dots,(x_n,y_n)\}$ with ...
Myath's user avatar
  • 133
3 votes
1 answer
1k views

Perceptron - Convergence proof

I studied the perceptron algorithm and I'm trying to prove the convergence by myself. However, I'm wrong somewhere and I am not able to find the error. Assumption: We assume that there is some $\...
Tantaros's user avatar
  • 161
3 votes
2 answers
2k views

Perceptron - Which step function to choose

I'm studying Perceptron algorithm. Some books use this step function 1 if x>=0 else -1 where x is a dot product between the weights w and a sample x. Other ...
Poiera's user avatar
  • 451
2 votes
1 answer
269 views

Conceptual questions on MLP and Perceptrons

I am facing some confusion regarding the terminologies assocaiated to classification and regression problems esp. using the MLP and Perceptron models. These are the following: 1) When the data is ...
Sm1's user avatar
  • 541
2 votes
2 answers
1k views

How should the hyper parameters be defined for the other algorithms defined for sklearn_crfsuite.CRF

In the example given for sklearn_crfsuite, the parameter space that needs to be passed to a cross_validating class like RandomizedSearchCV is defined as below. ...
joydeep bhattacharjee's user avatar
2 votes
1 answer
998 views

How do I include the Bias term in the Pegasos algorithm?

I have been asked to implement the Pegasos algorithm as below. It is similar to the Peceptron algorithm but includes eta and lambda terms. However, there is no bias term below and I don't know how ...
Matthew Martin's user avatar
2 votes
1 answer
457 views

Threshold logic unit (McCulloch-Pitts 1943) vs. Perceptron (Rosenblatt 1958)

I have searched various sources to find out what distinguishes the McCulloch-Pitts neuron from the perceptron invented by Rosenblatt. In most sources only one of these elements is considered, in ...
Pablo's user avatar
  • 21
2 votes
1 answer
229 views

SciKit Learn: Multilayer perceptron early stopping, restore best weights

In the SciKit documentation of the MLP classifier, there is the early_stopping flag which allows to stop the learning if there is not any improvement in several ...
volperossa's user avatar
2 votes
3 answers
231 views

About different structures of neural network

https://www.mathworks.com/help/deeplearning/ref/fitnet.html is the tutorial that I am following to understand fitting data to a function. I have few doubts regarding structure and terminologies which ...
Sm1's user avatar
  • 541
2 votes
1 answer
401 views

Interpreting MLP output

I just wrote an MLP in Python. After having trained it, I pass in some test data to see the result, and I get an array of decimal numbers at the output, rather than the desired binary output. For ...
Liam F-A's user avatar
2 votes
1 answer
634 views

Perceptron Primitive Boolean Functions

Thanks for reading. I'm currently reading Tom Mitchell's Machine Learning (I'm a beginner into ML), and I'm on chapter 4 about perceptrons. I'm really confused about this paragraph: I understand the ...
joshuaronis's user avatar
2 votes
0 answers
307 views

How does permutation of training data improve convergence time when training a perceptron or neural network model? [duplicate]

I'm currently studying some basic concepts regarding Deep Learning and Neural Networks with this material. When discussing the training algorithm for a perceptron, the author states that looping ...
Nilton Junior's user avatar
2 votes
1 answer
70 views

Version of Perceptron

If we change the $ywx<0$ condition (for performing update) to $ywx<1$ like in SVM (but without adding regularization to maximize the margin), is there any difference from the basic perceptron (...
Ben's user avatar
  • 51
1 vote
1 answer
535 views

How does combining two linear perceptrons create non-linear boundaries?

I don't understand the equation that you get from combining the two linear perceptrons is non-linear? The video starts with two linear perceptrons with the equations: $$e1 = 5x_1 -2x_2 - 8 = 0 \...
rtindru's user avatar
  • 111
1 vote
1 answer
52 views

Multilayer Perceptron: What is the value used to update the weights in the hidden layers?

As i understand for the output layer the error rate is used with the mean squared error function to update the weights. For the hidden layers as well? Does that make sense?
Luke's user avatar
  • 113
1 vote
1 answer
3k views

How to find bias for perceptron algorithm?

My question is very basic. I am starting with ML and am working on the perceptron algorithm. I successfully computed the weights for this input data: ...
Jan Pisl's user avatar
  • 195
1 vote
2 answers
1k views

Learning rate of perception

I don't understand the following statement: The choice of learning rate m does not matter (for Perceptron) because it just changes the scaling of w (weights). The site with this statement that ...
user2573741's user avatar
1 vote
1 answer
135 views

Visualizing a Perceptron

I wanted to visualize how a perceptron learns, so I made a class that performs gradient descent. To show the decision, I plot a plane showing where positive examples and negative examples are, ...
K. Shores's user avatar
  • 113
1 vote
1 answer
34 views

What issue is there, when training this network with gradient descent? [closed]

Suppose we have the following fully connected network made of perceptrons with a sign function as the activation unit, what issue arises, when trying to train this network with gradient descent?
WhatAMesh's user avatar
  • 131
1 vote
3 answers
69 views

Why there is only one type of artificial neuron?

I find it strange that so many deep learning tricks and improvements have been invented in the past decade but I never heard about someone trying out different models of the artificial neuron other ...
Jan Pisl's user avatar
  • 195
1 vote
2 answers
37 views

Should a bias neuron be connected to previous neurons?

I'm trying to add a bias neuron to my neural network that uses the backpropagation algorithm. I'm trying to figure out how I should go about this, should I treat the bias neuron as a regular neuron? ...
davegri's user avatar
  • 131
1 vote
1 answer
445 views

Confusion on Delta Rule and Error

I'm currently reading Mitchell's book for Machine Learning, and he just started gradient descent. There's one part that's really confusing me. At one point, he gives this equation for the error of a ...
joshuaronis's user avatar
1 vote
1 answer
6k views

sklearn.GridSearchCV predict method not providing the best estimate and accuracy score

I was playing around with the credit default dataset in UCI ("https://archive.ics.uci.edu/ml/machine-learning-databases/00350/default%20of%20credit%20card%20clients.xls") These are the steps i have ...
user1906450's user avatar
1 vote
1 answer
680 views

Trouble with accuracy of multiclass perceptron

I have built a multiclass perceptron, but it has low accuracy (around 80%). I think I'm missing something. One possibility is that I should add a bias, but I'm not sure how to incorporate that. The ...
Adam_G's user avatar
  • 91
1 vote
1 answer
1k views

Perceptron learning rate irrelevant in convergence [duplicate]

Via this MIT document i studied the single layer Perceptron convergence proof (= maximum number of steps). In the convergence proof inside this document , the learning rate is implicitly defined as 1 ...
Qwerto's user avatar
  • 705
1 vote
1 answer
236 views

What is the hypothesis space used by this AND gate Perceptron?

Per this post The hypothesis space used by a machine learning system is the set of all hypotheses that might possibly be returned by it. Per this post, the Perceptron algorithm makes prediction ...
czlsws's user avatar
  • 157
1 vote
1 answer
169 views

neural networks error function: is global minimum desirable?

In "Elements of statistical learning" page 395 the authors state that, relative to R(θ), the regression/classification error function in a neural network such as a multi layer perceptron: Typically ...
amiando's user avatar
  • 35