Questions tagged [perceptron]

Perceptron is a basic linear classifier that outputs binary labels.

Filter by
Sorted by
Tagged with
0 votes
0 answers
7 views

Selection of learning rate parameter for perceptron

I have a project where I should implement perceptron in Matlab. I can select any vector Wo that does not classify the 2 classes correctly. But when it comes to the learning rate parameter, I have to ...
user avatar
0 votes
0 answers
25 views

Online Learning Perceptron Mistake Bound

Consider the modification of Perceptron algorithm with the following update rule: $$ w_t+1 ← w_t + η_ty_tx_t $$ whenever $\hat{y_t } \neq y_t$ ($w_t+1 ← w_t$ otherwise).for $η_t = 1 /\sqrt{t}$ i need ...
user avatar
  • 1
0 votes
1 answer
18 views

Can a multilayer perceptron classify binary values?

I have a dataset in which the response variable is Sick(1) or not sick (2). As for the variables, there are a few numeric ones (2/14), all the others are variables by levels (example: 1-Abdominal pain,...
user avatar
  • 302
0 votes
0 answers
7 views

Novice machine learner wondering how to interpret big variance in batch error across batches in MNIST perceptron

I'm trying to get a better understanding of basic neural networks by implementing a little framework in C++. I've started with the classical MNIST exercise. I get to 91% accuracy on the test sample ...
user avatar
  • 121
0 votes
0 answers
107 views

Can we have neural network emulate XOR logic gate with single neuron in the hidden layer?

I came across following neural networks emulating logical XOR gate: Approach 1: Approach 2: But today, I came across below one: I dont get how this behaves as XOR, especially what does those ...
user avatar
  • 175
0 votes
0 answers
26 views

What is correct equation for LR decision boundary?

I read that the equation perceptron decision boundary is given as follows:$$w^Tx-w_0=0$$ This can be proven as follows: Assuming $w$ is a unit vector (as we can multiply above equation with a ...
user avatar
  • 175
0 votes
0 answers
15 views

"Saliency map" of perceptron?

I am using keras currently, and I want to see which inputs the model is "looking at". It would be like a saliency map, but my model is a simple two-layered perceptron for classification, so ...
user avatar
0 votes
0 answers
21 views

Proof of Correctness of Perceptron Training Rule

The Perceptron Training Rule is basically applying Stochastic Gradient Descent for finding the coefficients of a hyperplane (which works as a Decision Boundary) for doing binary classification of data ...
user avatar
0 votes
1 answer
38 views

Why we use an activation function for introducing nonlinearity instead of a polynomial Perceptron implementation?

I perceive a single perceptron as a single linear function $y = a_1x_1 + a_2x_2 + ... + a_nx_n + b_0$ with a goal to calculate the best weights combination $ w_1, w_2, ..., w_n $ that minimizes the ...
user avatar
  • 3
1 vote
1 answer
40 views

Visualizing a Perceptron

I wanted to visualize how a perceptron learns, so I made a class that performs gradient descent. To show the decision, I plot a plane showing where positive examples and negative examples are, ...
user avatar
  • 113
1 vote
1 answer
31 views

What issue is there, when training this network with gradient descent? [closed]

Suppose we have the following fully connected network made of perceptrons with a sign function as the activation unit, what issue arises, when trying to train this network with gradient descent?
user avatar
  • 131
1 vote
0 answers
21 views

why use one regularisation technique over another?

why should I prefer L1 over L2, in fully-connected-layer or convolution? why use dropout between 2 layers, when there is the option of regularising a layer(or both) with something like L1 or L2? and ...
user avatar
1 vote
1 answer
286 views

How do I include the Bias term in the Pegasos algorithm?

I have been asked to implement the Pegasos algorithm as below. It is similar to the Peceptron algorithm but includes eta and lambda terms. However, there is no bias term below and I don't know how ...
user avatar
1 vote
3 answers
52 views

Why there is only one type of artificial neuron?

I find it strange that so many deep learning tricks and improvements have been invented in the past decade but I never heard about someone trying out different models of the artificial neuron other ...
user avatar
  • 175
1 vote
0 answers
24 views

predict multiple letters in pixels matrix

I have a multilayer perceptron model that is trained to recognize handwritten English letters from an image. In the training set each image matrix had 784 pixel values. The labels of these images ...
user avatar
1 vote
2 answers
28 views

Should a bias neuron be connected to previous neurons?

I'm trying to add a bias neuron to my neural network that uses the backpropagation algorithm. I'm trying to figure out how I should go about this, should I treat the bias neuron as a regular neuron? ...
user avatar
  • 131
1 vote
0 answers
24 views

I'm worried that I'm training my model wrong

So I'm trying to classify some fashion mnist like photos into either boots or sneakers. I'm using a perception from sklearn to do so. The data set is a CSV containing pixel values. The model is ...
user avatar
2 votes
1 answer
189 views

Threshold logic unit (McCulloch-Pitts 1943) vs. Perceptron (Rosenblatt 1958)

I have searched various sources to find out what distinguishes the McCulloch-Pitts neuron from the perceptron invented by Rosenblatt. In most sources only one of these elements is considered, in ...
user avatar
  • 21
1 vote
1 answer
171 views

Calculation of VC dimension of simple neural network

Suppose I have a perceptron with one-hidden layer, with the input - one real number $x \in \mathbb{R}$, and the activation function of the output layers - threshold functions: $$ \theta(x) = \begin{...
user avatar
1 vote
1 answer
244 views

Performing a linear regression with Perceptron

I was wondering about the link between the linear regression and the perceptron! Perceptrons were used as binary classifiers i.e to classify binary labels ( 0 or 1 ). My question is How can you ...
user avatar
0 votes
1 answer
753 views

How is calculated the error with multiple output neurons in neural network?

Machine Learning books generally explains that the error calculated for a given sample $i$ is: $e_i = y_i - \hat{y_i}$ Where $\hat{y}$ is the target output and $y$ is the actual output given by the ...
user avatar
3 votes
1 answer
66 views

What does n means in neural network neuron output?

I've found this equation that explains the output of a neuron in a MLP network: $y(n) = f(\mathbf{w}^T \mathbf{x}(n) + b)$ I can understand the general context, but since i have no background with ...
user avatar
0 votes
1 answer
1k views

Equation of a Multi-Layer Perceptron Network

I'm writing an article about business management of wine companies where I use a Multi-Layer Perceptron Network. My teacher then asked me to write an equation that lets me calculate the output of the ...
user avatar
4 votes
1 answer
322 views

What's wrong with my implementation of the Adaline algorithm?

I'm working through the textbook called Learning From Data and one of the problems from the first chapter has the reader implement the Adaline algorithm from scratch and I chose to do so using Python. ...
user avatar
2 votes
1 answer
106 views

SciKit Learn: Multilayer perceptron early stopping, restore best weights

In the SciKit documentation of the MLP classifier, there is the early_stopping flag which allows to stop the learning if there is not any improvement in several ...
user avatar
1 vote
0 answers
612 views

linear perceptron algorithm

Linear Classification Consider a labeled training set shown in figure below: ...
user avatar
3 votes
1 answer
220 views

Understanding computations of Perceptron and Multi-Layer Perceptrons on Geometric level

I am currently watching amazing Deep Learning lecture series from Carnegie Melllon University, but I am having little bit of trouble understanding how Perceptrons and MLP are making their decisions on ...
user avatar
4 votes
2 answers
1k views

How do we define a linearly separable problem?

When we talk about Perceptrons, we say that they are limited for approximating functions that are linearly separable, while Neural Networks that use non-linear transformations are not. I am having ...
user avatar
5 votes
1 answer
4k views

difference between empirical risk minimization and structural risk minimization?

I understand the meaning of empirical risk minimization as separate topic and was reading about structural risk minimization, it is hard for me to understand the difference between these two. I read ...
user avatar
  • 286
8 votes
4 answers
9k views

Why must a CNN have a fixed input size?

Right now I'm studying Convolutional Neural Networks. Why must a CNN have a fixed input size? I know that it is possible to overcome this problem (with fully convolutional neural networks etc...), and ...
user avatar
1 vote
0 answers
92 views

Convergence speed of perceptron algorithm

I was reading the convergence proof for the perceptron algorithm. It says under the assumption that there are some $R$, $\theta^*$ with $|\theta^*| = 1$ and $\gamma > 0$, such that $y_t(x_t\cdot \...
user avatar
1 vote
1 answer
37 views

Multilayer Perceptron: What is the value used to update the weights in the hidden layers?

As i understand for the output layer the error rate is used with the mean squared error function to update the weights. For the hidden layers as well? Does that make sense?
user avatar
  • 113
2 votes
3 answers
146 views

About different structures of neural network

https://www.mathworks.com/help/deeplearning/ref/fitnet.html is the tutorial that I am following to understand fitting data to a function. I have few doubts regarding structure and terminologies which ...
user avatar
  • 521
2 votes
1 answer
205 views

Conceptual questions on MLP and Perceptrons

I am facing some confusion regarding the terminologies assocaiated to classification and regression problems esp. using the MLP and Perceptron models. These are the following: 1) When the data is ...
user avatar
  • 521
0 votes
1 answer
2k views

How to find bias for perceptron algorithm?

My question is very basic. I am starting with ML and am working on the perceptron algorithm. I successfully computed the weights for this input data: ...
user avatar
  • 175
1 vote
1 answer
67 views

Error while writing perceptron algorithm binary classifier

I am a beginner and I am designing an binary classifier using Perceptron algorithm using FASHION-MNIST dataset. While designing the same I have written the following code: ...
user avatar
1 vote
0 answers
48 views

Simple perceptron on python

I want to create a simple perceptron with three inputs. $ \sigma(w_1x_1 + w_2x_2 + w_3x_3) $, where $ \sigma = \frac{1}{1 + e^{-x}.} $ $$ L(w) = \sum_i^3(y_i -\sigma(w_1x_1 + w_2x_2 + w_3x_3)); $$ $ \...
user avatar
  • 47
0 votes
0 answers
66 views

Similarity of perceptron criterion and SVM

In the book "Neural Networks and Deep Learning" by Aggarwal there is an exercise 2.10.1: Consider the following loss function for training pair $(\overline{X},y)$: $$L=max(0, a -y(\overline{W} \...
user avatar
  • 9
0 votes
0 answers
30 views

What are contexts where a perceptron model could be defined?

per this post, a perceptron could use the logit function as the activation function. per wiki In the context of neural networks, a perceptron is an artificial neuron using the Heaviside step ...
user avatar
  • 157
1 vote
1 answer
190 views

What is the hypothesis space used by this AND gate Perceptron?

Per this post The hypothesis space used by a machine learning system is the set of all hypotheses that might possibly be returned by it. Per this post, the Perceptron algorithm makes prediction ...
user avatar
  • 157
2 votes
0 answers
303 views

How does permutation of training data improve convergence time when training a perceptron or neural network model? [duplicate]

I'm currently studying some basic concepts regarding Deep Learning and Neural Networks with this material. When discussing the training algorithm for a perceptron, the author states that looping ...
user avatar
2 votes
1 answer
51 views

Version of Perceptron

If we change the $ywx<0$ condition (for performing update) to $ywx<1$ like in SVM (but without adding regularization to maximize the margin), is there any difference from the basic perceptron (...
user avatar
  • 51
3 votes
2 answers
233 views

Geometric interpretation of MLP output

I am really interested in the geometric interpretation of perceptron outputs, mainly as a way to better understand what the network is really doing, but I can't seem to find much information on this ...
user avatar
0 votes
1 answer
168 views

Abstraction in Neural Networks

In my nonlinear dynamics class in college, we discussed a simple perceptron with two input neurons and one output neuron that is trained on the patterns ...
user avatar
  • 248
3 votes
2 answers
247 views

Perceptron algorithm

I am trying to build a simple linear classifier. I have two classes A and B each with two features [x, y] and hence a 2d dataset. Now, I need to find the equation of the line that separates the two ...
user avatar
4 votes
1 answer
400 views

Ambiguity in Perceptron loss function (C. Bishop vs F. Rosenblatt)

Bishop's Perceptron loss On one hand, it is stated in equation 4.54 of Chris Bishop's book (pattern recognition and machine learning) that the loss function of perceptron algorithm is given by: $${...
user avatar
  • 1,207
1 vote
0 answers
1k views

(Almost) linearely separable dataset, where can I find one?

I'm implementing the perceptron algorithm and the voted perceptron algorithm for an assignment for university. For that I need to find some decent datasets.. I've tried the UCI repos and I've come up ...
user avatar
1 vote
1 answer
105 views

neural networks error function: is global minimum desirable?

In "Elements of statistical learning" page 395 the authors state that, relative to R(θ), the regression/classification error function in a neural network such as a multi layer perceptron: Typically ...
user avatar
  • 23
1 vote
3 answers
544 views

How do two perceptrons produce different linear decision boundaries?

I'm trying to visualize how two perceptrons converge to two different decision boundaries (which is ultimately used to create the classifier for the non-linearly separable data). Source: https://tdb-...
user avatar
  • 13
2 votes
1 answer
236 views

Interpreting MLP output

I just wrote an MLP in Python. After having trained it, I pass in some test data to see the result, and I get an array of decimal numbers at the output, rather than the desired binary output. For ...
user avatar