Questions tagged [perceptron]

Perceptron is a basic linear classifier that outputs binary labels.

Filter by
Sorted by
Tagged with
0 votes
1 answer
1k views

How is calculated the error with multiple output neurons in neural network?

Machine Learning books generally explains that the error calculated for a given sample $i$ is: $e_i = y_i - \hat{y_i}$ Where $\hat{y}$ is the target output and $y$ is the actual output given by the ...
0 votes
1 answer
69 views

Is it possible to train a single input->neuron->relu->neuron->relu for input > 0.5?

The neural network is simply: y=max(max(x*w+b,0)*v+d,0) w,b is weight and bias of first neuron. v,d is weight and bias of second neuron. If data is for example: <...
3 votes
2 answers
2k views

Perceptron - Which step function to choose

I'm studying Perceptron algorithm. Some books use this step function 1 if x>=0 else -1 where x is a dot product between the weights w and a sample x. Other ...
1 vote
1 answer
84 views

Error while writing perceptron algorithm binary classifier

I am a beginner and I am designing an binary classifier using Perceptron algorithm using FASHION-MNIST dataset. While designing the same I have written the following code: ...
2 votes
1 answer
66 views

Version of Perceptron

If we change the $ywx<0$ condition (for performing update) to $ywx<1$ like in SVM (but without adding regularization to maximize the margin), is there any difference from the basic perceptron (...
0 votes
0 answers
7 views

Generalization error of a simple perceptron for a student-teacher network

I have been asked to prove the following expression given the following density probability function for a student-teacher $ P(x,y) = \frac{1}{2\pi\sqrt{Q-R^2}} \cdot \exp\left(-\frac{1}{2}\left(\frac{...
0 votes
0 answers
25 views

what happens when weghts in perceptron algorithm is first initialized with some random values which is very distant from the correct values?

take this example of a small dataset so here there was a question that instead of initializing weight vector as zeroes what if we initialize to [1000 , -1000] (there is no offset i.e classifiers ...
2 votes
1 answer
407 views

Threshold logic unit (McCulloch-Pitts 1943) vs. Perceptron (Rosenblatt 1958)

I have searched various sources to find out what distinguishes the McCulloch-Pitts neuron from the perceptron invented by Rosenblatt. In most sources only one of these elements is considered, in ...
6 votes
1 answer
6k views

difference between empirical risk minimization and structural risk minimization?

I understand the meaning of empirical risk minimization as separate topic and was reading about structural risk minimization, it is hard for me to understand the difference between these two. I read ...
0 votes
0 answers
25 views

Calculating weight and bias of linear perceptron on convergence given number mistakes for each sample

A linear perceptron has been trained with a set of n points (∈ ℝ²) and their corresponding labels ...
0 votes
0 answers
66 views

Classical Perceptron Algorithm taking too long to evaluate on un-normalized data

I need to implement classical perceptron algorithm from scratch using numpy and pandas for an assignments. I have done so using this algorithm: I have a linearly seperable dataset of 568 rows and 30 ...
0 votes
0 answers
50 views

Understanding perceptron learning algorithm

I was revisiting perceptron learning algorithm. The wikipedia page gives the algorithm as follows: Initialize the weights to 0 or a small random value. For each example $j$ in our training set $D$, ...
0 votes
0 answers
17 views

OR gate perceptron in plain python - Loss won't converge

I am coding a perceptron from scratch just out of curiosity in plain python for OR gate, but a loss won't converge. ...
0 votes
0 answers
18 views

Is bias nothing but perceptron threhold value?

I was revisiting neural network basics from this post. The perceptron follows below equation: $$\begin{align} y & = 1 & \text{if } \sum_{i=1}^n w_i\times x_i \geq \theta \\ & = 0 & \...
2 votes
1 answer
843 views

How do I include the Bias term in the Pegasos algorithm?

I have been asked to implement the Pegasos algorithm as below. It is similar to the Peceptron algorithm but includes eta and lambda terms. However, there is no bias term below and I don't know how ...
0 votes
1 answer
381 views

Can a multilayer perceptron classify binary values?

I have a dataset in which the response variable is Sick(1) or not sick (2). As for the variables, there are a few numeric ones (2/14), all the others are variables by levels (example: 1-Abdominal pain,...
1 vote
0 answers
1k views

Optimizing an averaged perceptron algorithm using numpy and scipy instead of dictionaries

So I'm trying to write an averaged perceptron algorithm (page 48 here for the equation) in python. Instead of storing the historical weights, I simply accumulate the weights and then multiply ...
3 votes
2 answers
641 views

Normalizing the final weights vector in the upper bound on the Perceptron's convergence

The convergence of the "simple" perceptron says that: $$k\leqslant \left ( \frac{R\left \| \bar{\theta} \right \|}{\gamma } \right )^{2}$$ where $k$ is the number of iterations (in which the weights ...
10 votes
3 answers
16k views

What is the difference between Perceptron and ADALINE?

What is the difference between Perceptron and ADALINE?
0 votes
1 answer
59 views

Why we use an activation function for introducing nonlinearity instead of a polynomial Perceptron implementation?

I perceive a single perceptron as a single linear function $y = a_1x_1 + a_2x_2 + ... + a_nx_n + b_0$ with a goal to calculate the best weights combination $ w_1, w_2, ..., w_n $ that minimizes the ...
2 votes
3 answers
206 views

About different structures of neural network

https://www.mathworks.com/help/deeplearning/ref/fitnet.html is the tutorial that I am following to understand fitting data to a function. I have few doubts regarding structure and terminologies which ...
1 vote
1 answer
84 views

Visualizing a Perceptron

I wanted to visualize how a perceptron learns, so I made a class that performs gradient descent. To show the decision, I plot a plane showing where positive examples and negative examples are, ...
1 vote
1 answer
34 views

What issue is there, when training this network with gradient descent? [closed]

Suppose we have the following fully connected network made of perceptrons with a sign function as the activation unit, what issue arises, when trying to train this network with gradient descent?
1 vote
0 answers
24 views

why use one regularisation technique over another?

why should I prefer L1 over L2, in fully-connected-layer or convolution? why use dropout between 2 layers, when there is the option of regularising a layer(or both) with something like L1 or L2? and ...
1 vote
3 answers
61 views

Why there is only one type of artificial neuron?

I find it strange that so many deep learning tricks and improvements have been invented in the past decade but I never heard about someone trying out different models of the artificial neuron other ...
2 votes
1 answer
208 views

SciKit Learn: Multilayer perceptron early stopping, restore best weights

In the SciKit documentation of the MLP classifier, there is the early_stopping flag which allows to stop the learning if there is not any improvement in several ...
1 vote
0 answers
24 views

predict multiple letters in pixels matrix

I have a multilayer perceptron model that is trained to recognize handwritten English letters from an image. In the training set each image matrix had 784 pixel values. The labels of these images ...
5 votes
1 answer
384 views

Whether add bias or not in a perceptron

In some places, perceptron is described as having added bias, while in some places, bias is not added. Which one is right for you?
1 vote
2 answers
35 views

Should a bias neuron be connected to previous neurons?

I'm trying to add a bias neuron to my neural network that uses the backpropagation algorithm. I'm trying to figure out how I should go about this, should I treat the bias neuron as a regular neuron? ...
1 vote
0 answers
25 views

I'm worried that I'm training my model wrong

So I'm trying to classify some fashion mnist like photos into either boots or sneakers. I'm using a perception from sklearn to do so. The data set is a CSV containing pixel values. The model is ...
1 vote
1 answer
392 views

Calculation of VC dimension of simple neural network

Suppose I have a perceptron with one-hidden layer, with the input - one real number $x \in \mathbb{R}$, and the activation function of the output layers - threshold functions: $$ \theta(x) = \begin{...
1 vote
1 answer
401 views

Performing a linear regression with Perceptron

I was wondering about the link between the linear regression and the perceptron! Perceptrons were used as binary classifiers i.e to classify binary labels ( 0 or 1 ). My question is How can you ...
3 votes
1 answer
72 views

What does n means in neural network neuron output?

I've found this equation that explains the output of a neuron in a MLP network: $y(n) = f(\mathbf{w}^T \mathbf{x}(n) + b)$ I can understand the general context, but since i have no background with ...
0 votes
1 answer
2k views

Equation of a Multi-Layer Perceptron Network

I'm writing an article about business management of wine companies where I use a Multi-Layer Perceptron Network. My teacher then asked me to write an equation that lets me calculate the output of the ...
4 votes
1 answer
458 views

What's wrong with my implementation of the Adaline algorithm?

I'm working through the textbook called Learning From Data and one of the problems from the first chapter has the reader implement the Adaline algorithm from scratch and I chose to do so using Python. ...
3 votes
1 answer
378 views

Understanding computations of Perceptron and Multi-Layer Perceptrons on Geometric level

I am currently watching amazing Deep Learning lecture series from Carnegie Melllon University, but I am having little bit of trouble understanding how Perceptrons and MLP are making their decisions on ...
9 votes
4 answers
13k views

Why must a CNN have a fixed input size?

Right now I'm studying Convolutional Neural Networks. Why must a CNN have a fixed input size? I know that it is possible to overcome this problem (with fully convolutional neural networks etc...), and ...
3 votes
2 answers
318 views

Geometric interpretation of MLP output

I am really interested in the geometric interpretation of perceptron outputs, mainly as a way to better understand what the network is really doing, but I can't seem to find much information on this ...
4 votes
2 answers
1k views

How do we define a linearly separable problem?

When we talk about Perceptrons, we say that they are limited for approximating functions that are linearly separable, while Neural Networks that use non-linear transformations are not. I am having ...
3 votes
2 answers
1k views

Why perceptron does not converge on data not linearly separable

This is how I understand the perceptron algorithm. The perceptron loss function is the hinge loss $\ell(w,x,y) = \max(0, -yw\cdot x)$. Suppose the data set is $D = \{(x_1,y_1),\dots,(x_n,y_n)\}$ with ...
5 votes
1 answer
2k views

Are single input single output neural networks possible?

This might be a weird question but I'm trying to have a deep understanding of how neural networks work theoretically. I was doing some tests with my perceptron and I decided to test it on a single ...
1 vote
1 answer
3k views

How to find bias for perceptron algorithm?

My question is very basic. I am starting with ML and am working on the perceptron algorithm. I successfully computed the weights for this input data: ...
2 votes
1 answer
365 views

Interpreting MLP output

I just wrote an MLP in Python. After having trained it, I pass in some test data to see the result, and I get an array of decimal numbers at the output, rather than the desired binary output. For ...
1 vote
0 answers
835 views

linear perceptron algorithm

Linear Classification Consider a labeled training set shown in figure below: ...
1 vote
1 answer
175 views

Effect of adding extra unrelated features to linear perceptron

Suppose that we are training a linear regressor (perceptron). Adding extra features that are not related to the target (e.g. randomly generated values) before training will typically ____ our training ...
9 votes
5 answers
16k views

Perceptron learning rate

Today I've seen many Perceptron implementations with learning rates. According to Wikipedia: there is no need for a learning rate in the perceptron algorithm. This is because multiplying the ...
1 vote
0 answers
108 views

Convergence speed of perceptron algorithm

I was reading the convergence proof for the perceptron algorithm. It says under the assumption that there are some $R$, $\theta^*$ with $|\theta^*| = 1$ and $\gamma > 0$, such that $y_t(x_t\cdot \...
1 vote
1 answer
50 views

Multilayer Perceptron: What is the value used to update the weights in the hidden layers?

As i understand for the output layer the error rate is used with the mean squared error function to update the weights. For the hidden layers as well? Does that make sense?
2 votes
1 answer
256 views

Conceptual questions on MLP and Perceptrons

I am facing some confusion regarding the terminologies assocaiated to classification and regression problems esp. using the MLP and Perceptron models. These are the following: 1) When the data is ...
3 votes
2 answers
323 views

Perceptron algorithm

I am trying to build a simple linear classifier. I have two classes A and B each with two features [x, y] and hence a 2d dataset. Now, I need to find the equation of the line that separates the two ...