Questions tagged [perceptron]
Perceptron is a basic linear classifier that outputs binary labels.
88
questions
0
votes
0
answers
6
views
Generalization error of a simple perceptron for a student-teacher network
I have been asked to prove the following expression given the following density probability function for a student-teacher
$ P(x,y) = \frac{1}{2\pi\sqrt{Q-R^2}} \cdot \exp\left(-\frac{1}{2}\left(\frac{...
- 1
0
votes
0
answers
20
views
what happens when weghts in perceptron algorithm is first initialized with some random values which is very distant from the correct values?
take this example of a small dataset
so here there was a question that instead of initializing weight vector as zeroes what if we initialize to [1000 , -1000] (there is no offset i.e classifiers ...
0
votes
1
answer
57
views
Is it possible to train a single input->neuron->relu->neuron->relu for input > 0.5?
The neural network is simply:
y=max(max(x*w+b,0)*v+d,0)
w,b is weight and bias of first neuron.
v,d is weight and bias of second neuron.
If data is for example:
<...
- 1
0
votes
0
answers
18
views
Calculating weight and bias of linear perceptron on convergence given number mistakes for each sample
A linear perceptron has been trained with a set of n points (∈ ℝ²) and their corresponding labels ...
- 131
0
votes
0
answers
39
views
Classical Perceptron Algorithm taking too long to evaluate on un-normalized data
I need to implement classical perceptron algorithm from scratch using numpy and pandas for an assignments.
I have done so using this algorithm:
I have a linearly seperable dataset of 568 rows and 30 ...
- 1
0
votes
0
answers
30
views
Understanding perceptron learning algorithm
I was revisiting perceptron learning algorithm. The wikipedia page gives the algorithm as follows:
Initialize the weights to 0 or a small random value.
For each example $j$ in our training set $D$, ...
- 103
0
votes
0
answers
16
views
OR gate perceptron in plain python - Loss won't converge
I am coding a perceptron from scratch just out of curiosity in plain python for OR gate, but a loss won't converge.
...
- 101
0
votes
0
answers
15
views
Is bias nothing but perceptron threhold value?
I was revisiting neural network basics from this post. The perceptron follows below equation:
$$\begin{align}
y & = 1 & \text{if } \sum_{i=1}^n w_i\times x_i \geq \theta \\
& = 0 & \...
- 103
0
votes
0
answers
57
views
How to balance an imbalanced dataset
I'm using the UNSW-NB15 dataset, which is really imbalanced, to train a Multi-class classificationMLP. I encode the categorical features of the dataset which leads to a 2+ million x 205 columns ...
0
votes
1
answer
311
views
Can a multilayer perceptron classify binary values?
I have a dataset in which the response variable is Sick(1) or not sick (2). As for the variables, there are a few numeric ones (2/14), all the others are variables by levels (example: 1-Abdominal pain,...
- 304
0
votes
1
answer
55
views
Why we use an activation function for introducing nonlinearity instead of a polynomial Perceptron implementation?
I perceive a single perceptron as a single linear function $y = a_1x_1 + a_2x_2 + ... + a_nx_n + b_0$ with a goal to calculate the best weights combination $ w_1, w_2, ..., w_n $ that minimizes the ...
- 3
1
vote
1
answer
68
views
Visualizing a Perceptron
I wanted to visualize how a perceptron learns, so I made a class that performs gradient descent. To show the decision, I plot a plane showing where positive examples and negative examples are, ...
- 113
1
vote
1
answer
34
views
What issue is there, when training this network with gradient descent? [closed]
Suppose we have the following fully connected network made of perceptrons with a sign function as the activation unit, what issue arises, when trying to train this network with gradient descent?
- 131
1
vote
0
answers
24
views
why use one regularisation technique over another?
why should I prefer L1 over L2, in fully-connected-layer or convolution?
why use dropout between 2 layers, when there is the option of regularising a layer(or both) with something like L1 or L2? and ...
2
votes
1
answer
679
views
How do I include the Bias term in the Pegasos algorithm?
I have been asked to implement the Pegasos algorithm as below. It is similar to the Peceptron algorithm but includes eta and lambda terms. However, there is no bias term below and I don't know how ...
1
vote
3
answers
59
views
Why there is only one type of artificial neuron?
I find it strange that so many deep learning tricks and improvements have been invented in the past decade but I never heard about someone trying out different models of the artificial neuron other ...
- 195
1
vote
0
answers
24
views
predict multiple letters in pixels matrix
I have a multilayer perceptron model that is trained to recognize handwritten English letters from an image. In the training set each image matrix had 784 pixel values. The labels of these images ...
- 111
1
vote
2
answers
34
views
Should a bias neuron be connected to previous neurons?
I'm trying to add a bias neuron to my neural network that uses the backpropagation algorithm. I'm trying to figure out how I should go about this, should I treat the bias neuron as a regular neuron? ...
- 131
1
vote
0
answers
25
views
I'm worried that I'm training my model wrong
So I'm trying to classify some fashion mnist like photos into either boots or sneakers. I'm using a perception from sklearn to do so. The data set is a CSV containing pixel values. The model is ...
- 35
2
votes
1
answer
370
views
Threshold logic unit (McCulloch-Pitts 1943) vs. Perceptron (Rosenblatt 1958)
I have searched various sources to find out what distinguishes the McCulloch-Pitts neuron from the perceptron invented by Rosenblatt. In most sources only one of these elements is considered, in ...
- 21
1
vote
1
answer
377
views
Calculation of VC dimension of simple neural network
Suppose I have a perceptron with one-hidden layer, with the input - one real number $x \in \mathbb{R}$, and the activation function of the output layers - threshold functions:
$$
\theta(x) =
\begin{...
1
vote
1
answer
363
views
Performing a linear regression with Perceptron
I was wondering about the link between the linear regression and the perceptron!
Perceptrons were used as binary classifiers i.e to classify binary labels ( 0 or 1 ).
My question is How can you ...
0
votes
1
answer
1k
views
How is calculated the error with multiple output neurons in neural network?
Machine Learning books generally explains that the error calculated for a given sample $i$ is:
$e_i = y_i - \hat{y_i}$
Where $\hat{y}$ is the target output and $y$ is the actual output given by the ...
- 395
3
votes
1
answer
72
views
What does n means in neural network neuron output?
I've found this equation that explains the output of a neuron in a MLP network:
$y(n) = f(\mathbf{w}^T \mathbf{x}(n) + b)$
I can understand the general context, but since i have no background with ...
- 395
0
votes
1
answer
2k
views
Equation of a Multi-Layer Perceptron Network
I'm writing an article about business management of wine companies where I use a Multi-Layer Perceptron Network.
My teacher then asked me to write an equation that lets me calculate the output of the ...
- 111
4
votes
1
answer
426
views
What's wrong with my implementation of the Adaline algorithm?
I'm working through the textbook called Learning From Data and one of the problems from the first chapter has the reader implement the Adaline algorithm from scratch and I chose to do so using Python. ...
- 51
2
votes
1
answer
193
views
SciKit Learn: Multilayer perceptron early stopping, restore best weights
In the SciKit documentation of the MLP classifier, there is the early_stopping flag which allows to stop the learning if there is not any improvement in several ...
- 121
1
vote
0
answers
809
views
linear perceptron algorithm
Linear Classification
Consider a labeled training set shown in figure below:
...
- 71
3
votes
1
answer
353
views
Understanding computations of Perceptron and Multi-Layer Perceptrons on Geometric level
I am currently watching amazing Deep Learning lecture series from Carnegie Melllon University, but I am having little bit of trouble understanding how Perceptrons and MLP are making their decisions on ...
- 716
4
votes
2
answers
1k
views
How do we define a linearly separable problem?
When we talk about Perceptrons, we say that they are limited for approximating functions that are linearly separable, while Neural Networks that use non-linear transformations are not.
I am having ...
- 716
6
votes
1
answer
5k
views
difference between empirical risk minimization and structural risk minimization?
I understand the meaning of empirical risk minimization as separate topic and was reading about structural risk minimization, it is hard for me to understand the difference between these two.
I read ...
- 316
9
votes
4
answers
12k
views
Why must a CNN have a fixed input size?
Right now I'm studying Convolutional Neural Networks.
Why must a CNN have a fixed input size?
I know that it is possible to overcome this problem (with fully convolutional neural networks etc...), and ...
1
vote
0
answers
105
views
Convergence speed of perceptron algorithm
I was reading the convergence proof for the perceptron algorithm. It says under the assumption that there are some $R$, $\theta^*$ with $|\theta^*| = 1$ and $\gamma > 0$, such that $y_t(x_t\cdot \...
- 11
1
vote
1
answer
44
views
Multilayer Perceptron: What is the value used to update the weights in the hidden layers?
As i understand for the output layer the error rate is used with the mean squared error function to update the weights.
For the hidden layers as well? Does that make sense?
- 113
2
votes
3
answers
186
views
About different structures of neural network
https://www.mathworks.com/help/deeplearning/ref/fitnet.html is the tutorial that I am following to understand fitting data to a function. I have few doubts regarding structure and terminologies which ...
- 511
2
votes
1
answer
249
views
Conceptual questions on MLP and Perceptrons
I am facing some confusion regarding the terminologies assocaiated to classification and regression problems esp. using the MLP and Perceptron models.
These are the following:
1) When the data is ...
- 511
1
vote
1
answer
2k
views
How to find bias for perceptron algorithm?
My question is very basic. I am starting with ML and am working on the perceptron algorithm. I successfully computed the weights for this input data:
...
- 195
1
vote
1
answer
84
views
Error while writing perceptron algorithm binary classifier
I am a beginner and I am designing an binary classifier using Perceptron algorithm using FASHION-MNIST dataset. While designing the same I have written the following code:
...
- 11
1
vote
0
answers
55
views
Simple perceptron on python
I want to create a simple perceptron with three inputs.
$ \sigma(w_1x_1 + w_2x_2 + w_3x_3) $, where $ \sigma = \frac{1}{1 + e^{-x}.} $
$$ L(w) = \sum_i^3(y_i -\sigma(w_1x_1 + w_2x_2 + w_3x_3)); $$
$ \...
- 47
0
votes
0
answers
101
views
Similarity of perceptron criterion and SVM
In the book "Neural Networks and Deep Learning" by Aggarwal there is an exercise 2.10.1:
Consider the following loss function for training pair $(\overline{X},y)$:
$$L=max(0, a -y(\overline{W} \...
- 19
0
votes
0
answers
31
views
What are contexts where a perceptron model could be defined?
per this post, a perceptron could use the logit function as the activation function.
per wiki
In the context of neural networks, a perceptron is an artificial
neuron using the Heaviside step ...
- 157
1
vote
1
answer
228
views
What is the hypothesis space used by this AND gate Perceptron?
Per this post
The hypothesis space used by a machine learning system is the set of
all hypotheses that might possibly be returned by it.
Per this post, the Perceptron algorithm makes prediction
...
- 157
2
votes
0
answers
306
views
How does permutation of training data improve convergence time when training a perceptron or neural network model? [duplicate]
I'm currently studying some basic concepts regarding Deep Learning and Neural Networks with this material.
When discussing the training algorithm for a perceptron, the author states that looping ...
2
votes
1
answer
66
views
Version of Perceptron
If we change the $ywx<0$ condition (for performing update) to $ywx<1$ like in SVM (but without adding regularization to maximize the margin), is there any difference from the basic perceptron (...
- 51
3
votes
2
answers
302
views
Geometric interpretation of MLP output
I am really interested in the geometric interpretation of perceptron outputs, mainly as a way to better understand what the network is really doing, but I can't seem to find much information on this ...
0
votes
1
answer
178
views
Abstraction in Neural Networks
In my nonlinear dynamics class in college, we discussed a simple perceptron with two input neurons and one output neuron that is trained on the patterns
...
- 342
3
votes
2
answers
314
views
Perceptron algorithm
I am trying to build a simple linear classifier. I have two classes A and B each with two features [x, y] and hence a 2d dataset. Now, I need to find the equation of the line that separates the two ...
- 131
4
votes
1
answer
422
views
Ambiguity in Perceptron loss function (C. Bishop vs F. Rosenblatt)
Bishop's Perceptron loss
On one hand, it is stated in equation 4.54 of Chris Bishop's book (pattern recognition and machine learning) that the loss function of perceptron algorithm is given by:
$${...
- 1,237
1
vote
0
answers
2k
views
(Almost) linearely separable dataset, where can I find one?
I'm implementing the perceptron algorithm and the voted perceptron algorithm for an assignment for university.
For that I need to find some decent datasets.. I've tried the UCI repos and I've come up ...
1
vote
1
answer
158
views
neural networks error function: is global minimum desirable?
In "Elements of statistical learning" page 395 the authors state that, relative to R(θ), the regression/classification error function in a neural network such as a multi layer perceptron:
Typically ...
- 35