Questions tagged [perceptron]

Perceptron is a basic linear classifier that outputs binary labels.

Filter by
Sorted by
Tagged with
2
votes
0answers
16 views

Understanding computations of Percetron and Multi-Layer Perceptrons on Geometric level

I am currently watching amazing Deep Learning lecture series from Carnegie Melllon University, but I am having little bit of trouble understanding how Perceptrons and MLP are making their decisions on ...
4
votes
1answer
64 views

How do we define a linearly separable problem?

When we talk about Perceptrons, we say that they are limited for approximating functions that are linearly separable, while Neural Networks that use non-linear transformations are not. I am having ...
3
votes
1answer
90 views

difference between empirical risk minimization and structural risk minimization?

I understand the meaning of empirical risk minimization as separate topic and was reading about structural risk minimization, it is hard for me to understand the difference between these two. I read ...
6
votes
3answers
2k views

Why must a CNN have fixed input size?

I'm studying right now Convolutional Neural Networks. Why a CNN must have fixed input? I know that it's possible to overcome this problem (with fully convolutional neural networks ecc...), and i also ...
1
vote
0answers
38 views

Convergence speed of perceptron algorithm

I was reading the convergence proof for the perceptron algorithm. It says under the assumption that there are some $R$, $\theta^*$ with $|\theta^*| = 1$ and $\gamma > 0$, such that $y_t(x_t\cdot \...
1
vote
1answer
21 views

Multilayer Perceptron: What is the value used to update the weights in the hidden layers?

As i understand for the output layer the error rate is used with the mean squared error function to update the weights. For the hidden layers as well? Does that make sense?
2
votes
2answers
55 views

About different structures of neural network

https://www.mathworks.com/help/deeplearning/ref/fitnet.html is the tutorial that I am following to understand fitting data to a function. I have few doubts regarding structure and terminologies which ...
2
votes
1answer
44 views

Conceptual questions on MLP and Perceptrons

I am facing some confusion regarding the terminologies assocaiated to classification and regression problems esp. using the MLP and Perceptron models. These are the following: 1) When the data is ...
0
votes
1answer
118 views

How to find bias for perceptron algorithm?

My question is very basic. I am starting with ML and am working on the perceptron algorithm. I successfully computed the weights for this input data: ...
0
votes
0answers
25 views

Error while writing perceptron algorithm binary classifier

I am a beginner and I am designing an binary classifier using Perceptron algorithm using FASHION-MNIST dataset. While designing the same I have written the following code: ...
1
vote
0answers
30 views

Simple perceptron on python

I want to create a simple perceptron with three inputs. $ \sigma(w_1x_1 + w_2x_2 + w_3x_3) $, where $ \sigma = \frac{1}{1 + e^{-x}.} $ $$ L(w) = \sum_i^3(y_i -\sigma(w_1x_1 + w_2x_2 + w_3x_3)); $$ $ \...
0
votes
0answers
21 views

Similarity of perceptron criterion and SVM

In the book "Neural Networks and Deep Learning" by Aggarwal there is an exercise 2.10.1: Consider the following loss function for training pair $(\overline{X},y)$: $$L=max(0, a -y(\overline{W} \...
0
votes
0answers
12 views

What are contexts where a perceptron model could be defined?

per this post, a perceptron could use the logit function as the activation function. per wiki In the context of neural networks, a perceptron is an artificial neuron using the Heaviside step ...
1
vote
1answer
71 views

What is the hypothesis space used by this AND gate Perceptron?

Per this post The hypothesis space used by a machine learning system is the set of all hypotheses that might possibly be returned by it. Per this post, the Perceptron algorithm makes prediction ...
2
votes
0answers
38 views

How does permutation of training data improve convergence time when training a perceptron or neural network model? [duplicate]

I'm currently studying some basic concepts regarding Deep Learning and Neural Networks with this material. When discussing the training algorithm for a perceptron, the author states that looping ...
1
vote
0answers
28 views

Version of Perceptron

If we change the $ywx<0$ condition (for performing update) to $ywx<1$ like in SVM (but without adding regularization to maximize the margin), is there any difference from the basic perceptron (...
0
votes
0answers
10 views

Why kernel perceptron relies more on the training data than kernel SVM?

In UCSD machine learning course, it is said that: "for Kernel Perceptron, the solution is likely to depend on more of the training points than the ...
3
votes
1answer
87 views

Geometric interpretation of MLP output

I am really interested in the geometric interpretation of perceptron outputs, mainly as a way to better understand what the network is really doing, but I can't seem to find much information on this ...
0
votes
1answer
100 views

Abstraction in Neural Networks

In my nonlinear dynamics class in college, we discussed a simple perceptron with two input neurons and one output neuron that is trained on the patterns ...
3
votes
2answers
101 views

Perceptron algorithm

I am trying to build a simple linear classifier. I have two classes A and B each with two features [x, y] and hence a 2d dataset. Now, I need to find the equation of the line that separates the two ...
4
votes
1answer
232 views

Ambiguity in Perceptron loss function (C. Bishop vs F. Rosenblatt)

Bishop's Perceptron loss On one hand, it is stated in equation 4.54 of Chris Bishop's book (pattern recognition and machine learning) that the loss function of perceptron algorithm is given by: $${...
0
votes
0answers
111 views

(Almost) linearely separable dataset, where can I find one?

I'm implementing the perceptron algorithm and the voted perceptron algorithm for an assignment for university. For that I need to find some decent datasets.. I've tried the UCI repos and I've come up ...
1
vote
1answer
50 views

neural networks error function: is global minimum desirable?

In "Elements of statistical learning" page 395 the authors state that, relative to R(θ), the regression/classification error function in a neural network such as a multi layer perceptron: Typically ...
1
vote
3answers
187 views

How do two perceptrons produce different linear decision boundaries?

I'm trying to visualize how two perceptrons converge to two different decision boundaries (which is ultimately used to create the classifier for the non-linearly separable data). Source: https://tdb-...
1
vote
0answers
68 views

Interpreting MLP output

I just wrote an MLP in Python. After having trained it, I pass in some test data to see the result, and I get an array of decimal numbers at the output, rather than the desired binary output. For ...
3
votes
1answer
38 views

Coding MLP: good practices?

I recently finished coding my own MLP neural network in Python. To make my code easier to read, I separated the MLP, into classes; the network class, the layers class and the neuron class, where the ...
5
votes
4answers
188 views

Is the prediction algorithm absolutely the same for all linear classifiers?

Is the prediction algorithm absolutely the same for all linear classifiers and linear regression algorithms? As known, any linear classifier can be described as: ...
0
votes
1answer
91 views

Confusion on Delta Rule and Error

I'm currently reading Mitchell's book for Machine Learning, and he just started gradient descent. There's one part that's really confusing me. At one point, he gives this equation for the error of a ...
2
votes
1answer
123 views

Perceptron Primitive Boolean Functions

Thanks for reading. I'm currently reading Tom Mitchell's Machine Learning (I'm a beginner into ML), and I'm on chapter 4 about perceptrons. I'm really confused about this paragraph: I understand the ...
0
votes
1answer
135 views

Why can't the XOR linear inseparability problem be solved with one perceptron - like this?

Consider a perceptron where $w_0=1$ and $w_1=1$: Now, say we use an activation function $f(x)=1,~for~x=1$$~~~~~~~~~~~~~0, otherwise$ The output is then summarised as: $x_0~~~~~x_1~~~~~w_0*x_0 + ...
1
vote
1answer
78 views

Effect of adding extra unrelated features to linear perceptron

Suppose that we are training a linear regressor (perceptron). Adding extra features that are not related to the target (e.g. randomly generated values) before training will typically ____ our training ...
0
votes
1answer
2k views

sklearn.GridSearchCV predict method not providing the best estimate and accuracy score

I was playing around with the credit default dataset in UCI ("https://archive.ics.uci.edu/ml/machine-learning-databases/00350/default%20of%20credit%20card%20clients.xls") These are the steps i have ...
0
votes
1answer
79 views

Why my perceptron doesn't train well and produces bad results on test data?

I am a newbie in Machine learning and I am writing a small code for Perceptron. This is the first time I am writing code in Python. I have four training data points (X). As they are used for ...
1
vote
0answers
182 views

Why perceptron does not converge on data not linearly separable

This is how I understand the perceptron algorithm. The perceptron loss function is the hinge loss $\ell(w,x,y) = \max(0, -yw\cdot x)$. Suppose the data set is $D = \{(x_1,y_1),\dots,(x_n,y_n)\}$ with ...
1
vote
2answers
383 views

Learning rate of perception

I don't understand the following statement: The choice of learning rate m does not matter (for Perceptron) because it just changes the scaling of w (weights). The site with this statement that ...
1
vote
1answer
182 views

Machine Learning Perceptron Algorithm

I'm studying Machine Learning using Sebastian Raschka's book. Wonder if someone could please help me to confirm if I have the following steps correct if I apply Perceptron Algorithm to Iris dataset ...
5
votes
2answers
4k views

What is the difference between Perceptron and ADALINE?

What is the difference between Perceptron and ADALINE?
3
votes
1answer
28 views

In some places, perceptron is described as having added bias, and in some places as bias is not added

In some places, perceptron is described as having added bias, and in some places as bias is not added. Which one is right for you?
1
vote
1answer
90 views

How does combining two linear perceptrons create non-linear boundaries?

I don't understand the equation that you get from combining the two linear perceptrons is non-linear? The video starts with two linear perceptrons with the equations: $$e1 = 5x_1 -2x_2 - 8 = 0 \...
1
vote
0answers
524 views

Optimizing an averaged perceptron algorithm using numpy and scipy instead of dictionaries

So I'm trying to write an averaged perceptron algorithm (page 48 here for the equation) in python. Instead of storing the historical weights, I simply accumulate the weights and then multiply ...
0
votes
1answer
103 views

Trouble with accuracy of multiclass perceptron

I have built a multiclass perceptron, but it has low accuracy (around 80%). I think I'm missing something. One possibility is that I should add a bias, but I'm not sure how to incorporate that. The ...
1
vote
1answer
403 views

Predict method of the perceptron algorithm

Can someone explain to me how the predict method of the perceptron algorithm works? ...
2
votes
0answers
655 views

Are single input single output neural networks possible?

This might be a weird question but I'm trying to have a deep understanding of how neural networks work theoretically. I was doing some tests with my perceptron and I decided to test it on a single ...
1
vote
1answer
15 views

Weights and bias' relative to preprocessed X

I am currently using sklearn scale to preprocess my X data before being put into a perceptron - mean/stddev so as to prevent the ...
0
votes
1answer
3k views

number of neurons for mnist dataset using mlp?

I am trying to find out what is optimum number of neurons that can be used in MNIST dataset(60,000 training and 10,000 testing data). I build a single hidden layer model using keras,with relu ...
0
votes
1answer
610 views

Perceptron learning rate irrelevant in convergence [duplicate]

Via this MIT document i studied the single layer Perceptron convergence proof (= maximum number of steps). In the convergence proof inside this document , the learning rate is implicitly defined as 1 ...
3
votes
1answer
701 views

Learning rate in the Perceptron Proof and Convergence

Every perceptron convergence proof i've looked at implicitly uses a learning rate = 1. However, the book I'm using ("Machine learning with Python") suggests to use a small learning rate for ...
2
votes
2answers
236 views

Normalizing the final weights vector in the upper bound on the Perceptron's convergence

The convergence of the "simple" perceptron says that: $$k\leqslant \left ( \frac{R\left \| \bar{\theta} \right \|}{\gamma } \right )^{2}$$ where $k$ is the number of iterations (in which the weights ...
3
votes
1answer
664 views

Perceptron - Convergence proof

I studied the perceptron algorithm and I'm trying to prove the convergence by myself. However, I'm wrong somewhere and I am not able to find the error. Assumption: We assume that there is some $\...
3
votes
2answers
559 views

Perceptron - Which step function to choose

I'm studying Perceptron algorithm. Some books use this step function 1 if x>=0 else -1 where x is a dot product between the weights w and a sample x. Other ...