Questions tagged [perceptron]

Perceptron is a basic linear classifier that outputs binary labels.

Filter by
Sorted by
Tagged with
1
vote
0answers
12 views

difference between empirical risk minimization and structural risk minimization?

I understand the meaning of empirical risk minimization as separate topic and was reading about structural risk minimization, it is hard for me to understand the difference between these two. I read ...
5
votes
3answers
840 views

Why must a CNN have fixed input size?

I'm studying right now Convolutional Neural Networks. Why a CNN must have fixed input? I know that it's possible to overcome this problem (with fully convolutional neural networks ecc...), and i also ...
1
vote
0answers
27 views

Convergence speed of perceptron algorithm

I was reading the convergence proof for the perceptron algorithm. It says under the assumption that there are some $R$, $\theta^*$ with $|\theta^*| = 1$ and $\gamma > 0$, such that $y_t(x_t\cdot \...
1
vote
1answer
20 views

Multilayer Perceptron: What is the value used to update the weights in the hidden layers?

As i understand for the output layer the error rate is used with the mean squared error function to update the weights. For the hidden layers as well? Does that make sense?
2
votes
2answers
51 views

About different structures of neural network

https://www.mathworks.com/help/deeplearning/ref/fitnet.html is the tutorial that I am following to understand fitting data to a function. I have few doubts regarding structure and terminologies which ...
2
votes
1answer
38 views

Conceptual questions on MLP and Perceptrons

I am facing some confusion regarding the terminologies assocaiated to classification and regression problems esp. using the MLP and Perceptron models. These are the following: 1) When the data is ...
0
votes
1answer
55 views

How to find bias for perceptron algorithm?

My question is very basic. I am starting with ML and am working on the perceptron algorithm. I successfully computed the weights for this input data: ...
0
votes
0answers
15 views

Error while writing perceptron algorithm binary classifier

I am a beginner and I am designing an binary classifier using Perceptron algorithm using FASHION-MNIST dataset. While designing the same I have written the following code: ...
1
vote
0answers
26 views

Simple perceptron on python

I want to create a simple perceptron with three inputs. $ \sigma(w_1x_1 + w_2x_2 + w_3x_3) $, where $ \sigma = \frac{1}{1 + e^{-x}.} $ $$ L(w) = \sum_i^3(y_i -\sigma(w_1x_1 + w_2x_2 + w_3x_3)); $$ $ \...
0
votes
0answers
19 views

Similarity of perceptron criterion and SVM

In the book "Neural Networks and Deep Learning" by Aggarwal there is an exercise 2.10.1: Consider the following loss function for training pair $(\overline{X},y)$: $$L=max(0, a -y(\overline{W} \...
0
votes
0answers
12 views

What are contexts where a perceptron model could be defined?

per this post, a perceptron could use the logit function as the activation function. per wiki In the context of neural networks, a perceptron is an artificial neuron using the Heaviside step ...
1
vote
1answer
50 views

What is the hypothesis space used by this AND gate Perceptron?

Per this post The hypothesis space used by a machine learning system is the set of all hypotheses that might possibly be returned by it. Per this post, the Perceptron algorithm makes prediction ...
2
votes
0answers
28 views

How does permutation of training data improve convergence time when training a perceptron or neural network model? [duplicate]

I'm currently studying some basic concepts regarding Deep Learning and Neural Networks with this material. When discussing the training algorithm for a perceptron, the author states that looping ...
1
vote
0answers
28 views

Version of Perceptron

If we change the $ywx<0$ condition (for performing update) to $ywx<1$ like in SVM (but without adding regularization to maximize the margin), is there any difference from the basic perceptron (...
0
votes
0answers
8 views

Why kernel perceptron relies more on the training data than kernel SVM?

In UCSD machine learning course, it is said that: "for Kernel Perceptron, the solution is likely to depend on more of the training points than the ...
3
votes
1answer
75 views

Geometric interpretation of MLP output

I am really interested in the geometric interpretation of perceptron outputs, mainly as a way to better understand what the network is really doing, but I can't seem to find much information on this ...
0
votes
1answer
76 views

Abstraction in Neural Networks

In my nonlinear dynamics class in college, we discussed a simple perceptron with two input neurons and one output neuron that is trained on the patterns ...
3
votes
2answers
93 views

Perceptron algorithm

I am trying to build a simple linear classifier. I have two classes A and B each with two features [x, y] and hence a 2d dataset. Now, I need to find the equation of the line that separates the two ...
3
votes
1answer
188 views

Ambiguity in Perceptron loss function (C. Bishop vs F. Rosenblatt)

Bishop's Perceptron loss On one hand, it is stated in equation 4.54 of Chris Bishop's book (pattern recognition and machine learning) that the loss function of perceptron algorithm is given by: $${...
0
votes
0answers
77 views

(Almost) linearely separable dataset, where can I find one?

I'm implementing the perceptron algorithm and the voted perceptron algorithm for an assignment for university. For that I need to find some decent datasets.. I've tried the UCI repos and I've come up ...
0
votes
0answers
28 views

Use of MLP with one hidden layer and direct weights from input to output units

One of the questions I saw online while reading about MLPs was - "Consider an MLP architecture with one hidden layer where there are also direct weights from the inputs directly to the output units. ...
1
vote
1answer
43 views

neural networks error function: is global minimum desirable?

In "Elements of statistical learning" page 395 the authors state that, relative to R(θ), the regression/classification error function in a neural network such as a multi layer perceptron: Typically ...
1
vote
3answers
156 views

How do two perceptrons produce different linear decision boundaries?

I'm trying to visualize how two perceptrons converge to two different decision boundaries (which is ultimately used to create the classifier for the non-linearly separable data). Source: https://tdb-...
1
vote
0answers
58 views

Interpreting MLP output

I just wrote an MLP in Python. After having trained it, I pass in some test data to see the result, and I get an array of decimal numbers at the output, rather than the desired binary output. For ...
3
votes
1answer
33 views

Coding MLP: good practices?

I recently finished coding my own MLP neural network in Python. To make my code easier to read, I separated the MLP, into classes; the network class, the layers class and the neuron class, where the ...
5
votes
4answers
177 views

Is the prediction algorithm absolutely the same for all linear classifiers?

Is the prediction algorithm absolutely the same for all linear classifiers and linear regression algorithms? As known, any linear classifier can be described as: ...
0
votes
1answer
79 views

Confusion on Delta Rule and Error

I'm currently reading Mitchell's book for Machine Learning, and he just started gradient descent. There's one part that's really confusing me. At one point, he gives this equation for the error of a ...
2
votes
1answer
108 views

Perceptron Primitive Boolean Functions

Thanks for reading. I'm currently reading Tom Mitchell's Machine Learning (I'm a beginner into ML), and I'm on chapter 4 about perceptrons. I'm really confused about this paragraph: I understand the ...
0
votes
1answer
126 views

Why can't the XOR linear inseparability problem be solved with one perceptron - like this?

Consider a perceptron where $w_0=1$ and $w_1=1$: Now, say we use an activation function $f(x)=1,~for~x=1$$~~~~~~~~~~~~~0, otherwise$ The output is then summarised as: $x_0~~~~~x_1~~~~~w_0*x_0 + ...
1
vote
1answer
67 views

Effect of adding extra unrelated features to linear perceptron

Suppose that we are training a linear regressor (perceptron). Adding extra features that are not related to the target (e.g. randomly generated values) before training will typically ____ our training ...
0
votes
1answer
1k views

sklearn.GridSearchCV predict method not providing the best estimate and accuracy score

I was playing around with the credit default dataset in UCI ("https://archive.ics.uci.edu/ml/machine-learning-databases/00350/default%20of%20credit%20card%20clients.xls") These are the steps i have ...
0
votes
1answer
74 views

Why my perceptron doesn't train well and produces bad results on test data?

I am a newbie in Machine learning and I am writing a small code for Perceptron. This is the first time I am writing code in Python. I have four training data points (X). As they are used for ...
1
vote
0answers
165 views

Why perceptron does not converge on data not linearly separable

This is how I understand the perceptron algorithm. The perceptron loss function is the hinge loss $\ell(w,x,y) = \max(0, -yw\cdot x)$. Suppose the data set is $D = \{(x_1,y_1),\dots,(x_n,y_n)\}$ with ...
1
vote
2answers
271 views

Learning rate of perception

I don't understand the following statement: The choice of learning rate m does not matter (for Perceptron) because it just changes the scaling of w (weights). The site with this statement that ...
1
vote
1answer
167 views

Machine Learning Perceptron Algorithm

I'm studying Machine Learning using Sebastian Raschka's book. Wonder if someone could please help me to confirm if I have the following steps correct if I apply Perceptron Algorithm to Iris dataset ...
5
votes
2answers
3k views

What is the difference between Perceptron and ADALINE?

What is the difference between Perceptron and ADALINE?
3
votes
1answer
26 views

In some places, perceptron is described as having added bias, and in some places as bias is not added

In some places, perceptron is described as having added bias, and in some places as bias is not added. Which one is right for you?
1
vote
1answer
77 views

How does combining two linear perceptrons create non-linear boundaries?

I don't understand the equation that you get from combining the two linear perceptrons is non-linear? The video starts with two linear perceptrons with the equations: $$e1 = 5x_1 -2x_2 - 8 = 0 \...
1
vote
0answers
484 views

Optimizing an averaged perceptron algorithm using numpy and scipy instead of dictionaries

So I'm trying to write an averaged perceptron algorithm (page 48 here for the equation) in python. Instead of storing the historical weights, I simply accumulate the weights and then multiply ...
0
votes
1answer
82 views

Trouble with accuracy of multiclass perceptron

I have built a multiclass perceptron, but it has low accuracy (around 80%). I think I'm missing something. One possibility is that I should add a bias, but I'm not sure how to incorporate that. The ...
1
vote
1answer
325 views

Predict method of the perceptron algorithm

Can someone explain to me how the predict method of the perceptron algorithm works? ...
2
votes
0answers
583 views

Are single input single output neural networks possible?

This might be a weird question but I'm trying to have a deep understanding of how neural networks work theoretically. I was doing some tests with my perceptron and I decided to test it on a single ...
1
vote
1answer
15 views

Weights and bias' relative to preprocessed X

I am currently using sklearn scale to preprocess my X data before being put into a perceptron - mean/stddev so as to prevent the ...
0
votes
1answer
2k views

number of neurons for mnist dataset using mlp?

I am trying to find out what is optimum number of neurons that can be used in MNIST dataset(60,000 training and 10,000 testing data). I build a single hidden layer model using keras,with relu ...
0
votes
1answer
535 views

Perceptron learning rate irrelevant in convergence [duplicate]

Via this MIT document i studied the single layer Perceptron convergence proof (= maximum number of steps). In the convergence proof inside this document , the learning rate is implicitly defined as 1 ...
3
votes
1answer
527 views

Learning rate in the Perceptron Proof and Convergence

Every perceptron convergence proof i've looked at implicitly uses a learning rate = 1. However, the book I'm using ("Machine learning with Python") suggests to use a small learning rate for ...
2
votes
2answers
198 views

Normalizing the final weights vector in the upper bound on the Perceptron's convergence

The convergence of the "simple" perceptron says that: $$k\leqslant \left ( \frac{R\left \| \bar{\theta} \right \|}{\gamma } \right )^{2}$$ where $k$ is the number of iterations (in which the weights ...
3
votes
1answer
558 views

Perceptron - Convergence proof

I studied the perceptron algorithm and I'm trying to prove the convergence by myself. However, I'm wrong somewhere and I am not able to find the error. Assumption: We assume that there is some $\...
3
votes
2answers
502 views

Perceptron - Which step function to choose

I'm studying Perceptron algorithm. Some books use this step function 1 if x>=0 else -1 where x is a dot product between the weights w and a sample x. Other ...
10
votes
2answers
7k views

Strange behavior with Adam optimizer when training for too long

I'm trying to train a single perceptron (1000 input units, 1 output, no hidden layers) on 64 randomly generated data points. I'm using Pytorch using the Adam optimizer: ...