2

I think the problem is in your predict method: (self.bias + self.weights * inputs).sum(axis=1) adds the bias to both of the weight*input values before summing (the arrays are broadcast to the same shape). Hence why the 2*intercept makes things match up.


2

what issue arises, when trying to train this network with gradient descent? The activation function is sign function or signum function (A little modified). So, its Derivative will be 0 at all the points Hence, the Gradient descent won’t be able to make progress in updating the weights and backpropagation will fail.


1

I think that the "neurons" analogy is not very helpful to understand what is going on with artificial neural networks. Neural networks are not comprised by "neurons", but by differentiable operations. These operations are arbitrary, e.g. convolutions, indexing (in embeddings), pooling, etc. What you proposed is a perfectly valid building ...


1

The example you cited (using x^2 instead of x) is the idea more popular outside deep learning community, called feature engineering. The trend in neural network modeling is instead to, Play with weights (w) and fine tune them. Not change the input vector (x) but feed it to the network directly. If a single layer neural network is not good enough, add more ...


1

In a fully connected setting the bias shifts the weighted sum of the previous node output by a certain amount before applying the activation function. In practice, it's a column vector b (bias [initialized as a constant vector]) added to the vector Wx (the product of weight matrix (W) and input vector (x)) as: $$\mathrm{Layer2output} = W.\mathrm{Layer1output}...


1

No, bias neurons are not connected to any previous neuron. This is visualized like this: (source)


1

MP model : 1) inputs are binary values; 2) has not weights Rosenblatt model: 1) inputs can take any real numbers; 2) has weights. Thank you, https://medium.com/@manushaurya/mcculloch-pitts-neuron-vs-perceptron-model-8668ed82c36


1

I will answer your questions one by one: By hidden layer we mean the layer that is inbetween the input and output. If number of layers = 1 with 10 hidden neurons (as shown in second figure) then is it essentially a neural network which is termed as an MLP. Is my understanding correct? The fundamental building block of a Neural Network is the perceptron. It'...


Only top voted, non community-wiki answers of a minimum length are eligible