Well, thanks to the universal approximation theorem from a purely theoretical point of view, absolutely nothing.
The main issue is with computation. You can find more information here. Mainly, you want functions easy to calculate (polynomials are ok) but with specific regions where derivatives are monotonic (here polynomials are not good) and approximating ...
I will answer your questions one by one:
By hidden layer we mean the layer that is inbetween the input and output. If number of layers = 1 with 10 hidden neurons (as shown in second figure) then is it essentially a neural network which is termed as an MLP. Is my understanding correct?
The fundamental building block of a Neural Network is the perceptron. It'...
I think the problem is in your predict method:
(self.bias + self.weights * inputs).sum(axis=1)
adds the bias to both of the weight*input values before summing (the arrays are broadcast to the same shape). Hence why the 2*intercept makes things match up.