There's a whole theory of statistical inference based off calculus studying consistency, efficiency, robustness, BLUE, unbiasedness of linear models (Gaussian,Exponential, Chi-square, F-distribution, etc)... that make up regression models.
When is a neural network better than these traditional models based off calculus that are used regression-analysis? Is there a whole mathematical theory including robustness, Rao-blackwell stuff, consistency, sufficency dedicated to them that someone like Wackerly/HOFF writes about in a book?
Are neural networks not explainable by calculus as logistic regression is explainable as the maximum likelihood estimator?
What's the mathematics behind neural networks or is that more CompSci theory?
Are neural networks unbiased estimators that are consistent, efficient, etc that are on average the right answer? Can they estimate the true parameters of the population?