Aim: To find the coefficients for the regression line (hyperplane in case of multiple variables?) that models the data best. Let's call this w
What is the difference between:
1) Estimating using MAP: $w=(XX^T+\lambda I )^{-1}Xy^T$ where $X$ is the input training data and $y$ is the training outputs
and
2) Using a neural network to perform regression (I don't know how this is implemented)
(and any other method used for linear regression)
In the first, you need a linear model and the cost function is RMSE. with this model and cost function, w is generated.
on the other hand, for the second equation, you use a non-linear NN equation as estimation and the cost function may RMSE, log or ... .( NN without activation-function is linear) NN without activation function is a linear model with bias. if you use RMSE it is like the first example.
any linear regression with the same cost function will generate, the same value for coefficient after convergence (In some condition the equation has many solutions, In this case, the cost function may converge to a new value)
Note: many solutions for an equation meaning, for example, if one point is the solution of the equations, then any hyperplane that crosses that point is the solution.
I think that we should clarify that linear regression is a model in which a linear combination of features produce one output. This holds true if we work with a frequents approach, a Bayes Ian framework or a machine learning/AI perspective. For the rest neural network simply (or not simply) fit linear regression with an identity activation function.
https://www.r-bloggers.com/using-neural-network-for-regression/amp/
