2
$\begingroup$

I've been thinking about the difference between ML modelling and statistical modelling.

I would to ask, on a philosophical level, is my thinking correct: modelling is basically a process of fitting a data-generating function to a set of data. Is this the case that in statistical modelling, we are explicitly finding a function that's expressible in parameters (in a manual way), but in ML modelling, we just automate this process, at the expense we can never write down explicitly a formula for the resultant model obtained from ML model?

Improve this question
$\endgroup$

1 Answer 1

Reset to default
0
$\begingroup$

I disagree. Deep neural networks, about as “machine learning” as one can get, are just functions of the features. You might have a big (huge) equation with millions or billions or parameters, but it’s just an equation, and a small neural network has a small equation that you can fit on a notebook page.

Frank Harrell’s blog has some interesting posts on statistical modeling vs machine learning.

https://www.fharrell.com/post/stat-ml/

Improve this answer
$\endgroup$
3
  • $\begingroup$ It’s a worthwhile exercise to try writing the equation for a small neural network, perhaps with two input features and two hidden layers, each with two neurons with some activation function (say ReLU). $\endgroup$
    – Dave
    Dec 17, 2021 at 4:15
  • $\begingroup$ how about tree-based ensemble models? And SVM? $\endgroup$
    – Student
    Dec 20, 2021 at 0:56
  • $\begingroup$ I’m not so convinced that tree-based ensemble methods and SVMs deviate from what I wrote, but all you need is the neural network example to show that writing an explicit function between the feature space and the output space is not unique to the statistical modeing. $\endgroup$
    – Dave
    Dec 20, 2021 at 1:01

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.