This definition does not quite apply since we are not always assuming an underlying distribution. So what is a model really? Can a Gradient Boosted Model (GBM) with specified hyperparameters be considered a model? Is a model a collection of rules?
$\begingroup$
$\endgroup$
1
asked Jul 21, 2016 at 14:12
-
2$\begingroup$ You could just take one step further generic-ness and look at en.wikipedia.org/wiki/Mathematical_model - most ML models will match that definition, even if they do not match "statistical model" (although I think almost all trained unsupervised or supervised ML models would be considered statistical models). $\endgroup$– Neil SlaterJul 21, 2016 at 14:19
Add a comment
|
8 Answers
$\begingroup$
$\endgroup$
3
I was interested in the same question recently and came to the realization that there is no single definition of a "model" in machine learning. It's highly dependent on the sources you're consulting, which may be the documentation for a particular software program, the slang adopted by its user community, or the definitions used in published academic papers, which can vary widely from journal to journal. Moreover, I had to learn to keep in mind that such papers are written not just by specialists in machine learning, but by experts in other disciplines who have a need for applying machine learning techniques (such as imaging, various medical fields, etc.). Many of them do not explicitly define the term "model," which is often used loosely. Here are just a couple of different definition of "model" I've seen in all this widely scattered literature:
• Statistical models, particularly the stats related to probability distributions.
• Regression data and related statistics.
• Mathematical models as mentioned by Neil Slater above.
• The data models used in machine learning, such as the columns involved, their data types, the data sources and other metadata. This is particularly tricky because there's nothing mathematical about this definition at all, unlike the first three I listed. For an example, see all the documentation for SQL Server "mining models," which serve double-duty for machine learning purposes.
• Sometimes all of the definitions above are expanded to include machine learning structures built on top of the equations and the metadata, such as the specifications of neural nets. In other cases, these are considered separate entities.
All of the above are sometimes mixed and matched together, depending on the source. I'm sure there are other definitions of "model" I've left off this list, which will complicate the matter even further. To deal with this ambiguity, I'm trying to train myself to divine the intentions of the author whenever they use the term "model." Sometimes it's easy to determine based on the context or the field the author works in, but other times I have to read deeply into an article or documentation before figuring it out. I wish I could be more definitive about it, but it's really a naturally fuzzy term; there's never going to be a simple one-size-fits-all answer to this. I hope that helps.
answered Jul 21, 2016 at 15:46
-
4$\begingroup$ While the term "model" might indeed apply to plenty of things, in the context of Machine Learning, the model is the artifact created through learning/training - e.g. in the case of neural networks the weights and biases of that network. $\endgroup$– SyzygyJan 7, 2018 at 15:35
-
$\begingroup$ I think you are missing the link between model, hypothesis and hypothesis space. In some way the ML algorithm searches through the hypothesis space to come up with the best fitting to make predictions on the data also check this for rererence $\endgroup$ Feb 15, 2022 at 20:58
-
$\begingroup$ Even though the ML model is often a quite theoretically approach the result of learning the model could definitely be applied to a concrete hypothesis so it relates back to the normal meaning of a model in general language (simplification for understanding). This thougt makes since at least when I think of a couple of different ML approaches. $\endgroup$ Feb 15, 2022 at 21:03
$\begingroup$
$\endgroup$
From article on Amazon Machine Learning
The process of training an ML model involves providing an ML algorithm (that is, the learning algorithm) with training data to learn from. The term ML model refers to the model artifact that is created by the training process.
$\begingroup$
$\endgroup$
3
I like the Machine Learning definition given by Tom Mitchell.
A computer program is said to learn from experience E with respect to some class of tasks T and performance measure P if its performance at tasks in T, as measured by P, improves with experience E.
So, given this definition, I should say that a model is the acquired experience after doing some task T.
-
1$\begingroup$ Isn't the experience E the training set? $\endgroup$ Nov 17, 2019 at 7:16
-
$\begingroup$ Experience would be a linking word, you need something that experiences - the ML algorithm, and something to experience - the training data. So the ML alg. "experiences" the data, and the result of that "experience" is some parameters which would substitute the variables of a formula/equation. That "explicited" / concretized equation is what is most generally used as "model". Mitchell doesn't use the word model in that definition, so nothing can be inferred about 'model' from that. The word is also used for other things than experience. For me it is the result of that experience: equat +values $\endgroup$– MartinJan 23, 2021 at 12:54
-
1$\begingroup$ Tom has also a great definition on hypothesis and hypothesis space, I should say that a model is the acquired experience after doing some task T learnt through falsifying hypothesis from a given hypothesis space. There even might be different hypothesis spaces relevant for a problem. For reference check both answers So S and Lerner Zhangs answere here $\endgroup$ Feb 15, 2022 at 21:15
$\begingroup$
$\endgroup$
1
A model, loosely speaking, is a simplification of some thing or process. For example, the shape of the Earth not actually a sphere, but we might treat it as one if we are designing a globe. Similarly, assuming the universe is deterministic, there is some natural process which determines if a customer will buy a product on a website. We might construct a something that approximates that process, which we could give some information about a customer and which tells us if it thinks that customer will buy a product.
A "machine learning model", then, is a model constructed by a machine learning system.
(Apologies for this not being a rigorous answer, but I hope this is still useful.)
-
$\begingroup$ I would love to see a connection to the hypothesis and hypothesis space in your answer $\endgroup$ Feb 15, 2022 at 20:53
$\begingroup$
$\endgroup$
In machine learning paradigm, model refers to a mathematical expression of model parameters along with input place holders for each prediction, class and action for regression, classification and reinforcement categories respectively.
This expression is embedded in the single neuron as a model.
For single layer perceptron and deep learning model, one needs to extract this model by carefully walking the neurons and layers to collect and stitch activation function in an ordered fashion.
$\begingroup$
$\endgroup$
In machine learning, a model is an abstraction that can perform a prediction, (re-)action or transformation to or in respect of an instance of input values. A model could be a single number such as the mean value of a set of observations which is often used as a baseline model, a polynomial expression or a set of rules (e.g. decision tree) that define how to get to generate the output.
In general, a model is defined by a set of rules and hyper-parameters that define the model's structure and capacity to be optimized to perform the task at hand. A hyper-parameter could be the degree of the polynomial or the depth of the decision tree. A model can be subjected to an optimization process where parameters are optimized against a certain objective.
The optimization process is often referred to as training for fitting and results in a fitted model, which also can be simply referred to as model. If a model was trained or not often needs to be deduced from the context.
$\begingroup$
$\endgroup$
This is a fun discussion! My two cents are that a model is stored information that a computer can interpret to estimate mappings from some set of possible inputs to a set of appropriate outputs. A model is nothing more or less than the definition of a simple function that approximates a more complex function. It’s not necessary for the complex function to be a real-world phenomenon, only for the model to approximate the complex function without storing sufficient information to reproduce it perfectly.
answered Apr 12, 2020 at 18:58
$\begingroup$
$\endgroup$
In machine learning, the model is the center of gravity, everything revolves around it. Yet, people have different definitions of 'model'; but in my opinion, the best definition of model in ML is "the hypothesis that has learnt to predict, i.e. to fit, unseen data".
