4
$\begingroup$

What is the difference between the following four categories:

  1. Artificial Intelligence (AI)
  2. Machine Learning (ML)
  3. Neural Network (NN)
  4. Deep Learning (DL)
  5. Data Science

My current understanding is that each of these encapsulates some set of algorithms. I feel like, amongst ML, NN, and DL, Machine Learning (ML) is the largest set, which contains NN, which in turn contains DL. See figure below. Data Science is the largest set amongst all 5. Is this correct? Where would AI be on this figure?

enter image description here

$\endgroup$
1
  • 1
    $\begingroup$ One of those are not like the others: a neural network is a thing while the rest of those are abstract topics. $\endgroup$ – NotThatGuy Nov 18 '20 at 20:14
1
$\begingroup$

Machine learning is the field that researches methods that fit (= optimize) model structures to data and output a final model that is based on the combination of model structure and optimized model parameters. Statistics and ML are intersecting fields such that some methods belong to both fields, however there exist some methods that only belong to ML and some that only belong to statistics.

Neural networks are a set of such machine learning methods and a subset of those methods are deep learning neural networks (DLNN).

Deep learning methods are a set of machine learning methods that use multiple layers of modelling units. Most research in deep learning is in the area of DLNN, but it is not exclusive, as for example probabilistic circuits are usually not considered to be NN (albeit they look similar on first glance) and they can be deeply layered. Approaches that have hierarchical nature are usually not considered to be "deep", which leads to the question what is meant by "deep" in the first place. An example might be hierarchical clustering methods, of which exist many very different ones - since (probably) every clustering method can be easily made hierarchical.

Artificial intelligence (AI) is difficult to define if one does not want to use it as a buzzword, like "the pursuit of human intelligence". One issue is to define what intelligence is. As far as I can see if one defines AI to be broader than ML, then one ends up with a definition that is basically identical to "computer science", otherwise one ends up with my definition (see above) for ML. One must not make the mistake to just name a regular algorithm - as the minimax algorithm or a rule-based system as deep blue - and call it AI. Otherwise any algorithm would be AI, which would be ridiculous. Possibly https://blog.piekniewski.info/2020/06/08/ai-the-no-bullshit-approach/ will give an answer in how expert system are AI and not ML and not just a rule-based system, but I did not get around to read it, yet.

Data science is used as a buzzword in industry and - except for the term "applied data science" - avoided in research. The main issue is that much of data "science" in industry is no science at all. However, I believe that there is a chance to consider "data science" not just as buzzword, but an actual science. In fact, I am working on a methodology for a while now. I have the chance to teach "data science as a science" at university right now. However, I noticed that my definition is not yet of such quality as I would want to. It is getting better though. To give you an idea, I wrote as a comment to another post:

Without going into detail regarding my methodology/definition, the result of my definition of data science is that you could say it is "mathematical epistemology" - not the actual appliance, but the methodology. Data analytics would then be appliance of the data science principles. You could say that data science is to data analytics, as physic is to mechanical engineering. With that, a company would hire a data scientist / physicist to work on an an engineering project. From https://academia.stackexchange.com/questions/158586/scientific-data-science-conferences-and-journals#comment423975_158587

In this line of argument, "communication skills" are not a part of data science, in the same way as they are not a part of medicine, even though a physician should be a good communicator in order to be effective.

So, ML is a superset of NN and DL, which intersect. AI is (based on my research on the topic so far) the same as ML. Data Science uses methods from ML, but it also uses other methods, e.g. from non-ML statistics.

$\endgroup$
5
$\begingroup$
  • AI: a super general term, means a bit of everything... and nothing at the same time. It's all about building intelligent machines, even though its meaning is not fully developed. It's not used in a rigorous way at all. In fact, scientists prefer to use more technical terms like the others you listed.

  • ML: it's an approach to data, and it's all based on building models that improve automatically from experience.

  • DL: Deep Learning, it's the subset of ML that makes use of Deep Neural Networks. So technically it's still Machine Learning, currently its most advanced and promising branch.

  • NN: The general class of models that is employed in DL. DL is the field of application of NN's.

Personally, I don't really like that Venn diagram you posted. DL being a subset of NN... that doesn't make sense to me.

$\endgroup$
3
  • $\begingroup$ Thanks! Isn't it the case, however, that DL is about using deep neural networks (as opposed to shallow neural networks)? This would make DL a narrower class than a more general NN set. $\endgroup$ – Capo Pavel Mestre Nov 18 '20 at 12:10
  • $\begingroup$ Oh ok, but let's distinguish between models and approaches. ML is an approach to data, while NN are a class of models. $\endgroup$ – Leevo Nov 18 '20 at 13:02
  • 2
    $\begingroup$ A common example that illustrates DL as subset of NN is all the "last century" neural network methods - e.g. perceptron classifiers of 1950s-1960s were considered a ML method in the AI field back then, they definitely are a neural network, but they definitely are not a DL method, they are shallow networks with few or even just a single layer, and the DL name was chosen to explicitly differentiate the "new" NNs against the old, non-deep neural networks. So no matter how you would draw a Venn diagram, you have to leave a substantial space for these techniques that's MM, AI and NN but not DL. $\endgroup$ – Peteris Nov 18 '20 at 19:26
3
$\begingroup$

Artificial Intelligence
It's a general idea to attain human-level intelligence using machines. This is more of a pursuit of human-kind i.e. an ongoing journey.
It goes back to 1950 when Alan Turing devised the Idea of the Turing test.

Machine Learning
This is an approach where the program(model) is automatically built using the data.
This approach is unlike the traditional programming approach. $\hspace{2cm}$ enter image description here

Deep Learning
A recent state of the art approach using a deep neural network enabled us to analyze very large and complex data e.g. Images/Text.

On Venn Diagram -
AI > ML > DL but Neural Network should be used interchangeably with Deep Learning

Data Science
Data Science will not fall into the above Categorization.
The above categorization is more on the evolution of AI but to describe Data Science, we should look into the flow of data from Capturing to Reporting. Modeling is just one step in the process.

Data science is an interdisciplinary field focused on extracting knowledge from data sets, which are typically large (see big data).[6] The field encompasses analysis, preparing data for analysis, and presenting findings to inform high-level decisions in an organization. As such, it incorporates skills from computer science, mathematics, statistics, information visualization, graphic design, complex systems, communication, and business Wikipedia

enter image description here

$\endgroup$
2
  • 1
    $\begingroup$ Thanks! Isn't it the case, however, that DL is about using deep neural networks (as opposed to shallow neural networks)? This would make DL a narrower class than a more general NN set. $\endgroup$ – Capo Pavel Mestre Nov 18 '20 at 12:11
  • $\begingroup$ We may, but then we can also add DL-->RNN-->LSTM, etc. if we see at a very crude technical ground but this flow is more on evolution in a long duration. In this frame, shallow NN didn't do any better than the traditional ML algorithms. Status quo changed only with deep NN success i.e. Yoshua Bengio paper, 2012 ImageNet competition, etc. $\endgroup$ – 10xAI Nov 18 '20 at 14:27
1
$\begingroup$

Machine Learning (ML) is a subset of Artificial Intelligence (AI). For example, the Minimax algorithm is also part of the larger field of AI, but the approach is not based on ML. In fact these algorithms showed much more promising results in the earlier stages of AI research (Deep Blue was the first computer to win a grandmaster in chess and it did not use any ML). You can get an good idea of the various other fields included in AI by looking at the well-known book: Artificial Intelligence: A Modern Approach.

In my opinion, I would not try to fit the concept of Data Science in the illustration above, since the definition is not well defined (or rather defined differently in various literature) and is actually quite broad. That being said, AI will probably be considered by most people as part of Data Science.

$\endgroup$

Not the answer you're looking for? Browse other questions tagged or ask your own question.