2
$\begingroup$

My understanding is that training a model is something done in machine learning using training data so that the model can predict values when new data is given to it.

Data mining is the process to find patterns in the existing dataset. So what exactly is the purpose of the 'train model' step in data mining?

$\endgroup$
1
$\begingroup$

There is a very important difference in Machine Learning (ML) between supervised methods and unsupervised methods:

  • Supervised learning consists in training a model with some labelled data in order to make the final model able to predict the label on some new (unlabelled) data. This means that the task is designed by choosing exactly what what one wants to predict. For example the task of predicting the author of a text is completely different from predicting the topic of a text, even if the text might be the same.
    • Note: A "label" is for categorical data, and the task is called classification in this case. The same principle can be applied to numerical values, in which case this is a regression task.
  • Unsupervised learning consists in detecting patterns in the data with no additional information than the data itself. This means that there's no specific "label" to predict. Very often unsupervised learning is some form of clustering, i.e. grouping instances by similarity. Data mining methods usually belong to this category. In general there is no need for separate training and testing steps with unsupervised methods. However one might need to tune some parameters or sometimes to evaluate the model with some annotated data, so this would require separating training and testing again.
$\endgroup$
2
  • $\begingroup$ I agree your answer is about machine learning. How does this fit in data mining which means finding new patterns in existing data? $\endgroup$
    – variable
    Sep 9 '20 at 13:41
  • $\begingroup$ As I said, data mining usually refers to unsupervised tasks. in other words, data mining can be seen as a subset of ML, although technically data mining is a set of problems and ML is a set of methods. $\endgroup$
    – Erwan
    Sep 9 '20 at 14:49
0
$\begingroup$

The purpose of training in data mining is finding the patterns.

  • If you want to get a segmentation using kmean, training means iteratively grouping data together in clusters until points no longer change clusters.
  • If you do basket analysis it means looking at item sets and seeing if they exceed your threshold metric and discarding them if they don't.
  • If you train an autoencoder to represent data in latent space it means using gradient descent to set weights so that they compress the data as best as possible.

    In all these situations we would say we "train a model" because it's simply a good mental framework for iterative methods of solving unsupervised and data mining problems even though there may be no test or validation set to make sure the model performs well.
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.