When discussing big data, it is sometimes mentioned that data modeling can be done by using a tool like map reduce, while data processing may be performed by apache spark. What is the difference between data modeling tasks, and data processing tasks? Thanks in advance
1 Answer
- Data modeling means representing the data, usually with a somewhat compact model. Modeling implies simplifications: this can lead to a good model which reliably represents the patterns in the data or a terrible model which simplifies too much or not enough.
- Data processing is applying any kind of process to the data.
For the record, map-reduce is relevant only as a technique for processing large data efficiently.
-
$\begingroup$ Could you refer a good site for learning modelling? $\endgroup$ Sep 23, 2022 at 4:43
-
$\begingroup$ @user17302 modelling is very specific to the domain, task, data. For example I'm doing NLP so I know techniques for representing text data, I know the type of features needed for different tasks in NLP, etc. That would be totally different for images, biomedical data... Basically you should choose which domain/application you're interested in imho. $\endgroup$– ErwanSep 23, 2022 at 9:17