Data cleaning and data transformation before EDA?

Question

What is the most common order of data cleaning, data transformation and exploratory data analysis?

For me it seems most logical to do data cleaning, then EDA and finally data transformation (encoding of categorical variables, and feature scaling).

Doing data transformation before the EDA, seems to make the EDA not that useful, as you cant ex. check for stuff like:

Passengers in the age interval 0-18 has a higher chance of survival

(if feature scaling has been applied to age feature).

But then again, doing data transformation after the EDA, also miss out on chance of encoding categorical variables and thereby visualize correlations of those with the target variable.

What is the order of the mentioned processes? And is there even an order?

Answer 1

Although not very helpful, the answer is probably "it depends".

I like to do data cleaning and some EDA together since EDA can highlight appropriate treatments to clean the data - e.g. influencing how to handle missing values.

I think data transformation should be done just prior to modelling; whether or not you need to do any transformation at all depends on the techniques you plan to use.

Answer 2

I agree about "it depends" - upon your goal, and upon the nature of the data, and upon how much you and your team know about the data. For text data, the cleaning methods are pretty clear, so I would probably do data cleaning first. For high-volume image data (cancer screening, seismology), there are major trade-offs between data-reduction and feature-detection. In this high-volume domain, I think your pathway would be less clear - UNLESS you or your team know what methods of data-reduction and cleaning have been useful in the past. Domain knowledge can be very important in selecting and formatting the data for your subsequent analyses.