0
$\begingroup$

While handling data, many anomalies occur that might change the results. For example, outliers, missing values, etc. So in this context, what is the better tool to pre process data? A visual tool like MS-Excel or programming tool like R or Python

$\endgroup$
3
$\begingroup$

I mean most of the time it's very much about preference really.

If you want any indication on when to use what:

  • MS-Excel can only handle so much data. Good for quick analysis of small data but neither good for production or middle to large amounts of data

  • R great tool for calculating stuff, running models, etc. Pre-Processing is good, lots of libraries to use. Some people don't like using R for production pipelines but I know companies where this is working just fine

  • Python is a great overall language for pre-processing and creating production worthy data pipelines. So if you want to have a scalable, reusable pre prcessing pipeline this is certainly one of the ways to go

Of course Excel has the lowest learning curve compared to R and Python. The latter ones are similar in that.

$\endgroup$
1
$\begingroup$

I tool like MS Excel or Libre Office Calc (open source) is nice to view data in a table and has a low barrier to entry - after playing around for 30 minutes, you can probably get most basic tasks done.

Using a programming language like R or Python opens up many many other opportunities for more advanced analysis. People write packages that will do a lot of work for you. Have a look at these examples R, or Pandas for Python for examples, both for some simple data analysis.

Detecting outliers can be done in a lot of ways:

$\endgroup$
0
$\begingroup$

I'd suggest you go straight to R or Python, or whatever, rather than use Excel or an open source alternative. You will be very limited in what you can do with a spreadsheet compared to a proper programming environment and you will probably very quickly abandon the spreadsheet, so why waste the time in the first place?

Although going straight to a programming environment might seem daunting, there are plenty of resources online where you will be able to copy and paste code for all common data preprocessing tasks. If you choose R, there are packages devoted exclusively to this task, e.g. vtreat ( see here or here ) or the Python version here.

When you have such powerful resources as these freely available, the spreadsheet route is a real dead end and is to be avoided as much as possible.

$\endgroup$

Not the answer you're looking for? Browse other questions tagged or ask your own question.