1
$\begingroup$

In R, there are libraries which help speed up the process of reading huge data files. Examples of these libraries include sqldf and ff.

What would be the disadvantage of using these packages to read a small file that one would typically read with read.csv?

If their is no disadvantage of using them to read small files, does this mean functions such as read.csv might not have much use in future given the emerging big-data-targeted readers?

$\endgroup$
  • 1
    $\begingroup$ sqldf creates a database, imports the data and then deletes the database. Seems like a lot of work for a small csv file $\endgroup$ – oW_ Aug 16 '17 at 0:24
1
$\begingroup$

In R many great improvements are typically made by creating new libraries rather than changing the actual functions in base R itself. datatable, read_csv in readr, tibbles, etc are very good examples.

read.csv vs fread as answered by Matt Dowle himself - "https://stackoverflow.com/questions/24424361/reason-behind-speed-of-fread-in-data-table-package-in-r"

read.csv coerces strings as factors by default which is not done by read_csv and fread. So if that is what you need, read.csv will work well. Now answering your question,there is no disadvantage using the fast readers to read small files apart from a small speedup that you might get.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.