1
$\begingroup$

I originally had a wide CSV dataset of about 18000 columns(and about 80 rows) that I am trying to read in R. It was stored in an Excel sheet,which unfortunately has a limit of only 16384 columns. Hence, taking the dimension I obtain:

> dim(train_set)
[1]    83 16384

i.e 1000+ columns are getting eaten up ,and this would badly affect the accuracy of the predictions. How can I read all the columns in R?

Your suggestions are much appreciated. Thanks a lot!

$\endgroup$
  • $\begingroup$ You should be able to directly read the csv file using read.csv. I am not sure I follow you completely. Are you saying you don't have the original file anymore just the spreadsheet? Or are you trying to read from the spreadsheet which for some reason you must do? $\endgroup$ – Drj Jun 5 '16 at 18:33
  • $\begingroup$ How are you going to get data from a file if it was already lost? how would reading in R help that? $\endgroup$ – Sean Owen Jun 6 '16 at 8:11
  • $\begingroup$ The csv contain a lot of rows (I don't know which is the limit). The xslx does not. Go back to your csv and use that to read. I had your same issue some days ago and I solved in that wat. $\endgroup$ – Andrea Ianni ௫ Jun 6 '16 at 18:39
1
$\begingroup$

Refer to my comment, I believe what you need is

df <- read.csv("./yourpath/yourfile", sep = ";", header = TRUE) # play around with the arguments per your file.
$\endgroup$
  • 1
    $\begingroup$ The command above cleared my confusion. Yes,you understood my question correctly- I did have the original file,but I had to stick to reading from the spreadsheet for some reason. $\endgroup$ – shaw38 Jun 7 '16 at 5:31
  • $\begingroup$ @shawn38 Ohh Ok. For R, the source does not really matter once the data is in. I deal with a lot of spreadsheets, but i prefer converting them to csv before reading in R to avoid formatting (especially dates and negative numbers) and other issues that spreadsheets bring. I guess what is good for human users is not necessarily good for R. I am glad you figured it out. $\endgroup$ – Drj Jun 7 '16 at 13:04
1
$\begingroup$

If the CSV really comes from Excel, I fear additional columns are lost... If you still have the source file (ie the text file with all the 18.000 columns), I recommend using fread from high-performance package data.table. Something else to learn but definitively worth for high volumes data... Syntax similar to read.csv.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.