I have a csv file with around 130 columns and 6000 rows
what is the best way to import them into python, so that I can later use them in a classification algorithm(columns are the labels and rows are individual samples)
$\begingroup$
$\endgroup$
2
-
$\begingroup$ Use one of the import functions? Your computer should be able to handle that. $\endgroup$– ABCDSep 15, 2016 at 17:11
-
$\begingroup$ See also: How do I read and write CSV files with Python? $\endgroup$– Martin ThomaMar 18, 2017 at 11:00
Add a comment
|
3 Answers
$\begingroup$
$\endgroup$
Use pandas library:
import pandas as pd
pd.read_csv('foo.csv')
Pandas identify the headers automatically and is a great tool for data wrangling.
10 Minutes intro to pandas
$\begingroup$
$\endgroup$
2
For small data, I think pandas.read_csv is the way to go.
For "medium" data, I recommend dask.read_csv
And for big data, I recommend spark.read.csv
-
$\begingroup$ Agreed. On my laptop (8G ram, i5 processor), reading in CSV's up to around 1GB isn't much of a problem using pandas.read_csv. Keep in mind also that read_csv has the handy feature of being able to read gzipped CSV's. $\endgroup$ Sep 15, 2016 at 17:43
-
$\begingroup$
$\endgroup$
2
You can also SFrame. First install graph lab. Then:
import graphlab as gl
data = gl.SFrame.read_csv('data.csv')
If you're 'hardcore' you can use python's basic csv reader, but then you will have to write loops to manage the data yourself, so why bother reinvent the wheel, just use pandas or Frame.
-
-
$\begingroup$ SFrame is technically better, but it's use is restricted and limited documentation. Pandas is free and has more community support. $\endgroup$ Sep 20, 2016 at 2:02