Questions tagged [csv]
Comma-Separated Values are a list of plain text values delimited by commas, or a file containing one or more lists in that format.
0
votes
0answers
8 views
How to merge and combine rows with same id (index) in python?
I am new in python and I am working with CSV file with over 10000 rows. In my CSV file, there are many rows with the same id which I would like to merge them in one and also combine their information ...
2
votes
1answer
42 views
Processing csv file with more than 700K rows of data
I have a .csv file (around 400MB in size) which contains 700K rows of structured data.
Table structure is:
...
1
vote
1answer
26 views
Pandas throwing “Error tokenizing data. C error” while loading data sets from URL
I am trying to work on the Titanic competition to get hands on experience with data science & machine learning. I tried to load up the datasets from GitHub but pandas threw the following error:
<...
-1
votes
2answers
22 views
How to calculate occurance of a name per hour basis of monthly csv data using python
I have csv file which have DateTime and application OS name.
DateTime have for 1 month ,per hour basis.
The csv looks as follows,
...
0
votes
0answers
20 views
Complex Value Neural Network with .csv
This is the part of my program in python:
Path of input data and output data for learning.
...
1
vote
1answer
38 views
how to transform a folder of images into csv file?
I have a data set for segmentation project. The dataset have 2 folders containing train folder and annotated_train_data folder, both folders have images. input and label both are images. I want to ...
0
votes
1answer
31 views
Extract or subset hundreds of columns from a data frame
I need to extract many columns from a dataset.
I have a very large csv file with thousands of columns and rows, and I read it into R using:
...
4
votes
1answer
2k views
ValueError: could not convert string to float: '���'
I have a (2M, 23) dimensional numpy array X. It has a dtype of <U26, i.e. unicode string ...
1
vote
3answers
770 views
What is a good way to store processed CSV data to train model in Python?
I have about 100MB of CSV data that is cleaned and used for training in Keras stored as Panda DataFrame. What is a good (simple) way of saving it for fast reads? I don't need to query or load part of ...
1
vote
1answer
72 views
Whole time column in csv file convert into UTC (epoch) using python
I have a datset with time and column. I want to plot a graph with time and value. I tried many methods but didn't come proper graph. Because I have a time series. Then I thought I will convert time ...
0
votes
0answers
31 views
How to plot graph including y axis =value and x axis = datetime with timedelta (2 hrs) reading from csv file using python?
I have dataset import from csv file. Here I want to plot graph with datetime including time delta (two hours). In x axis datetime should be visible like %d/%m/%Y %H:%M. y is value. I wrote the code ...
2
votes
1answer
58 views
day is out of range for month import from csv file in python
Here I have one csv file with different values and date_time. I wrote the code and run it. Then it gave me an error time data '8/6/2018 6:45' does not match format ' %d/%m/%Y %H:%M:%S'
I want to plot ...
0
votes
0answers
16 views
Repeated groups of columns in data analysis
I have a got a medical records for analysis, which have repeated groups of columns holding information about the medicines taken.
The records are initially stored as an Excel table, having multiple ...
1
vote
0answers
129 views
alternatives to pyspark to query a csv file locally (no distributed computation needed)
I am reading a csv file with pyspark to extract some information from it. I am running pyspark locally and I do not need distributed computation. On the contrary I need some plotting library such as ...
2
votes
2answers
955 views
How to transform a folder of images in a csv file
I have a folder with a lot of images that I want to use to bild a classificator using a SVM model in python with sklearn.
I've always used csv file as train/test set with sklearn, how can I make it? (...
1
vote
2answers
580 views
how to take CSV file input in list of tuples
I have a .txt(data.txt) file containing csv data like:
X Class
15.0001 Yes
18.00 NO
17.07 Yes
I need to make a function to return a list of ...
0
votes
1answer
153 views
Find ID for the name in the list from csv file in python [closed]
The CSV file contains:
ID,Name
1,AS
2,er
3,rtf
4,addfs
The list contains the name for example (er,rtf)
I want to find the ID's corresponding to these names mentioned above
How to find the ID's using ...
2
votes
1answer
37 views
Creating a single number from a numpy array - Python [closed]
I am working on a gender classification project. I am extracting the pixels of an image using a Numpy array in Python, similar to the one below:
...
1
vote
2answers
22 views
Any tool to load large csv files quickly on windows 7
I am working with relatively large CSV files (1.5 MB+) and while testing the code, I have to frequently open up the CSV files to read a particular row, validate the data are there and then close the ...
2
votes
5answers
22k views
I got the following error : 'DataFrame' object has no attribute 'data'
I am trying to get the 'data' and the 'target' of the iris setosa database, but I can't. For example, when I load the iris setosa directly from sklearn datasets I get a good result:
Program:
...
0
votes
1answer
8k views
How to store strings in CSV with new line characters?
My question is: what are ways I can store strings in a CSV that contain newline characters (i.e. \n), where each data point is in one line?
Sample data
This is a ...
0
votes
2answers
10k views
Search and Replace in specific column of csv with python
I created a program that search and replaces over an entire csv file but I need to make so it is column specific. Is there a way to only search via a column. Here is my code, I am pretty new to python ...
1
vote
1answer
145 views
Data scraping & NLP?
I'm scraping data from Bing search results for (non-commercial purposes, of course) on Python using BeautifulSoup. I've entered an Indian dessert name, called 'rasmalai' as the word that I am focusing ...
0
votes
1answer
132 views
How can I transpose a high dimensional dataset?
I have a (.csv) file with more than 35,000 rows and 100 columns, where the rows represent the attributes and columns represent the instances. In Excel the maximum number of columns is 16,384. ...
0
votes
1answer
3k views
Orange: Data has no target variable error
I am trying to apply Random Forest algorithm on a data set using Orange. The target variable is not set in the data set. However, I know which column is the target variable.
How can I specify the ...
1
vote
3answers
1k views
Creating data model out of .csv file using Python
I want to create a data model out of a .csv file using Python. I mean to create dependencies, for example the primary key and stuff such that I can check if the new ...
-1
votes
1answer
2k views
I have a csv file with time as the datetime format and want to edit the same column into miliseconds, how do I change that?
Input file is :
time event earthAcceleration.x earthAcceleration.y earthAcceleration.z
2017-11-22T09:20:13.944 motion -0.006379993 -0.001029071 -0.010781396
2017-11-22T09:20:13.954 motion 0....
2
votes
2answers
1k views
How to deal with large data sets
So i'm very new to this, forgive my silly questions.
I've got some data I need to analyse, I would usually just use excel it doesnt seem to be able to do the job.
I have .csv files containing data ...
0
votes
2answers
8k views
Python inserts newline by writing to csv
I am trying to scrape http://www.the-numbers.com/movie/budgets/all
but when I write the table into a csv file, there is an additional line with the counter index written in between each movie row... ...
0
votes
1answer
12k views
Converting Json file to Dataframe Python
I have a json file which has multiple events, each event starts with EventVersion Key. The data looks similar to the following synthesized data.
...
1
vote
1answer
418 views
R: reading big data files in R
In R, there are libraries which help speed up the process of reading huge data files. Examples of these libraries include sqldf and ff.
What would be the disadvantage of using these packages to read ...
2
votes
2answers
434 views
Breaking down a column in Pandas into a separate CSV for display in Tableau
My data is coming from a CSV, which should be visualized in Tableau.
However, the data contains the column category_list, which consists of values separated by a ...
0
votes
1answer
2k views
3
votes
1answer
709 views
ML algorithm for determining CSV file header names based on content
I have a large amount of CSV files, an example of which (for job titles) is listed below. The data is noisy (there are misspellings, difference in capitalisation, missing values, and they are not well-...
1
vote
1answer
2k views
Output data from scikit learn logistic regression
I have run a logistic regression using scikit learn in python. I know want to output the results to put into a csv and then load into Tableau. To do that I need to combine the y_test, y_actual, and ...
0
votes
2answers
5k views
Python - AttributeError: type object 'DataFrame' has no attribute 'read_csv'
I'm trying to create some charts using Python. I've this dataset in a CSV file:
Banana Water Rice
Rice Water
Bread Banana Juice
And I've this code:
<...
1
vote
3answers
6k views
importing csv data in python
I have a csv file with around 130 columns and 6000 rows
what is the best way to import them into python, so that I can later use them in a classification algorithm(columns are the labels and rows are ...
6
votes
3answers
8k views
Reading CSVs with new lines in fields with Spark
I was trying to load the below weblogic domain log (application error log) into Spark dataframe. I created a RDD and converted the RDD into dataframe. I was able to load the data successfully for the ...
1
vote
2answers
128 views
Reading a wide dataset in R
I originally had a wide CSV dataset of about 18000 columns(and about 80 rows) that I am trying to read in R. It was stored in an Excel sheet,which unfortunately has a limit of only 16384 columns. ...
1
vote
3answers
70 views
Creating queries on the fly and general manipulation for dataset of half a million data records
What would one use for manipulating data of the kind below ?
a) Data is bio-markers of different globs
GlobA 3 4 5 ....
GlobB 2 1 1 ....
GlobC 3 2 1 ....
b) Manipulations are queries like:
- show ...
5
votes
2answers
334 views
What kind of statistical analyses can I do with my data?
I'm trying to analyze human intentions in clicking google ad word keywords.
In this dataset I have the usual adword details, for example
...
0
votes
2answers
86 views
Reading a CSV using R [closed]
I have a CSV file that, by all appearances, is totally normal except that each line ends with ,^M. Not sure if that has anything to do with my issue or not, but I ...
3
votes
5answers
2k views
Quick way to visually explore data?
I have a very large csv file which Apple Numbers won't open. I can open it with TextEdit, but each row is so long it forms multiple lines in the document and makes the document difficult to understand....
0
votes
1answer
486 views
Batch convert json links to csv [closed]
I have a batch of links with the following naming convention:
https://api.cilabs.net/v1/conferences/ws15/info/attendees?page=1; and the page number incrementing.
I would like to extract the data ...
4
votes
2answers
697 views
Recommendations for storing time series data
As part of my thesis I've done some experiments that have resulted in a reasonable amount of time-series data (motion-capture + eye movements). I have a way of storing and organizing all of this data, ...
2
votes
3answers
1k views
Merging repeating data cells in csv
I have a CSV file with around 1 Million rows.
Let say its have details like
...
3
votes
3answers
13k views
How to read file from user in Shiny and assign it to a variable in global.r?
I want to read a csv file as input from user in Shiny and assign it to a variable in global.r file.The code I have in ui.R is
...
1
vote
0answers
402 views
How to create separate nodes for each column in a spreadsheet using Neo4J?
I have an excel file containing lot of columns. I want to create a graph database in Neo4J with all these columns as a separate node so that I can establish relationship between them and play around ...
6
votes
3answers
1k views
Summarize and visualize a CSV in Java/Scala?
I would like to summarize (as in R) the contents of a CSV (possibly after loading it, or storing it somewhere, that's not a problem). The summary should contain the quartiles, mean, median, min and ...
