Questions tagged [csv]
Comma-Separated Values are a list of plain text values delimited by commas, or a file containing one or more lists in that format.
80
questions
1
vote
0answers
9 views
Is high school GPA included in IPEDs NCES databases?
I’m currently developing a computer tool which will sort through the US college and university NCES data provided in the IPEDs website to match users with potential universities. NCES shows university ...
0
votes
1answer
45 views
How to specify different columns stacked vertically within CSV using pandas?
I have a CSV file that looks like below with columns labeled below data, and I'm have trouble trying to retrieve said data with pandas. I'm only able to address the columns at the top with the ...
0
votes
1answer
432 views
How do I get Multiple CSV files (csv file names will be column names) from a folder to a pandas dataframe?
I think the title is enough for what is my problem here. I have 100 tickers in a folder. I got all csv files to one list. but I wanna see the tickers name which these are csv file names.
what should ...
1
vote
2answers
204 views
Pandas loads a csv file incorrectly, but without throwing an error [closed]
So I tried loading some data through pandas to practice manipulating it, but I ran into a slight problem. Basically, the pandas loads the data improperly. And it does so consistently. Let me show you ...
0
votes
0answers
132 views
Unexpected arguments error appearing on the command line when running mapreduce job (MRjob) using python
I am fairly new to this process. I am trying to run a simple map-reduce job using python 3.8 with a csv on a local Hadoop cluster (Hadoop version 3.2.1). I am currently running it on Windows 10 (64-...
0
votes
1answer
27 views
How to filter in pandas based on string value? [closed]
I want to filter a pandas dataframe based on the id of the row. The id is a sequence of patient names. For example, I have columns in this format:
ID | A | B
And ...
6
votes
3answers
229 views
How to get K most different rows in csv?
We have boring CSV with 10000 rows of ages (float), titles (enum/int), scores (float). How to select 1000 most different rows? I look for a general solution that ...
2
votes
1answer
37 views
Recommendations for Data Visualization solution for large CSV file on a non-profit website
I would like your advice and recommendations for a non-profit that is trying to turn a large CSV file into meaningful information.
I am part of a non-profit that would like to turn a 23,000 row x 23 ...
0
votes
1answer
3k views
0
votes
1answer
24 views
Scrapping data from a CSV file [closed]
So, I'm a rookie at web scrapping and Python in general, and I'm having quite a hard time learning it. I'm trying to get some data from a csv file, but so far I could only split lines and columns. I ...
0
votes
1answer
33 views
CSVs: how to output missing data to make processing easier?
If supplying data (that can be either string or numeric) via CSVs, what's a good strategy for marking that a value is missing?
Some non-empty sentinel, like NA
...
0
votes
0answers
19 views
What's the best way to store and then call 3 Million+ records for training?
Good afternoon all,
I am working on a Deep Learning project where I'm proceeding in batches (rather than a continuous pipeline).
Essentially I've built a function that can convert raw data into ...
0
votes
0answers
28 views
0
votes
4answers
75 views
How to view entire 5788 rows and 7 columns.?
I want to see entire 5788 rows and 7 columns of my CSV data. I used below code and it is giving little data by stating [5788 rows x 7 columns] but showing only 10 rows and 5 columns in python prompt. ...
1
vote
2answers
611 views
Getting some data from a csv file into a list of tuples using Python
I have a csv file of almost 2 million rows and about 20 columns. I am interested in about 40000 of these rows, and in 5 of the columns 'squat', 'bench', 'deadlift', 'bodyweight', 'total'. I'd like to ...
1
vote
1answer
91 views
How do I change the location of Google BigQuery
So, I'm trying to use Google BigQuery for the first time for a project of mine, and I'm a bit confused. The documentation isn't helping much, and it looks like all the Google employees are gone thanks ...
1
vote
0answers
275 views
Dask error when reading data from a large zip file
I have a .zip folder with a lot of .csv files which I read into a Dask DataFrame like this:
...
1
vote
1answer
40 views
Representing 2 dimensional data for facial recognition
My goal is to train a neural network to recognize faces based on a list of landmark points generated by Google Firebase ML-Kit. Since I just started familiarizing myself with ML, I only want my model ...
1
vote
2answers
39 views
I am working on a pose dataset and want to convert this data into CSV format. How to convert this data into a single row of data
This is the data for one pose. I have a lot of data like this which i want to convert to CSV to apply machine learning algorithm.
I am assuming I have to convert this data into a single row of comma ...
1
vote
0answers
15 views
Built strong base for Unsupervised Learning [closed]
I’m am new into machine learning, recently I have put a task upon my shoulders to Detect Outliers in Dataset.
The anomaly detection should be done using Unsupervised learning and preferably use ...
0
votes
1answer
174 views
How to split a CSV into a dataframe without newline
Currently I'm using pyspark to make my df from a csv. However when I take the data in, it puts each element on a new line. Is there any way to keep the elements separate, and keep them on the same ...
1
vote
2answers
385 views
Should I use MongoDB instead of storing data in CSV in python?
I am currently storing data crawled from multiple websites having same but still different structure so every crawler is saving data in separate csv. I am planning to store the data using MongoDB ...
0
votes
1answer
35 views
Deep learning model gives random results
First I am new to machine learning if it is an obvious question, I am sorry.
...
0
votes
1answer
246 views
How can I override the values of 1 column in a big dataframe using a small second dataframe? [closed]
I have a Pandas dataframe (donations_df) that contains thousands of donations. Each donation row has many columns, but the two key ones for my question are:
A <...
1
vote
1answer
14k views
How to save API data into CSV format?
Is there an easy way to convert API data with python commands and output the results in CSV format?
Here's what I'm doing now:
...
0
votes
1answer
81 views
How do you see the element of a csv table with many columns (>30) which the names of its columns is more than 10 character in pandas?
How do you see in pandas the element of a csv table with many columns (>25) which the names of its columns is more than 10 character?
I have 5000 rows and 32 columns and the label of some columns ...
1
vote
2answers
2k views
How to create a parquet file from a query to a mysql table
Updating a legacy ~ETL; on it's base it exports some tables of the prod DB to s3, the export contains a query. The export process generates a csv file using the following logic:
...
0
votes
1answer
3k views
Pandas - how to make further sub-groups after groupby( ) and get_group( )
Sample data in .csv format
...
4
votes
1answer
1k views
Any one read the over 100gb csv file and successfully concatenation?
I have been searching for the deal with large CSV file read method
Its over 100gb and need to know how deal with the chunk file processing
and make concatenation faster
...
2
votes
2answers
84 views
File format where column names are repeated on each row
I have received a dataset in text file with the following format
...
2
votes
1answer
3k views
Getting stock data in a discipline manner from Yahoo finance
I used the below code for downloading stock data from yahoo finance:-
...
0
votes
1answer
8k views
np.loadtxt function showing error, could not convert string to float: '“Date”'
This is my stock market csv data:
...
1
vote
2answers
41 views
Removing # from header
I have a $(418,2)$ matrix and I want to convert it to csv. so I write:
np.savetxt('titanic1.csv', Sol, fmt='%.2f', delimiter=",",header="PassengerId,Survived")
...
2
votes
1answer
5k views
File converter: from CSV to HDF5
Can anyone recommend any command line tool for converting large CSV file into HDF5 format?
3
votes
1answer
185 views
Processing csv file with more than 700K rows of data
I have a .csv file (around 400MB in size) which contains 700K rows of structured data.
Table structure is:
...
3
votes
1answer
6k views
Pandas throwing “Error tokenizing data. C error” while loading data sets from URL
I am trying to work on the Titanic competition to get hands on experience with data science & machine learning. I tried to load up the datasets from GitHub but pandas threw the following error:
<...
-2
votes
2answers
699 views
How to calculate occurance of a name per hour basis of monthly csv data using python
I have csv file which have DateTime and application OS name.
DateTime have for 1 month ,per hour basis.
The csv looks as follows,
...
2
votes
1answer
13k views
how to transform a folder of images into csv file?
I have a data set for segmentation project. The dataset have 2 folders containing train folder and annotated_train_data folder, both folders have images. input and label both are images. I want to ...
1
vote
1answer
128 views
Extract or subset hundreds of columns from a data frame
I need to extract many columns from a dataset.
I have a very large csv file with thousands of columns and rows, and I read it into R using:
...
8
votes
2answers
106k views
ValueError: could not convert string to float: '���'
I have a (2M, 23) dimensional numpy array X. It has a dtype of <U26, i.e. unicode string ...
1
vote
3answers
873 views
What is a good way to store processed CSV data to train model in Python?
I have about 100MB of CSV data that is cleaned and used for training in Keras stored as Panda DataFrame. What is a good (simple) way of saving it for fast reads? I don't need to query or load part of ...
1
vote
1answer
627 views
Whole time column in csv file convert into UTC (epoch) using python
I have a datset with time and column. I want to plot a graph with time and value. I tried many methods but didn't come proper graph. Because I have a time series. Then I thought I will convert time ...
2
votes
1answer
2k views
day is out of range for month import from csv file in python
Here I have one csv file with different values and date_time. I wrote the code and run it. Then it gave me an error time data '8/6/2018 6:45' does not match format ' %d/%m/%Y %H:%M:%S'
I want to plot ...
2
votes
1answer
196 views
alternatives to pyspark to query a csv file locally (no distributed computation needed)
I am reading a csv file with pyspark to extract some information from it. I am running pyspark locally and I do not need distributed computation. On the contrary I need some plotting library such as ...
3
votes
2answers
7k views
How to transform a folder of images in a csv file
I have a folder with a lot of images that I want to use to bild a classificator using a SVM model in python with sklearn.
I've always used csv file as train/test set with sklearn, how can I make it? (...
1
vote
2answers
3k views
how to take CSV file input in list of tuples
I have a .txt(data.txt) file containing csv data like:
X Class
15.0001 Yes
18.00 NO
17.07 Yes
I need to make a function to return a list of ...
0
votes
1answer
1k views
Find ID for the name in the list from csv file in python [closed]
The CSV file contains:
ID,Name
1,AS
2,er
3,rtf
4,addfs
The list contains the name for example (er,rtf)
I want to find the ID's corresponding to these names mentioned above
How to find the ID's using ...
2
votes
1answer
62 views
Creating a single number from a numpy array - Python [closed]
I am working on a gender classification project. I am extracting the pixels of an image using a Numpy array in Python, similar to the one below:
...
1
vote
2answers
95 views
Any tool to load large csv files quickly on windows 7
I am working with relatively large CSV files (1.5 MB+) and while testing the code, I have to frequently open up the CSV files to read a particular row, validate the data are there and then close the ...
8
votes
6answers
213k views
I got the following error : 'DataFrame' object has no attribute 'data'
I am trying to get the 'data' and the 'target' of the iris setosa database, but I can't. For example, when I load the iris setosa directly from sklearn datasets I get a good result:
Program:
...
