Questions tagged [csv]

Comma-Separated Values are a list of plain text values delimited by commas, or a file containing one or more lists in that format.

Filter by
Sorted by
Tagged with
-1 votes
0 answers
9 views

Need malformed csv files

Is there a resource one can use to find malformed cvs files? I tried creating my own but I can only come up with so many errors. It is for a project I am working on to read and visualise malformed csv ...
StudentP's user avatar
-1 votes
0 answers
10 views

Delimited File Editor That's NOT Excel

I'm looking for Excel alternatives that DO NOT make assumptions about cell contents when opening a CSV or a similar delimited file. The text import wizard in Excel is not a viable solution: I don't ...
Tavaro Evanis's user avatar
1 vote
2 answers
102 views

Merging Turbine and Forecast Data Diagonally

I am trying to combine my main dataset, which is hourly data collected from a turbine, with forecast data. The forecast data is collected by calling an API every hour to get a new .csv file which is ...
Eli Orians's user avatar
0 votes
1 answer
23 views

What are these 'code' columns in this csv and what does Pandas pivot method do with it?

In a csv discussed in the book HoML (hands on machine learning), what do the values in capital letters represent? Are these some sort of code? Also, it seems the pandas.pivot() methods does something ...
Yorick Dantuma's user avatar
0 votes
1 answer
38 views

Writing python dictionary to a csv [closed]

I have a dictionary called dict_object dict_object=class_object.__dict__ print(type(dict_object)) print(dict_object) This code gave the following output: ...
Vasundhhara Singh Katoch's user avatar
4 votes
1 answer
66 views

Suitability of column store formats like Parquet for table data

I'm new to the data analytics world, but perhaps my question could help others, we all start somewhere. We have a process that extracts data from an SQL database and stores it in temp files as CSV ...
mogoman's user avatar
  • 143
0 votes
0 answers
698 views

How to read CSV File into Vector Store

I have a CSV file, and I am using langchain to read it into the vector store FAISS. My question is, since I have a CSV file, is RecursiveTextSplitter required? Put differently, consider the following ...
Karl 17302's user avatar
0 votes
2 answers
375 views

What is source_column argument in csv loader?

In this tutorial, what is the purpose of source_column argument? Does it act like a primary key in Databases? Thanks in advance. ...
Karl 17302's user avatar
0 votes
0 answers
44 views

How to train Keras Sequential model in batches?

newbie here. I have a large dataset in many local CSV files. It's impossible to fit it in memory within budget. How can I train a Sequential model in batches? I've found ...
Putnik's user avatar
  • 105
0 votes
0 answers
23 views

Need help in Webscraping T20 World Cup 2022 data

I have a list of links of match scorecards and i have stored the respective html content of each link in a list. I need to get the csv files I am attaching as the output. Any help would be ...
karan shah's user avatar
0 votes
0 answers
268 views

Finetune LLM model on tabular data

Is it possible or even recommended to finetune LLMs such as llama2 on tabular data? I have a csv with historical gold buy prices. ...
fpena06's user avatar
  • 101
1 vote
2 answers
154 views

Neural network not learning at all

I am training a MLP on a tabular dataset, the pendigits dataset. Problem is that training loss and accuracy are more or less stable, while validation and test loss and accuracy are completely constant....
CasellaJr's user avatar
  • 229
0 votes
1 answer
43 views

How to Manipulate a Dataset

I have a raw dataset of Images I got from Kaggle, It has been classified already, but I want to randomly split the information in a 80:20 ratio between train and test, problem is since all information ...
Unreal Capacity's user avatar
1 vote
0 answers
29 views

Python train convolutional on numerical values shape issue

I want to train a convolutional neural network autoencoder on a csv file which contains values pixel neighborhood position of an original image of 1024x1024. When I try to train it, I have the ...
user979974's user avatar
1 vote
1 answer
1k views

pands doesn't recognize missing values in CSV [closed]

In the CSV there's missing values in many columns but the code shows there's no missing values. how can i fix this?
070701's user avatar
  • 13
6 votes
4 answers
269 views

How can I observe my CSV files better? [closed]

I'm running a lot of experiments that give their output as CSV files. An experiment might be running for hours, with a new line being added to the CSV every 10 seconds. Right now I'm opening these CSV ...
Ram Rachum's user avatar
2 votes
4 answers
779 views

Algorithm to determine whether the first row in CSV is likely to be a header row or a data row

I have a fairly simple problem. I am trying to determine whether the first row in CSV is likely to be a header row or a data row. Looking at single column, the problem can be simplified to: I have a ...
David Ferris's user avatar
0 votes
1 answer
104 views

read_csv, non-printing ascii delimiters, and multi index

I have some text data which uses the ASCII data characters 0x1e "group separator" and 0x1d "record separator"...
rob's user avatar
  • 101
0 votes
2 answers
830 views

How to Insert a Row before Dataframe in Pandas

I have a csv file that is used as a pandas dataframe, now I only need to insert a dummy row before the dataframe starts like in the screenshot denoted as "Label". How can I do that?
Mohamed Abduljawad's user avatar
0 votes
1 answer
338 views

Pandas altering numerical data on import

I have a csv file where a column of data is a 19 digit number of which the last three digits are always 163, this is specific to identify that this is the correct id. If i open the file in sublime, ...
iFunction's user avatar
  • 111
0 votes
1 answer
2k views

How to fix ValueError: The truth value of a DataFrame is ambiguous. Use a.empty, a.bool(), a.item(), a.any() or a.all()

I am using pytest to test a function that reads a csv into a dictionary of lists. In the test function it writes a sample dictionary to a csv then calls the function to read it back out. To accomplish ...
Michael Wallentine's user avatar
2 votes
1 answer
589 views

The single CSV created by combining a large number of CSV files is too large to process. What options do I have?

The dataset I am currently working on has more than 100 csv files, with each of size more than 250MB. These are files containing time series data captured from different locations and all the files ...
odd_wolf's user avatar
0 votes
1 answer
661 views

multiple images inside one large CSV file

I'm very new to data science, and was admiring how people had made these massive open-source datasets, on places like kaggle. I noticed that all of the datasets where all in CSV format. I have lots of ...
Tom_nerd's user avatar
0 votes
1 answer
465 views

Online Generating Dendrograms with imported CSV file

Is there any website out there that performs hierarchical clustering on an imported file like CSV to generate dendrograms or similarity matrix? Free or paid does not matter. So far I found heatmapper....
mos2g's user avatar
  • 1
0 votes
1 answer
3k views

Error: `raise ValueError( ValueError: Missing column provided to 'parse_dates': 'Date' [closed]

I am using a .csv with two columns. The first has dates and the second has temperatures. I would like to plot it with dates on the x-axis and temperatures on the y-...
just_learning's user avatar
1 vote
1 answer
28 views

Defining a function in Pandas [closed]

Hello Data Scientists, Disclaimer: I am a Beginner learning Python and at the same time just trying out a lot of things. I am doing some data extraction on a file with with weekly deaths in ...
marlonboettger's user avatar
0 votes
0 answers
41 views

Save CSVs as hyper file

I have three separate CSV files (Points.csv, PointsSynonym.csv, and LocalDataPoints.csv). I made these files after extracting data from shape files and .gpkg files. I would like to save them all a ...
x89's user avatar
  • 191
1 vote
3 answers
1k views

Efficiently modify a large csv file in Pandas

I have a csv file and would like to do the following modification on it: ...
Jost A.'s user avatar
  • 11
-1 votes
3 answers
10k views

Convert .data file to .csv [closed]

I'm using a data called 'adults.data', I need to work with that data as a '.csv' file. Can anyone guide me on how can I change the format? I tried opening the file in excel and then save it as csv, ...
Parsa Khavarinejad's user avatar
2 votes
1 answer
68 views

How to work with hundreds of CSVs with millions of rows in each?

So I'm doing a project on the COVID-19 Tweets dataset from the IEEE port and I plan to analyse the tweets over the time period from March 2020 till date. The thing is there's more than 300 CSVs for ...
rick458's user avatar
  • 21
2 votes
0 answers
34 views

Is there a corpus of toy datasets specifically designed for finding bugs in data science software?

I'm looking for a corpus of toy tabular datasets that can be used to test data profiling, machine learning, data manipulation, etc. software. Some example attributes: Strange column names (empty ...
Shoeboxam's user avatar
2 votes
0 answers
37 views

Is high school GPA included in IPEDs NCES databases?

I’m currently developing a computer tool which will sort through the US college and university NCES data provided in the IPEDs website to match users with potential universities. NCES shows university ...
Grace Hopper's user avatar
0 votes
1 answer
896 views

How to specify different columns stacked vertically within CSV using pandas?

I have a CSV file that looks like below with columns labeled below data, and I'm have trouble trying to retrieve said data with pandas. I'm only able to address the columns at the top with the ...
Bloomfield's user avatar
1 vote
1 answer
1k views

How do I get Multiple CSV files (csv file names will be column names) from a folder to a pandas dataframe?

I think the title is enough for what is my problem here. I have 100 tickers in a folder. I got all csv files to one list. but I wanna see the tickers name which these are csv file names. what should ...
plastik's user avatar
  • 11
2 votes
2 answers
6k views

Pandas loads a csv file incorrectly, but without throwing an error [closed]

So I tried loading some data through pandas to practice manipulating it, but I ran into a slight problem. Basically, the pandas load the data improperly. And it does so consistently. Let me show you ...
Abubakar Popoola's user avatar
0 votes
1 answer
131 views

How to filter in pandas based on string value? [closed]

I want to filter a pandas dataframe based on the id of the row. The id is a sequence of patient names. For example, I have columns in this format: ID | A | B And ...
zoete aardappel's user avatar
6 votes
3 answers
349 views

How to get K most different rows in csv?

We have boring CSV with 10000 rows of ages (float), titles (enum/int), scores (float). How to select 1000 most different rows? I look for a general solution that ...
Blender's user avatar
  • 161
2 votes
1 answer
153 views

Recommendations for Data Visualization solution for large CSV file on a non-profit website

I would like your advice and recommendations for a non-profit that is trying to turn a large CSV file into meaningful information. I am part of a non-profit that would like to turn a 23,000 row x 23 ...
Brendan Spaar's user avatar
0 votes
1 answer
7k views

unhashable type: 'numpy.ndarray' [closed]

...
LeeAnn Capistran's user avatar
0 votes
1 answer
71 views

Scrapping data from a CSV file [closed]

So, I'm a rookie at web scrapping and Python in general, and I'm having quite a hard time learning it. I'm trying to get some data from a csv file, but so far I could only split lines and columns. I ...
HStoltz's user avatar
0 votes
1 answer
56 views

CSVs: how to output missing data to make processing easier?

If supplying data (that can be either string or numeric) via CSVs, what's a good strategy for marking that a value is missing? Some non-empty sentinel, like NA ...
Michal Charemza's user avatar
0 votes
4 answers
180 views

How to view entire 5788 rows and 7 columns.?

I want to see entire 5788 rows and 7 columns of my CSV data. I used below code and it is giving little data by stating [5788 rows x 7 columns] but showing only 10 rows and 5 columns in python prompt. ...
ADS KUL's user avatar
  • 13
0 votes
2 answers
2k views

Getting some data from a csv file into a list of tuples using Python

I have a csv file of almost 2 million rows and about 20 columns. I am interested in about 40000 of these rows, and in 5 of the columns 'squat', 'bench', 'deadlift', 'bodyweight', 'total'. I'd like to ...
Pel de Pinda's user avatar
1 vote
1 answer
2k views

How do I change the location of Google BigQuery

So, I'm trying to use Google BigQuery for the first time for a project of mine, and I'm a bit confused. The documentation isn't helping much, and it looks like all the Google employees are gone thanks ...
nick012000's user avatar
1 vote
0 answers
661 views

Dask error when reading data from a large zip file

I have a .zip folder with a lot of .csv files which I read into a Dask DataFrame like this: ...
Vlad's user avatar
  • 11
1 vote
1 answer
96 views

Representing 2 dimensional data for facial recognition

My goal is to train a neural network to recognize faces based on a list of landmark points generated by Google Firebase ML-Kit. Since I just started familiarizing myself with ML, I only want my model ...
oneManArmin's user avatar
1 vote
2 answers
118 views

I am working on a pose dataset and want to convert this data into CSV format. How to convert this data into a single row of data

This is the data for one pose. I have a lot of data like this which i want to convert to CSV to apply machine learning algorithm. I am assuming I have to convert this data into a single row of comma ...
Saumya Saxena's user avatar
1 vote
0 answers
16 views

Built strong base for Unsupervised Learning [closed]

I’m am new into machine learning, recently I have put a task upon my shoulders to Detect Outliers in Dataset. The anomaly detection should be done using Unsupervised learning and preferably use ...
E199504's user avatar
  • 605
0 votes
1 answer
283 views

How to split a CSV into a dataframe without newline

Currently I'm using pyspark to make my df from a csv. However when I take the data in, it puts each element on a new line. Is there any way to keep the elements separate, and keep them on the same ...
Kelo's user avatar
  • 109
1 vote
2 answers
1k views

Should I use MongoDB instead of storing data in CSV in python?

I am currently storing data crawled from multiple websites having same but still different structure so every crawler is saving data in separate csv. I am planning to store the data using MongoDB ...
Rishabh Sharma's user avatar