Questions tagged [dataframe]

A data frame is a tabular data structure. Usually, it contains data where rows are observations and columns are variables of various types. While "data frame" or "dataframe" is the term used for this concept in several languages (R, Apache Spark, deedle, Maple, the pandas library in Python and the DataFrames library in Julia), "table" is the term used in MATLAB and SQL.

Filter by
Sorted by
Tagged with
0
votes
1answer
23 views

Python: How to plot time interval from a Dataframe in Pandas

I have the a dataframe(df) which has the data of a Job being executed at different time intervals. It includes the following details about the execution of a job: ...
1
vote
0answers
18 views

Python: How to convert Pandas Dataframe Row Values to individual columns? [migrated]

I have the following dataframe which has the data of different jobs start and end time at different intervals. A small part of dataframe is shown below. Dataframe(df): ...
-1
votes
1answer
20 views

Multiplying a dataframe by a larger one [closed]

I have two dataframes df1 and df2 with the same columns but not the same row number. I want to multiply them element-wise such ...
0
votes
2answers
25 views

Sort Pandas Dataframe per column

I have a dataset with age + another 14 variables. I have created 13 bins representing different age groups like so: ...
0
votes
0answers
16 views

How i match with SequenceMatcher ratio two dataframe

I use the SequenceMatcher ratio to match two dataframe with the best ratio. I want to check first if the score A and AA is good then check if the score between B is BB is good then if the score ...
1
vote
1answer
33 views

Replace values in dataframe with another dataframes values based on condition [closed]

I have 2 dataframes with the same columns names and rows number but the elements' values are different. I need to change the value of each element in the first dataframe to 1 if its value in the the ...
0
votes
1answer
41 views

Python: convert variables into correct format for DataFrame

I have 3 variables that I would like to use to build my dataset but since they are in a weird shape/format, I had no success so far. I'm quite new to this and really appreciate any help!! The 3 ...
0
votes
1answer
25 views

Access keys of pandas dataframe when using groupby

I have the following database: And I would like to know how many times a combination of BirthDate and Zipcode is repeated throughout the data table: Now, my question is: How can I access the keys of ...
2
votes
1answer
28 views

How to fill df with data from another df - Python [closed]

I am trying to fill df with values form a different one. The first df is: The second df is: I want to add the column to first df having values from the second one to receive output: number trans ...
0
votes
1answer
36 views

How do I get Multiple CSV files (csv file names will be column names) from a folder to a pandas dataframe?

I think the title is enough for what is my problem here. I have 100 tickers in a folder. I got all csv files to one list. but I wanna see the tickers name which these are csv file names. what should ...
0
votes
0answers
23 views

ValueError: Could not guess the value column! Please hand it to the function as an argument

I'm using the tsfresh library to extract features on my data but I keep getting an error: ...
0
votes
0answers
16 views

How to use pivot_table on a dataframe? How to handle exceptions.UFuncTypeError?

I tried to use the pivot_table function on a dataframe. This was working at the beginning now I get all the time errors, and don't really get why. ...
0
votes
1answer
18 views

Is it ok to fill a pandas dataframe with NaN values?

Is it correct to fill a pandas dataframe with NaN values? In specific: if I have a dataframe with a user name and his age is it ok to fill the age column with int and NaN values. ...
1
vote
1answer
34 views

Replace Missing Values with Most Frequent number under Condition

I'm trying to replace missing values of column "Age" but under condition of other columns on this data Titanic - Machine Learning from Disaster ...
1
vote
0answers
32 views

Using numpy.ndarray in machine learning sklearn.preprocessing model

I'm having a problem storing and using an array on one model that I'm building in sklearn, Here is what I'm doing: I'm converting an image to numpy and storing as numpy.ndarray in my dataframe (there ...
0
votes
0answers
164 views

Excel error may be caused by pandas writing or big data? advise needed

I am reading multiple xml files extracting some data then forming a pandas Dataframe with my data. These are the main steps that I do: open an xml file extract some elements create a pandas dataframe ...
2
votes
1answer
26 views

How to fill NaN based on the previous row that contains a specific value [closed]

I would like to fill the missing data using the value from the previous trading day of the same stock. In this example, the AAPL stock should be 100. I've tried with ...
0
votes
0answers
33 views

isnull().sum() returning incorrect figure

I am doing a prediction using RandomForestRegressor. Below is the code for test data: ...
1
vote
1answer
16 views

How to cluster label (in a multilabel classification problem) which mostly appear together in a class

To cluster label (in a multilabel classification problem) which mostly appear together in a dataframe? For example, I have this dataframe: ...
1
vote
1answer
20 views

Merging 2 pandas dataframes on key with duplicates but do not want it to result in two lines

I am trying to merge a "roster" where there are supposed to be several positions with one person per office. However, I have come across two instances where there are two people in one ...
0
votes
0answers
17 views

replacing last duplicate values in pandas

I have a dataframe ,that looks like this | **site** | **Active**| 0 |: deals| Active:| 1 |: deals | InActive:| 2 | : deals | Active:| 3 |:...
0
votes
1answer
17 views

replace values based on Number of duplicate rows are occured

I have a dataframe ,that looks like this site Active 0 deals Active 1 deals Active 2 deals Active 3 discount Active 4 discount Active i ...
0
votes
0answers
9 views

How should I add reference key column to output after modeling is Complete?

0 I have created a csv of data with the following columns: (1) app_key (2) churn, (3)tenure https://i.stack.imgur.com/NAlFF.png I have performed the following code in order to drop app_key and churn <...
-1
votes
1answer
28 views

How can I get all column values from a panda dataframe and save them into one string [closed]

this question has been ask couple of times but none of the answer is working for that what I want. For example I have this excel Dokument: ...
0
votes
1answer
28 views

List value in Pandas DataFrame column makes analysis harder

Should I move to database? I have a list of courses data in JSON format that looks like this: ...
0
votes
1answer
241 views
0
votes
1answer
93 views

How to convert DataFrame column to Rows in Python?

I have the following dataset in df_1 which I want to convert into the format of df_2. In df_2...
0
votes
1answer
42 views

How to add Error Bar to Matplotlib line plot?

I have the following dataset which I use to plot a line plot. The plot is obtained as the mean of values obtained from the data. I want to add error bars to this ...
1
vote
0answers
22 views

How to Ensemble Machine Learning Models?

I want to Ensemble my predictions for the StratifiedKFold of LGBM and XGBoost into another LGBM Model. I had written the following code which works when the data set has an ID_COL, but in this data ...
0
votes
0answers
8 views

Get latest Item by Date for a Recommender System

I am building a Recommender System where I am giving the User 3 Recommendations depending upon for the Webpage he is on. Let's say My model gives me 3 Recommendations from 2020, 2019, 2015. I would ...
0
votes
0answers
572 views

How to fix IndexError: Too many levels: Index has only 1 level, not 2 in Python

I have written the following code that has a function model_data to perform a particular set of tasks. I have to pass the list of ...
0
votes
3answers
2k views

How to combine and separate test and train data for data cleaning?

I am working on an ML model in which I have been provided the data in 2 files test.csv and train.csv. I want to perform data ...
0
votes
1answer
76 views

Comparing 2 columns from separate dataframes and copy some row values from one df to another if column value matches in pandas

Im relatively new to pandas andpython . I have a simple problem but I can not find the solution . I have 2 dataframes DF1 and DF2 as shown in image Both dataframes have ID coluumns. I wannt to check ...
0
votes
1answer
165 views

Calculate all days for each years between two dates in a pandas dataframe

I'm new in python and coding. I'm doing a university project exercise. Last question is : " For each year, compute the total amount of loans. Each loan that has planned expiration time and ...
0
votes
0answers
58 views

Numpy array value error

This is my code to train a sigmoid neuron for Iris dataset . ...
0
votes
2answers
61 views

sklearn KNN fit throws out error : value too large for dtype('float64')

I have cleaned the data from nan values and infinite values, the only feature which has a large float is the column 8 (it's a sum) I have no Idea how to fix this last error, I tried all previous ...
0
votes
1answer
69 views

Bootstrap clarification for datasets with multiple columns

How do I bootstrap a Dataset/DataFrame with multiple continuous and categorical columns? For eg: Say I'm trying to bootstrap the colour distribution of M&M's and I have 50 bags (samples) each with ...
0
votes
1answer
31 views

RAM crashed for XML to DataFrame conversion function

I have created the following function which converts an XML File to a DataFrame. This function works good for files smaller than 1 GB, for anything greater than that the RAM(13GB Google Colab RAM) ...
0
votes
1answer
24 views

Time Series Plot for floating values

I have a Dataframe which looks as shown below I am trying to make a line plot for looking at the peaks for both columns (a,b), I have gotten as far as ...
0
votes
0answers
13 views

Does it make sense to train kalman filter on unscaled (Timeseries-)data, to clean it? [duplicate]

I have Timeseries data to clean and I got the tip to use a kalman Filter. My Question is, does it make sense to "fit" this kalman-filters parameters to an unscaled and unnormalized (...
1
vote
0answers
21 views

How to apply Kalman Filter for Cleaning Timeseries Data effectively without much optimization?

Someone gave me a tip to use kalman filter for my dataset. How time intensive is it to get a good kalman filter running, compared to simple interpolation methods like ...
0
votes
1answer
3k views

Pandas/Python - comparing two columns for matches not in the same row

I have this data: I wanted to compare A and B for matches not by row but rather search A0 if it is in column B and so on. Moreover, I wanted to ignore the .AX in column A because it would not find ...
1
vote
1answer
686 views

Pandas copy() different columns from different dataframes to a new dataframe

I have 2 dataframes that are coming from 2 different Excel files. I want to extract some columns from one file and other columns from the second file to print a new dataframe with the copied columns. ...
1
vote
0answers
48 views

cor() function giving unexpected result for numeric data frame(Updated) [closed]

I have a file named "specdata" which has a lot of CSV that each have the columns: ("Date", "sulfate", "nitrate", "ID"). I want to calculate the ...
1
vote
1answer
57 views

How to access the data of the column on which some groupby operation has been carried out? [closed]

Suppose there is a pandas dataframe which has one column consisting of names of something, and there are multiple entries respective to each entry in the first column. To count the number of entries ...
0
votes
1answer
3k views

ValueError: ('The truth value of a Series is ambiguous after applying if/else condition in Pandas data frames

I want to create a new variable for the dataframe details called lower after iterating through multiple ...
1
vote
2answers
4k views

AttributeError: 'DataFrame' object has no attribute 'ix' [closed]

I was learning Classification-based collaboration system and while running the code I faced the error AttributeError: 'DataFrame' object has no attribute 'ix'. Here is the code I write until now. <...
0
votes
1answer
21 views

How to Transpose dataframe with custom output? [closed]

I have a data like this: I want to transpose it like this in python:
1
vote
1answer
122 views

Pandas - Sum of multiple specific columns [closed]

I created this script: ...
0
votes
1answer
508 views

Unhashable type 'list' while looping through dataframe in Python [closed]

I have the following dataframe comments. I have segregated a list of users based on certain conditions. I want to get the count of words based on those users which ...

1
2 3 4 5