Questions tagged [pandas]

pandas is a python library for Panel Data manipulation and analysis, e.g. multidimensional time series and cross-sectional data sets commonly found in statistics, experimental science results, econometrics, or finance.

Filter by
Sorted by
Tagged with
0
votes
1answer
20 views

Python: How to plot time interval from a Dataframe in Pandas

I have the a dataframe(df) which has the data of a Job being executed at different time intervals. It includes the following details about the execution of a job: ...
0
votes
2answers
16 views

I want to re frame my dataframe to find the total number of products for each id. Do i use Pivot table for that in Pandas? [closed]

So basically I had performed GroupBy to find Retailer ID wise, the product, number of products and total Sales Amount- for which I used the following: ...
1
vote
0answers
18 views

Python: How to convert Pandas Dataframe Row Values to individual columns? [migrated]

I have the following dataframe which has the data of different jobs start and end time at different intervals. A small part of dataframe is shown below. Dataframe(df): ...
0
votes
0answers
12 views

Extracting structured data from semi structured data

I want to use machine learning and NLP to convert semi-structured data in text files to structured data by predicting the patterns in the files and splitting the fields for example if I have a text ...
1
vote
1answer
42 views

Plot Two Categorical Variables

How to plot two categorical variables in Python or using any library? I want to plot the Playing Role of a Cricketer (Batsman, Bowler, etc.) VS Bought_By (Franchise Names, e.g., CSK, DC, etc.). The ...
0
votes
0answers
18 views

What are the use-cases for heapsort option in numpy.sort()?

Could anybody share experience about use cases of the heapsort option in numpy.sort() (or pandas.DataFrame.sort_values()) in the data science field? I mean if the <...
1
vote
1answer
19 views

txt file into pandas

I have a txt file where values are randomly separated, sometimes there is a single space, and sometimes there are many. I am importing the file into pandas with <...
0
votes
1answer
18 views

IterativeImputer Evaluation

I am having a hard time evaluating my model of imputation. I used an iterative imputer model to fill in the missing values in all four columns. For the model on the iterative imputer, I am using a ...
-1
votes
1answer
19 views

Multiplying a dataframe by a larger one [closed]

I have two dataframes df1 and df2 with the same columns but not the same row number. I want to multiply them element-wise such ...
0
votes
1answer
27 views

Python Pandas KeyError: 12 [closed]

CODE ...
0
votes
0answers
13 views

Accessing group elements in groupby

I had a doubt about groupby operations. Lets suppose I have grouped my data based on one column and got 5 groups as output. I know that we can iterate over these groups and apply functions to them as ...
-2
votes
1answer
35 views

ImportError: Pandas requires version '0.3.0' or newer of 's3fs' [closed]

I'm trying to read files from S3, using boto3, pandas, anaconda, but I have the following error: ...
-1
votes
1answer
15 views

.1 after column name while read csv file from pandas

I am getting .1 after column name while read csv file from pandas
0
votes
0answers
21 views

Finding correlation between different columns

I have a dataframe in following shape: ...
1
vote
1answer
36 views

Perform PCA on columns of different length

I have about 20-30 columns, all with different lengths. The first column has 25000 rows, the second column has 19000 rows, and it's different for all the columns. All are the survey data with 0 (No),1(...
0
votes
2answers
24 views

Sort Pandas Dataframe per column

I have a dataset with age + another 14 variables. I have created 13 bins representing different age groups like so: ...
1
vote
1answer
19 views

How to Iterate over rows in a dataframe [closed]

So I've been trying to iterate over rows of my dataframe and my goal is to find matching rows based on two particular columns (say C and P). Then i need to do some manipulations as well in the data ...
0
votes
1answer
10 views

Dealing with dates in dataframe

I have a dataframe which has two date columns - Start Date and End Date, Let's suppose start date is 4/10/2019 and end date is 4/10/2020 (MM/DD/YYYY) now I want to split these dates in a list which ...
0
votes
2answers
26 views

Pandas - Create a Integer Primary Key based in string columns - python [closed]

I need to create a primary key based in string columns in my dataframe ...
0
votes
1answer
13 views

How can I replace NaN values with other values in the CustomerID column if they have a same InvoiceNo?

InvoiceNo StockCode Quantity InvoiceDate UnitPrice CustomerID Country 573415 23434 20 2011-10-30 16:17:00 0.79 13607.0 UK :-------- :--------- :--------- :------------------- :--------- :---------- :...
0
votes
1answer
20 views

Why does storing the output of .count() result in all 1's or 0's? [closed]

I have a dataframe which I run loc on to find all values within the loc parameters. Whatever the loc returns gets stored in a variable. I try to use this variable to store it in another dataframe ...
1
vote
1answer
31 views

Replace values in dataframe with another dataframes values based on condition [closed]

I have 2 dataframes with the same columns names and rows number but the elements' values are different. I need to change the value of each element in the first dataframe to 1 if its value in the the ...
0
votes
1answer
15 views

Applying a robust scaler to column 0 to 6 of my data frame?

I have a dataframe X and would like to scale columns 0 to 6 using robust scaler, how do i achieve this please? the following code only returns the scaled section of the data frame without the ...
0
votes
1answer
21 views

Why is np.where not returning '1'? Only returns '0'

This code should return a new column called orc_4 with the value of 1 if the value of the row in ...
0
votes
1answer
25 views

Access keys of pandas dataframe when using groupby

I have the following database: And I would like to know how many times a combination of BirthDate and Zipcode is repeated throughout the data table: Now, my question is: How can I access the keys of ...
1
vote
2answers
27 views

Does binning a time series with pd.qcut (using quantiles) create data leakage?

Let's say I want to predict whether a company will default on it's debt at some point in time (so binary classification) and one of the time series variables I'm using is the "revenue" of ...
0
votes
2answers
35 views

Finding the duplicate values between all columns and sort in new column with Pandas?

I have this DataFrame: ...
0
votes
1answer
35 views

Extract data from json format and paste to column using python

In my column with json data, I have this list I want to extract to column: ...
0
votes
1answer
17 views

copy data between two dataframes in python

I have two different dataframes, both containing a common element (identifier). I want to copy data from df1 for each identifier in df2 (each identifier can repeat maximum 5 times in df1) df1 ---data ...
1
vote
0answers
11 views

KNN with high-variance data [closed]

KNN doesn't work well with high-variance data, so how should I fit my data? Here is an example of what the data looks like:
0
votes
1answer
25 views

whats wrong with the graph [closed]

I wanted to plot a bar graph which shows covid cases reported in different regions in different regions of USA. So here is my code: ...
2
votes
1answer
28 views

How to fill df with data from another df - Python [closed]

I am trying to fill df with values form a different one. The first df is: The second df is: I want to add the column to first df having values from the second one to receive output: number trans ...
1
vote
0answers
22 views

How do I convert strings in a dataframe column to int or float [closed]

Encoding the column seperately works ,but when I try it on the dataset directly it throws an error. The error is something like this. ...
4
votes
1answer
93 views

Why removing rows with NA values from the majority class improves model performance

I have an imbalanced dataset like so: df['y'].value_counts(normalize=True) * 100 ...
2
votes
2answers
39 views

What to replace outliers with? (supermarket transaction data)

I have a transaction dataset from a supermarket. Let's say the average spend is $50. I want to get each customer's average spend and rank them based on where they fall based on this $50 average spend. ...
0
votes
0answers
24 views

How to assign date into pandas Dataframe?

Im trying to code correlation between exctracted stock prices and data frame with "count" and "date" rows. My dataframe (cvd) looks like this: `` ...
0
votes
0answers
56 views

ValueError: array length 13996092 does not match index length 214200 for Kaggle submission

My model is based on the model linked here on Kaggle. Specifically, the following lines of code to generate an uploadable table to make my submission: ...
1
vote
1answer
27 views

How to bin a distribution data reported with different frequencies ( salaries ), showing mixed linearity and non-linearity?

I am researching on pay-scales, and wish to receive advise to treat data of salaries. Objective My interest is to approximate the salary corresponding to different hierarchical levels in an ...
0
votes
1answer
32 views

How do I get Multiple CSV files (csv file names will be column names) from a folder to a pandas dataframe?

I think the title is enough for what is my problem here. I have 100 tickers in a folder. I got all csv files to one list. but I wanna see the tickers name which these are csv file names. what should ...
0
votes
0answers
23 views

ValueError: Could not guess the value column! Please hand it to the function as an argument

I'm using the tsfresh library to extract features on my data but I keep getting an error: ...
0
votes
0answers
15 views

How to use pivot_table on a dataframe? How to handle exceptions.UFuncTypeError?

I tried to use the pivot_table function on a dataframe. This was working at the beginning now I get all the time errors, and don't really get why. ...
0
votes
1answer
18 views

Is it ok to fill a pandas dataframe with NaN values?

Is it correct to fill a pandas dataframe with NaN values? In specific: if I have a dataframe with a user name and his age is it ok to fill the age column with int and NaN values. ...
0
votes
2answers
73 views

Error when trying .transform for OrdinalEncoder from Scikit Learn

I'm having a lot of issues using scikit learn recently and was hoping someone could help me with my problem. I can use other methods to ordinal encode but i want to figure this one out. ...
0
votes
2answers
34 views
0
votes
0answers
12 views

Forecasting Weekly Average Usage

In python/pandas what would you suggest as a couple of good basic methodologies to use to forecast weekly average usage of user activity on a given platform?
1
vote
1answer
33 views

Replace Missing Values with Most Frequent number under Condition

I'm trying to replace missing values of column "Age" but under condition of other columns on this data Titanic - Machine Learning from Disaster ...
0
votes
0answers
10 views

Rating Sports Teams: How to perform least squares minimization with a constraint?

I am trying to rate NFL teams by minimizing the sum of squared errors subject to a constraint. My data looks like: ...
1
vote
1answer
357 views

AttributeError: 'numpy.ndarray' object has no attribute 'nan_to_num'

I'm trying to run a Random Forest model from sklearn but I keep getting an error: ValueError: Input contains NaN, infinity or a value too large for dtype('float32')....
0
votes
0answers
13 views

How to find anomalies in (almost) constant stream of data?

I have a process that (simply put), starts every 5 minutes, collects data, and put that data into the database. More detailed explanation would be that process starts, collects data (which takes some ...

1
2 3 4 5
21