Skip to main content

Questions tagged [pandas]

pandas is a python library for Panel Data manipulation and analysis, e.g. multidimensional time series and cross-sectional data sets commonly found in statistics, experimental science results, econometrics, or finance.

Filter by
Sorted by
Tagged with
0 votes
0 answers
34 views

improve my f1_score for classification - pandas/sklearn

I would like advice on how to improve my f1_score for classification. I currently have something around 0.57. Dataset: lotWaferDie - lot, board and chip on which defects were measured string values ...
user162343's user avatar
0 votes
0 answers
11 views

a few records for training, and one record after training

I try to make some credit score task. I stuck in conceputal problem. There is: train_data (62 columns, 10339239 rows, 1250000 unique ID values [0 - 1249999]([min-max] ID values)) test_data (62 ...
Excentricitet's user avatar
0 votes
1 answer
37 views

how to group rows from a pandas data frame that match multiple date-time conditions

I used the code below to create the following data frame: df = pd.DataFrame({'date':[['2024-02-22 03:42:06.065'],['2024-02-22 03:47:29.152'], ['2024-02-21 19:37:05.142'], ['2024-02-21 19:40:13.851'], ...
Tamanna Mostafa's user avatar
0 votes
0 answers
23 views

Doing a join between excel tables/sheets in Python Pandas, why do all of the entries in one particular column have the same values after the join?

I have two excel docs: invasive_plants_in_and_around_park_with_compared_and_with_habitat_data_hand_edited.xlsx -with 3 sheets "Species_only_at_park", "Species_only_outside_park", ...
Henry 's user avatar
1 vote
1 answer
301 views

FutureWarning in pandas when working with Series

I'm coding a custom (but simple) dataset for training an MLP with PyTorch. Basically, my data are numeric vectors of two classes so the whole dataset is a ...
leapofFaith's user avatar
0 votes
0 answers
7 views

comparing class length of multilabelbinarizer

...
Luna Sandler's user avatar
0 votes
0 answers
25 views

How can I use two different datasets to train an ML model?

I am trying to create a machine learning model that takes in two different pandas data-frames from a basketball stats website and given multiple variables, will output a prediction of how many points ...
jshargo7's user avatar
1 vote
1 answer
59 views

When would pandas.get_dummies()'s parameter of drop_first=False be appropriate to use?

While working on some case studies that use various machine learning models, I came across a project for predicting churn in the telecom industry. The Jupyter Notebook I saw had the following lines of ...
Rakesh Poluri's user avatar
-2 votes
1 answer
40 views

How to decide your first self-project for Data Science/Analysis [closed]

Suppose a complete beginner(in the field of data analysis/data science), which is my case indeed, decided to make a project for his/her resume. These are the following skills he have learnt: Python(...
Swayam Jha's user avatar
0 votes
0 answers
15 views

Calculating Readmission Metrics in Python

I need to compute some Hospital Readmission Variables using Python. I would need to compute the following metrics: Simple Readmission: Compute variables for different periods 3, 7, 14 30 and 45 days ...
Carmen Morales's user avatar
0 votes
0 answers
25 views

How to work with tuples in pandas?

I have a dataset in a pandas dataframe where each cell contains a tuple (1,2), (3, 4)... I would like to write a function which adds the second number to a total_if_one column for each row.How does ...
Cory Blake's user avatar
0 votes
1 answer
23 views

What are these 'code' columns in this csv and what does Pandas pivot method do with it?

In a csv discussed in the book HoML (hands on machine learning), what do the values in capital letters represent? Are these some sort of code? Also, it seems the pandas.pivot() methods does something ...
Yorick Dantuma's user avatar
3 votes
1 answer
234 views

Pandas Profiling Not Working

I have been working on EDA lately and i got to know about Pandas Profiling , but i am unable to import the module Pandas_Profiling , I could import the module on Google Collab but couldnt import the ...
Sahil Rathout's user avatar
0 votes
1 answer
56 views

Use prediction after using get_dummies in pandas?

I found similar question on this topic but no answer was helpful. I had a data frame with a categorical column in it with 5 different values. I used get_dummies and used linear regression for ...
Ali.A's user avatar
  • 71
1 vote
1 answer
65 views

How to filter a dataframe and get a sub data frame based on frequency?

I have a data-frame. Lets call it df. My data frame has a column with categorical values e.g. a column named Place. But the problem is that there are many different (string) values in that column and ...
Ali.A's user avatar
  • 71
0 votes
0 answers
10 views

How to make duplicates unique in a DataFrame robustly?

I have the following table. I was able to parse it and display it as a DataFrame with pandas in python I need to change the Ref for every value that is duplicate such that it becomes unique. For ...
bem22's user avatar
  • 101
0 votes
1 answer
125 views

Python SK-Learn KNN Imputer ( "ValueError: could not convert string to float: )

I have data with missing values. All columns are integer, except for a column that has missing values. These missing values, were set with a "?" which was converted to NaN using the Numpy ...
Palu's user avatar
  • 103
0 votes
0 answers
49 views

Understanding the working of 'Interpolation'

'Interpolation' is a technique that is used to predict a null/empty value by studying its neighbouring points. Interpolation doesn't take into consideration the entire dataset when predicting the ...
Apoorva's user avatar
  • 297
0 votes
1 answer
40 views

Time Series Forecasting for Multiple Store Sales with Simultaneous Timestamps

I have a sales dataset with each store having a unique identifier. The dataset contains daily sales data for each store over a period of around two-years. I'm looking to build a time series ...
SIDDHARTH CHAVAN's user avatar
0 votes
0 answers
18 views

df['col'] refactoring

I made a big mistake by using df.iterrows() in a computationally heavy project. Now I need to convert it to np loop and need to rename all df['col'] occurrences to np[i]. How can I do it not manually, ...
Myron's user avatar
  • 123
0 votes
1 answer
33 views

Optimization problem of find a pair of values (x, y) such that they produce an output z

Problem I am trying to create a process that determines the rent and purchase price of a home that gives me a capitalization rate of 0.08 for instance. Background Normally I calculate the ...
Olek's user avatar
  • 11
0 votes
3 answers
2k views

Pandas Heatmap: could not convert string to float

please tell me, I'm trying to do a training competition on Kaggle, I want to build a heatmap based on a dataset, as one author of the guide did, but I complain that there are categorical signs, but ...
lemintare's user avatar
0 votes
1 answer
31 views

Finding the labels of a shuffled pandas dataframe

I have a dataframe (namely 'original') with over a thousand numerical columns as features and one label column. I also have a shuffled version of the same dataframe (namely 'shuffled') with label ...
asmani's user avatar
  • 101
0 votes
0 answers
7 views

how to convert these string columns so that i can put the data to the regressor

https://drive.google.com/file/d/1yQR1xJvzWd2UGgGQS2bHhdywZEE_Nu0J/view?usp=sharing I have to change the name of the columns: city, batting_team, bowling_team, batsman, non_striker, bowler. These ...
gaurav najpande's user avatar
0 votes
0 answers
116 views

How to deal with "Could not broadcast input array from shape (1141,2) into shape (1141,)" to get Out-of-Bag error while using Random Forest

I have a dataset that consists of 171 features and 39 labels. I captured both features and labels of the dataset through slicing: ...
Ralph Henry's user avatar
0 votes
0 answers
24 views

How to parse the data from the json file describing via dataframe

This is the link to the json file I want to parse the data (resolve [a sentence] into its components and describe their syntactic roles.) Form this file into a dataframe describing: keys as columns ...
Charlie Max's user avatar
0 votes
0 answers
17 views

Service Desk calls reports - Report Implementation assistance using Pandas

From the dataframe below I need to get the following information per USER using Pandas: Total Number of calls per day (the csv report contains a month of calls per user) Total number of RONA calls ...
Conde's user avatar
  • 1
0 votes
1 answer
30 views

Trying to write a code that detects a maximum value before a drop off in slope

I'm trying to write code for a force test that will output the maximum force before structural failure occurs. I'm a bit of a novice to python, so the issue here might be something simple that I'm ...
Miriam Cubstead's user avatar
1 vote
2 answers
83 views

Pandas dataframe changing 2 columns values based on another columns value

I have a df with the following values: id Count Exposure pct_affected 104 4 150,000 1.2 104 0 150,000 1.2 When I merged my 2 dataframes, the values from one ...
Ramoss's user avatar
  • 11
0 votes
1 answer
55 views

calculate the predicted value based on coefficient and constant in python

i have the coefficients and the constant (alpha). i want to multiply and add the values together like this example. (it has to be done for 300000 rows) Prediction = constant + (valOfRow1 * col1) + (-...
Mostafa Bouzari's user avatar
0 votes
1 answer
574 views

What is the use of using Vector databases over Pandas Dataframes?

My question is why can't we use a pandas dataframe to do all the data retrieval? Can someone give reasons apart from the fact that databases has the persistence of Data?
Denuwan Weerarathne's user avatar
0 votes
1 answer
78 views

How to parse month string as Pandas Period

I have imported data as a Pandas DataFrame which has string representing months of various years such as: 'Nov-23', 'Mar-24' etc. I would like to parse these strings into proper Pandas Periods. If I ...
Thomas Arildsen's user avatar
0 votes
0 answers
59 views

pandas problem in calling rolling window and lambda function

I have a pandas dataframe with the following set of columns (from the df.columns function): Index(['open', 'high', 'low', 'close'], dtype='object') I am trying to ...
coder2018's user avatar
0 votes
0 answers
11 views

clustering droplet data along trajectories

I have regular t,x,y,z-datapoints of droplets, that move continuously from top to bottom on different but similar paths. The droplets also have a volume that is approximately measured, they can appear ...
Jonas Hilti's user avatar
0 votes
1 answer
60 views

Creating well-separated bins

I want to distribute my hospital data into 4 tiers (bins) based on the score. Tier 1 (good hospitals) has the highest score and Tier 4 (not so good hospitals) has the lowest score. The tiers can be of ...
museshad's user avatar
  • 101
0 votes
0 answers
25 views

Need help in Webscraping T20 World Cup 2022 data

I have a list of links of match scorecards and i have stored the respective html content of each link in a list. I need to get the csv files I am attaching as the output. Any help would be ...
karan shah's user avatar
0 votes
0 answers
27 views

Encode multiple label categorical variables with consideration of the frequency and standardization

I'm currently working on a dataset containing, among other features, multi-labeled categorical data per person for the last 3 years. I'm not sure how to handle this kind of categorical data where ...
codade's user avatar
  • 1
0 votes
0 answers
58 views

Perform k-means clustering over multiple columns using python but without library sklearn.kmeans

I'm on a project with the topic "clustering" using the KMeans method. so I have a data set with 10 columns and 1000 rows divided into 5 clusters (in this case there are 5 robot movements). I'...
Higashida Cyza's user avatar
0 votes
0 answers
29 views

Imputation of multi-peaked dataset

I am a beginner to model fitting, and I have been working on generating a model for a CO2 emissions data set. The distribution of the data points in a number of these columns are very markedly dual-...
Phindolin's user avatar
0 votes
0 answers
66 views

time series analysis: lag features

I plan to include lag features in my multivariate time series data. I have 17 input feature and one output feature. both input and output features are time series. I will insert one more input feature ...
mehmet's user avatar
  • 1
1 vote
2 answers
860 views

Why am I getting this TypeError: quantile() got an unexpected keyword argument 'numeric_only' for pandas quantile()?

I ran into a weird error with pandas quantile() (pandas.DataFrame.quantile()) method. I have a dataframe with both numeric and non-numeric columns. I wanted to ...
user62198's user avatar
  • 1,091
0 votes
1 answer
46 views

What data am I supposed to store into the "input_data" variable?

I'm supposed to pass preprocessed input to a TFLite model to make inferences. I have an excel file containing data and a tensorflowlite model file. I received instructions on how to do this; here is ...
Feynman's user avatar
0 votes
2 answers
396 views

Genuinely stumped. I can't divide using a dataframe

I loaded some state by state covid data from: https://www.cdc.gov/nchs/pressroom/sosmap/covid19_mortality_final/COVID19.htm Then I loaded it into a data frame and try to do some basic analysis. But I ...
CognizantApe's user avatar
0 votes
0 answers
17 views

Formal terminology: metafeatures then groupby (GROUP BY) operation

In ML training and other analytics I often combine features to produce a 'metafeature' and then perform a 'groupby' (pandas) or 'GROUP BY' (SQL) query. What is the technical term for this operation? ...
M__'s user avatar
  • 206
0 votes
0 answers
29 views

How to check for correlations using ANOVA between the selected feature columns and the target feature column

Suppose I have a set of feature columns that I select to check for correlations with the target feature 'is_Star'; which is of binary values and the former consists of numeric values Why do i end up ...
meKafka's user avatar
0 votes
1 answer
21 views

Best measure to inform how predicted value can differ from real one

I have trained a regression model and obtained a pandas series of the predicted values. I am working on a "calculator" that will be able to return a predicted value after entering an input ...
Paulina's user avatar
  • 15
0 votes
0 answers
35 views

Panda's SAS PROC transpose equivalent in case of only ID, multiple VARs

I would like to transform the following: week_no count mean std min max 24 762.0 774.002734 0.995213 771.0022 776.4584 25 2971.0 759.702417 4.241902 750.9586 767.6225 ... ... ... ... ... ... To: ...
matias's user avatar
  • 1
0 votes
1 answer
34 views

How can I find what kind of args and kargs a pandas function takes?

I was wondering how do I find what are valid kargs and args for a given function? For example in pandas there is this function cumin DataFrame.cummin(axis=None, skipna=True, *args, **kwargs); how do I ...
Sam's user avatar
  • 111
0 votes
2 answers
1k views

Filter rows with multiple conditions using two dataframes

I have two dataframes that share an ID number column. I'd like to filter df1's rows based on two conditions: 1) it shares the ID with df2 and 2) it meets a condition in a column in df2. I have this ...
Larry Tate's user avatar
0 votes
1 answer
802 views

Parsing Python List values from column in pandas data frame??? Data cleaning help me please

I am new to the community and data science. I have found this hub to be of extreme value and hope to be an additive contributor one day. VS Code: Python: Pandas Question I am working with a data set ...
Justin White's user avatar

1
2 3 4 5
27