Questions tagged [pandas]

pandas is a python library for Panel Data manipulation and analysis, e.g. multidimensional time series and cross-sectional data sets commonly found in statistics, experimental science results, econometrics, or finance.

Filter by
Sorted by
Tagged with
0 votes
0 answers
5 views

I want to parse the data from the employee.json file

https://drive.google.com/file/d/1tqlKIm5leTS-ALj8w8fYtCrOqs62bNXL/view?usp=drive_link This is the link to the json file i want to parse the data(resolve (a sentence) into its component parts and ...
Charlie Max's user avatar
0 votes
0 answers
16 views

Service Desk calls reports - Report Implementation assistance using Pandas

From the dataframe below I need to get the following information per USER using Pandas: Total Number of calls per day (the csv report contains a month of calls per user) Total number of RONA calls ...
Conde's user avatar
  • 1
0 votes
1 answer
17 views

Trying to write a code that detects a maximum value before a drop off in slope

I'm trying to write code for a force test that will output the maximum force before structural failure occurs. I'm a bit of a novice to python, so the issue here might be something simple that I'm ...
Miriam Cubstead's user avatar
1 vote
2 answers
31 views

Pandas dataframe changing 2 columns values based on another columns value

I have a DF w' the following values: id Count Exposure pct_affected 104 4 150,000 1.2 104 0 150,000 1.2 WHen I merged my 2 dataframes, the values from one carried onto the other. ...
Ramoss's user avatar
  • 11
0 votes
1 answer
16 views

calculate the predicted value based on coefficient and constant in python

i have the coefficients and the constant (alpha). i want to multiply and add the values together like this example. (it has to be done for 300000 rows) Prediction = constant + (valOfRow1 * col1) + (-...
Mostafa Bouzari's user avatar
0 votes
1 answer
44 views

What is the use of using Vector databases over Pandas Dataframes?

My question is why can't we use a pandas dataframe to do all the data retrieval? Can someone give reasons apart from the fact that databases has the persistence of Data?
Denuwan Weerarathne's user avatar
0 votes
1 answer
16 views

How to parse month string as Pandas Period

I have imported data as a Pandas DataFrame which has string representing months of various years such as: 'Nov-23', 'Mar-24' etc. I would like to parse these strings into proper Pandas Periods. If I ...
Thomas Arildsen's user avatar
0 votes
0 answers
15 views

pandas problem in calling rolling window and lambda function

I have a pandas dataframe with the following set of columns (from the df.columns function): Index(['open', 'high', 'low', 'close'], dtype='object') I am trying to ...
coder2018's user avatar
0 votes
0 answers
10 views

clustering droplet data along trajectories

I have regular t,x,y,z-datapoints of droplets, that move continuously from top to bottom on different but similar paths. The droplets also have a volume that is approximately measured, they can appear ...
Jonas Hilti's user avatar
0 votes
1 answer
35 views

Creating well-separated bins

I want to distribute my hospital data into 4 tiers (bins) based on the score. Tier 1 (good hospitals) has the highest score and Tier 4 (not so good hospitals) has the lowest score. The tiers can be of ...
museshad's user avatar
  • 101
0 votes
0 answers
10 views

Need help in Webscraping T20 World Cup 2022 data

I have a list of links of match scorecards and i have stored the respective html content of each link in a list. I need to get the csv files I am attaching as the output. Any help would be ...
karan shah's user avatar
0 votes
0 answers
9 views

Encode multiple label categorical variables with consideration of the frequency and standardization

I'm currently working on a dataset containing, among other features, multi-labeled categorical data per person for the last 3 years. I'm not sure how to handle this kind of categorical data where ...
codade's user avatar
  • 1
0 votes
0 answers
28 views

Perform k-means clustering over multiple columns using python but without library sklearn.kmeans

I'm on a project with the topic "clustering" using the KMeans method. so I have a data set with 10 columns and 1000 rows divided into 5 clusters (in this case there are 5 robot movements). I'...
Higashida Cyza's user avatar
0 votes
0 answers
28 views

Imputation of multi-peaked dataset

I am a beginner to model fitting, and I have been working on generating a model for a CO2 emissions data set. The distribution of the data points in a number of these columns are very markedly dual-...
Phindolin's user avatar
0 votes
0 answers
24 views

time series analysis: lag features

I plan to include lag features in my multivariate time series data. I have 17 input feature and one output feature. both input and output features are time series. I will insert one more input feature ...
mehmet's user avatar
  • 1
1 vote
2 answers
138 views

Why am I getting this TypeError: quantile() got an unexpected keyword argument 'numeric_only' for pandas quantile()?

I ran into a weird error with pandas quantile() (pandas.DataFrame.quantile()) method. I have a dataframe with both numeric and non-numeric columns. I wanted to ...
user62198's user avatar
  • 1,091
0 votes
1 answer
42 views

What data am I supposed to store into the "input_data" variable?

I'm supposed to pass preprocessed input to a TFLite model to make inferences. I have an excel file containing data and a tensorflowlite model file. I received instructions on how to do this; here is ...
Feynman's user avatar
0 votes
2 answers
47 views

Genuinely stumped. I can't divide using a dataframe

I loaded some state by state covid data from: https://www.cdc.gov/nchs/pressroom/sosmap/covid19_mortality_final/COVID19.htm Then I loaded it into a data frame and try to do some basic analysis. But I ...
CognizantApe's user avatar
0 votes
0 answers
17 views

Formal terminology: metafeatures then groupby (GROUP BY) operation

In ML training and other analytics I often combine features to produce a 'metafeature' and then perform a 'groupby' (pandas) or 'GROUP BY' (SQL) query. What is the technical term for this operation? ...
M__'s user avatar
  • 208
0 votes
0 answers
21 views

How to check for correlations using ANOVA between the selected feature columns and the target feature column

Suppose I have a set of feature columns that I select to check for correlations with the target feature 'is_Star'; which is of binary values and the former consists of numeric values Why do i end up ...
meKafka's user avatar
0 votes
1 answer
17 views

Best measure to inform how predicted value can differ from real one

I have trained a regression model and obtained a pandas series of the predicted values. I am working on a "calculator" that will be able to return a predicted value after entering an input ...
Paulina's user avatar
0 votes
0 answers
27 views

Panda's SAS PROC transpose equivalent in case of only ID, multiple VARs

I would like to transform the following: week_no count mean std min max 24 762.0 774.002734 0.995213 771.0022 776.4584 25 2971.0 759.702417 4.241902 750.9586 767.6225 ... ... ... ... ... ... To: ...
matias's user avatar
  • 1
0 votes
1 answer
22 views

How can I find what kind of args and kargs a pandas function takes?

I was wondering how do I find what are valid kargs and args for a given function? For example in pandas there is this function cumin DataFrame.cummin(axis=None, skipna=True, *args, **kwargs); how do I ...
Sam's user avatar
  • 111
0 votes
2 answers
66 views

Filter rows with multiple conditions using two dataframes

I have two dataframes that share an ID number column. I'd like to filter df1's rows based on two conditions: 1) it shares the ID with df2 and 2) it meets a condition in a column in df2. I have this ...
Larry Tate's user avatar
0 votes
1 answer
116 views

Parsing Python List values from column in pandas data frame??? Data cleaning help me please

I am new to the community and data science. I have found this hub to be of extreme value and hope to be an additive contributor one day. VS Code: Python: Pandas Question I am working with a data set ...
Justin White's user avatar
1 vote
1 answer
29 views

How to proceed when correlation plot doesn't show as much multicorrelation as is seen by statsmodels variance_inflation_factor in a regression task?

I am working with the kaggle Blueberry Yield prediction dataset. There are 17 columns including the target variable. Below is the correlation heat map: It can be seen that multiple features are ...
forest's user avatar
  • 111
1 vote
1 answer
31 views

How to prepare data if each item has multiple categories (like tags)

I'm working on a recommender system that will recommend movies to users. Movie ratings Movie User Rating 100 201 5 105 256 8 ... ... ... Movie tags Movie Tag 100 1 100 2 100 8 105 2 105 5 ....
Silver Light's user avatar
0 votes
2 answers
21 views

How to filter out redundant id rows with deviant values

First post, so thank you in advance for any pointers! I have a dataframe with an issue similar to the following: id value 10 25.03 11 26.00 11 0.95 12 32.95 13 34.43 13 1.02 One can see ...
OldDogNewTrick's user avatar
0 votes
0 answers
24 views

what is the best practice to handle carriage returns when exporting data from pandas dataframe to CSV?

While trying to export data from a pandas dataframe to a CSV, we noticed that carriage returns (inserted at the middle of a line or at the end of a line) produce outputs that are different from ...
Daniel S.'s user avatar
  • 101
0 votes
2 answers
72 views

How to create a plot of specific row with every column using python and which package to choose matplotlib or seaborn?

I'm trying to create a plot for the ranking of each country from 2002 to 2023. I created this dataset by loading each csv file from the respective years, which contains the Countries and Ranking ...
SAUBHIK KUMAR's user avatar
2 votes
2 answers
69 views

Why does pandas.dataframe.info() not update when I delete the first row of a dataset?

I am working with a dataset that comes in with nonsense field names in the first row, with the actual field names in the second row. Currently I'm using this script: ...
SchrodingersStat's user avatar
0 votes
1 answer
33 views

What is the minimum ratio to consider the data set as balanced for the classification algorithm?

Can I consider 20% / 80% as a balanced dataset? My target variable ratio is 62% and 37%. I hope It is a balanced dataset. Please let me know if I am wrong. However, I would like to know, what is the ...
Iynga Iyngaran Iyathurai's user avatar
0 votes
1 answer
455 views

Pandas: To find duplicate columns

I have a dataframe with different dtypes like int, float, object, datatime etc. I am performing data cleaning, to list or find ...
winter's user avatar
  • 115
0 votes
2 answers
469 views

corr() is giving an error. please help out of this problem and tell me what is this error about

when I am trying to run sns.heatmap(df.corr(),annot=True) this code in my jupyter notebook. this error is occuring. I cannot understand this problem. please help me.
Subhajit Sarkar's user avatar
4 votes
1 answer
75 views

Working with time series data with several times stamps on a dates, and implementing machine learning

I'm trying to implement predictive analytics on a production data. my goal is to predict next downtime, it's reason and issues. My data looks like below; ...
def __init__'s user avatar
0 votes
1 answer
25 views

'Merge' two identically sized pandas dataframes to one equal sized dataframe [closed]

If I have a dataframe e.g. | A | B --------- | 1 | 21 --------- | 3 | -1 and | C | D --------- | 15 | 2 --------- | 7 | 9 ...
Socorro's user avatar
  • 91
0 votes
1 answer
26 views

how to prepare pandas dataframe from Solar Flare Data Set?

I am looking to download and use Solar Flare Data Set (https://archive.ics.uci.edu/ml/datasets/Solar+Flare) but unable to figure out how to get the pandas dataframe from the file that is being ...
Sachin's user avatar
  • 101
0 votes
0 answers
5 views

How to identify best fit sub groups in time series data

I am working with a set of time series user activity data where i'm trying to figure out if the user performed an action daily for at least X days in Y. My use case is 15 days in 30, doesn't have to ...
Lookaflyingdonkey's user avatar
0 votes
0 answers
13 views

I am creating a new column, income to debt ratio, but output is coming in integer, instead of float

I need the income to debt ratio in float, when i tried manually it is coming infloat, but when i do in pandas, it is coming as integer. how can i take output in float numbers.
vinaysirige's user avatar
0 votes
3 answers
55 views

Fill NaN values with values from another column

I have data that look like this: ...
Kareit's user avatar
  • 3
1 vote
1 answer
152 views

Filling na values with condition from other column [closed]

I am working on famous titanic dataset and I want to replace na values in Age column but on such a condition that these people whose Pclass=3 receive 25, Pclass=2 29 and Pclass=1 38. I was trying to ...
H0t_blue_B0i's user avatar
0 votes
1 answer
51 views

Understanding correlation - Machine Learning

I am experimenting a project on identifying cancer or not - Binary classification The dataset has many columns. Here, I added correlation values between few input columns and the target column[cancer/...
Vishak Raj's user avatar
0 votes
0 answers
176 views

How to filter rows using custom function in pyarrow

I've a parquet dataset that contains latitude and longitude values as separate columns. And I want to filter those rows that are inside a polygon, I'm able to do this in pandas dataframe but unable to ...
cicada_'s user avatar
  • 101
0 votes
0 answers
19 views

Running all cells again adding new rows instead of return new dataframe

i am creating a class just like lazypredict library but with k fold support and return a data frame withh the mean score of every model every thing works and i can ...
Burhan Khanzada's user avatar
0 votes
1 answer
20 views

Force network to weigh specific variables during learning

I have a pandas data frame containing around 100000 observations of plant species and their age with additional numerical predictors (climate). I used tensorflow ...
kjtheron's user avatar
0 votes
0 answers
15 views

What is the best way to whittle down rendundant categorical data?

I'm trying to build a linear regression model in Tensorflow (and preprocessing with pandas) that will help me categorize bank transactions. I'm trying to whittle down the vendor parameter, because the ...
Ethan Leyden's user avatar
0 votes
0 answers
16 views

Getting probability to finish in the last 3 ones after each game week

I'am working on a dataframe with five differents features : team game_week season cum_points : pre game cumulative number of points final_position Those datas are covering 10 seasons of Premier ...
Thomas H.'s user avatar
0 votes
0 answers
23 views

passing pandas dataframe to function, but it appears as numpy array

In the function I call sklearn train_test_split In Pycharm debugger everything looks ok until it gets to that call. Then it backtracks and tells me ...
mLstudent33's user avatar
0 votes
1 answer
27 views

Counting values from diffrent columns

I'm working with a pandas dataframe that has some columns like these: ...
Arwa Ali's user avatar
0 votes
0 answers
22 views

Which statistical test is suitable for analyze the correlation between non-independent categorical variables?

Im working on an EDA on chocolate bar ratings, trying to indentify relationships between variables. In this case, im trying to test if the ingredients of a chocolate bar area associated with its ...
Sebastian Rodriguez's user avatar

1
2 3 4 5
27